A Word That Forgot What It Meant
Ask ten people what a "bot" is and you'll get ten answers: a spam account, a chatbot, the thing that buys all the concert tickets, a search crawler, a Twitter troll, a customer-service menu that won't let you reach a human. They're all right, which is another way of saying the word has gone soft. At its core a bot is just a program that does a repetitive task automatically, without a person driving each step. That's it. The drama comes from what task, and who benefits.
The name is older than the internet. It's clipped from robot, which comes from the Czech word robota — meaning forced labor or drudgery. It entered the language through a 1920 play, Karel Čapek's R.U.R., about artificial workers who eventually stop taking orders. A century later we shortened the word and handed the drudgery to software. The lineage is fitting: a bot is a tireless worker that never sleeps, never gets bored, and does exactly what it's told — for better or worse.
The Three Buckets
For years the honest split was just two: good bots and bad bots. The dividing line isn't the technology — it's permission and intent. Roughly:
- Good bots announce themselves, respect the rules, and do work the site owner generally wants. Search crawlers (Googlebot, Bingbot) that index pages so people can find them. Uptime monitors that ping a server to check it's alive. Price-comparison and feed fetchers. The AI crawlers that read the web to train and answer.
- Bad bots hide what they are and take what they weren't given. Scrapers that strip-mine content, credential-stuffers that test stolen passwords by the million, scalpers that hoover up sneakers and tickets, spam and fake-engagement bots, and the brute-force swarms behind denial-of-service attacks.
- Agents — the new third category. As of 2025, the security industry stopped lumping AI agents in with the other two and started counting them separately, because they don't behave like either. More on that next.
Is an "Agent" Just a Bot With a Brain?
Short answer: more or less, yes — and the words are cousins. "Agent" comes from the Latin agere, "to do" or "to act," so an agent is literally "one who acts on your behalf." A bot is a little worker that follows a fixed script. An agent is a bot that can make decisions toward a goal instead of just repeating steps — it can read a page, decide the next move, click a button, change its plan when something fails. A classic bot fills the same form a thousand times. An agent is told "book me the cheapest flight" and figures out the rest. The lines blur, but the rule of thumb holds: a bot repeats, an agent decides. The reason traffic analysts now track them apart is that agents look human in ways old bots never did, which makes them far harder to tell from a real visitor.
Half the Internet Stopped Being Human
This is the fact that stops people cold. According to the 2026 Thales Bad Bot Report — the thirteenth annual study of automated traffic — bots made up 53% of all web traffic in 2025. That breaks down to about 40% bad bots and 13% good ones, meaning humans are now the minority online for the second year running. The security network behind that report blocked 17.2 trillion bot requests in a single year. AI-enabled attacks jumped more than twelvefold, with the daily average of blocked attacks climbing from roughly 2 million to 25 million. None of this is slowing down.
Who Builds Them, and Why It's Rarely Personal
Bots are built by basically everyone. Search engines and AI labs build the big crawlers. Retailers and banks run monitoring and pricing bots. Marketers run engagement bots. Criminals and fraud rings run the malicious fleets. And increasingly, anyone can build one — generative AI has knocked the floor out from under bot-making. A task that once needed a coder now takes a few plain-English prompts, which is a big reason the simple-bot volume exploded. The motive is almost never about you specifically. It's economics: a bot that costs pennies to run can test a million stolen passwords, scrape a competitor's entire catalog, or fake enough clicks to drain an ad budget. When the upside is automated and the cost is near zero, scale does the rest.
The Map Is a Lie
People love to ask which country the bots come from, and the honest answer is that the map mostly lies to you. By raw origin, the United States "sends" the largest share — around 40% of global bot traffic in 2025 — but that's not American hackers. It's because Amazon Web Services and Google Cloud live there, and roughly a quarter of all bot traffic is just cloud servers doing their thing. Renting a server in Virginia is cheap, fast, and close to everything.
The real origins are deliberately hidden. Serious operators route through residential proxy networks — pools of real home IP addresses, often borrowed from ordinary people's devices — so their traffic looks like a grandmother in Ohio rather than a data center. That's why blocking "by country" barely works. When researchers do trace deployment, the recurring names are the ones you'd guess: the US, China, Russia, Vietnam, and the UK show up repeatedly. The motives sort cleanly by who's behind them — commercial scraping and fraud where there's money, and state-linked surveillance and influence operations where there's geopolitics. China hosts enormous scraping and AI operations; Russia is associated with disinformation and influence bots; and the big powers all run intelligence-gathering swarms while complaining about being targets of the same.
Following the Money — and What's Worth More
The cash motives are obvious enough. Scalper bots buy scarce goods at retail and flip them at a markup. Ad-fraud bots fake the clicks and views that advertisers pay for, skimming from a multi-billion-dollar pool. Credential-stuffing bots turn leaked passwords into drained accounts and stolen loyalty points. Scraper bots lift pricing and product data to undercut a rival.
But a lot of the most valuable bot work isn't paid in dollars at all. The currencies are data, attention, influence, and intelligence. AI crawlers harvest the open web to build models — training data is the asset. Engagement bots manufacture the appearance of consensus, inflating a follower count or burying a hashtag, because perception moves markets and elections. State bots gather and shape information as a strategic resource, not a sellable one. In each case the bot isn't there to make a sale. It's there to tilt what people see, believe, or know — which is often worth more than money to whoever's running it.
Why Nobody Actually Wants to Kill Them All
Here's the catch that makes "just block the bots" naive: the internet you use runs on them. Search literally cannot function without crawlers cataloging the web. Wikipedia, weather, flight prices, stock tickers, fraud detection on your credit card, the system that texts you a code at login — all bots, all working in the background. Strip them out and most of the modern web stops.
Governments need them too, and not only the spy kind. They run crawlers to archive public records, monitor for child-exploitation material, track disease outbreaks and disinformation, enforce sanctions, and watch their own critical infrastructure for attacks. The same automation that lets a criminal scan a million sites for a weakness lets a defender scan a million sites for the same weakness first. Bots aren't good or evil any more than a crowbar is. The fight is never about banning them — it's about telling the welcome ones from the rest, fast, at a scale no human could.
The Fight: Your Toolbox vs. Cloudflare's Maze
What you can do as an individual site operator is real but modest. You can post a robots.txt file asking bots to stay out of certain areas — though it's an honor system, and the badly behaved ones ignore it. You can rate-limit, block obvious bad IPs, add a CAPTCHA, and watch your logs for the tells: sessions that last zero seconds, impossible request rates, traffic spiking from one network at 3 a.m. It's a losing arms race to fight alone, because a determined operator just rotates proxies and tries again.
That's why most of the actual war is fought by infrastructure giants — above all Cloudflare, which sits in front of roughly a fifth of the web and sees patterns no single site ever could. Its defenses have gotten clever. There's one-click bot blocking, and as of mid-2025 it began blocking AI crawlers by default on new sites, flipping the web from opt-out to opt-in. It launched Pay Per Crawl, a marketplace letting publishers charge AI companies for access instead of just allowing or denying it. And the most fun one: AI Labyrinth, which doesn't block misbehaving crawlers at all — it lures them into an endless maze of convincing, AI-generated decoy pages full of harmless science facts, wasting their time and compute while quietly fingerprinting them. As Cloudflare put it, no real human clicks four links deep into a maze of nonsense, so anything that does is a bot, caught.
Fast Facts Most People Don't Know
- The US is the top bot "source" and the top bot target at the same time — origin and destination are different questions, and the same country can lead both.
- AI crawlers take far more than they give back. In Cloudflare's 2025 numbers, the leading AI crawlers requested enormous volumes of pages while sending back almost no referral traffic — a lopsided exchange publishers increasingly resent.
- "Robot" was coined by a playwright's brother. Karel Čapek wanted to use a different word; his brother Josef suggested robot. It stuck for a century.
- Residential proxies often run on borrowed devices. The "real home IP" disguising a bot may belong to a person who installed a free app and never read what it does in the background.
- Blocking a bot can backfire. A hard block tells the operator they've been spotted, so they adapt — which is exactly why Cloudflare's maze quietly misleads them instead.
- By some forecasts, bad-bot traffic alone could exceed all human traffic by the end of 2026. Not all bots — just the malicious ones.