Esco Logics

Most Web Traffic Is No Longer Human. Some of It Is About to Buy

Marketing

In March 2026, Cloudflare's CEO Matthew Prince told an audience at SXSW that bots would overtake humans on the web by 2027. He was wrong, but not in the direction you would expect. On the company's Q2 earnings call in August, he reported that for the first time, more than 50% of the traffic across Cloudflare's network was not human. That is a network-wide figure, not a number for any one site, but Cloudflare sits in front of roughly one in five websites, so it is a fair picture of the web your site lives on.

A large share of the machines hitting it are not all the same. Some are shopping on behalf of a customer. Some are copying your content and sending nobody back. Some are trying to break into your customers' accounts. This post is about telling them apart.

Why It Happened So Fast

Before generative AI, bots were about 20% of web traffic, and most of that was Google's crawler, according to Prince. What changed is how AI agents work. His example at SXSW was a person shopping for a digital camera. A human might visit five websites. An agent doing the same job, he said, will often visit a thousand times as many, so around 5,000 sites, because it is checking everything rather than picking a few.

The growth curve bears that out. HUMAN Security's 2026 State of AI Traffic report found automated traffic grew 23.5% in 2025 while human traffic grew 3.1%. Monthly AI-driven traffic grew 187% between January and December, and traffic from AI agents and agentic browsers grew 7,851%. That last number started from almost nothing, but it is the category that is now reading product pages and filling in checkout forms. One detail worth holding onto: HUMAN found more than 95% of that AI-driven traffic was concentrated in retail, media and travel. If you are not in one of those, it is coming, but it has not arrived in force yet.

None of this is a reason to panic. It is a reason to stop treating "traffic" as one number. There are three kinds of machine on your site, and they need three different responses.

Group 1: Bots That Bring Buyers

This is the group most owners have not noticed yet. When someone asks ChatGPT or a similar assistant which plumber to call or which laptop to buy, the assistant reads your site and either recommends you or does not. If it does, the person who arrives has already been pre-qualified.

Adobe Analytics tracks over a trillion visits to U.S. retail sites. Its Q1 2026 data showed AI-referred traffic up 393% year over year. By March 2026, that traffic was converting 42% better than other channels, with 37% higher revenue per visit. A year earlier it had converted 38% worse. The swing happened in twelve months. Adobe also found that large parts of U.S. retail sites still cannot be read properly by AI, which is the gap the next section is about.

One scope warning: that is U.S. retail. If you run a law firm or a manufacturing company, treat it as a signal that the same shift is coming, not as proof that it has arrived in your industry.

What does it mean? 

An AI assistant cannot recommend what it cannot read. If your prices sit behind a form, your service pages are vague, or your site is slow, you are invisible to the fastest-growing source of ready-to-buy visitors.

Solution:

Make the site easy for a machine to understand. That means structured data markup, clear pricing and service pages, and fast load times. It is the same web development work that helps Google, just with a new and less forgiving reader.

Group 2: Bots That Take and Give Nothing Back

The second group reads your site to train AI models or to answer questions without ever sending the person to you. Cloudflare Radar publishes a crawl-to-referral ratio for each AI company: how many pages its bots fetch for every visitor it sends back. For October 2026, Perplexity sat at about 4,056 pages crawled per visitor, OpenAI at about 775, Anthropic at about 482, and Google at about 5. Cloudflare says the figures may be overstated, and the order changes month to month, but the shape is clear. Google still sends people. Most AI companies mostly do not.

If you publish content, this is the group eating your server costs. TollBit's State of the Bots report, which looked only at publishers (3,906 of them), found European sites saw four times the median AI scrapes of North American ones, received one human visit from AI apps for every 179 bot visits, and had their robots.txt rules ignored nearly three times as often. Digiday notes that Cloudflare and DataDome data did not show the same gap, so treat the Europe finding as contested.

The law is catching up slowly. On November 3, 2026, an adviser to the EU's top court said Google's Gemini may infringe publishers' rights when it reuses their articles in answers. The opinion is non-binding and the ruling is due in the coming months, but it is the first time the question has been put squarely.

What does it mean? 

Every AI crawler you allow costs you bandwidth. Some of them will recommend you to customers in return. Some will not.

Solution: 

Decide which AI crawlers you allow in robots.txt. This is a business decision, not a technical one. If you sell services and want to be recommended, allowing the assistants that send traffic makes sense. If you are a publisher whose content is the product, blocking the training crawlers is reasonable. Either way, know that a robots.txt rule is a request, not a lock.

Group 3: Bots That Attack

The third group is not reading anything. It is trying passwords. HUMAN Security's customers, who are mostly large companies, faced an average of more than 400,000 attempted account takeover attacks after login in 2025, more than four times the 2024 level. Your numbers will be smaller. The method will be the same. These attacks are automated, they run around the clock, and they do not care whether you are a bank or a bakery with a customer login.

What does it mean? 

If your site has a login, a checkout, or a contact form, it is already being probed. The question is whether anything is stopping it.

Solution: 

 Put multi-factor authentication on customer and admin accounts. Rate-limit login attempts. Add bot protection to forms so the contact page is not a spam pipe. If you are not sure whether any of that is in place, that is itself the answer.

The Hidden Problem

Here is the part that trips up most owners. Google Analytics and similar tools filter known bots, but a growing share of automated traffic does not identify itself. It looks like a visitor. It inflates your pageviews, flattens your conversion rate, and makes last month's traffic spike look like a marketing win when it was a scraper.

GA4 removes known bots automatically and does not show you how much it removed, so there is no setting to check. The only way to know what is slipping through is to compare a month of raw server logs against what analytics reports. If the gap is large, your numbers are partly fiction, and a marketing decision made on that data is a decision made on noise.

A Short Checklist

Let the Good Bots In

Add structured data to your service and product pages, publish clear pricing, and get page load times down. If an AI assistant can read it, it can recommend it.

Decide on AI Crawlers in robots.txt

List which crawlers you allow and which you block, and write the reason down. Revisit it every quarter, because the companies and their referral behavior change. Treat it as a policy, not a setting.

Protect Logins and Forms

Multi-factor authentication on every account that matters. Rate limits on login pages. Bot protection on contact and checkout forms. None of this is expensive. All of it is cheaper than a breach.

Check Your Analytics Filter

Pull a month of server logs and compare them against what analytics shows. GA4 filters known bots on its own, but it cannot see the ones that look human. If the gap is large, your dashboards need fixing before your marketing plan does.


Not sure what is really visiting your site? The team at Esco Logics will check your logs, tell you which bots are buyers, which are freeloaders and which are attackers, and what to do about each. It takes a few days and it usually changes how people read their own numbers.

Shariq Khalid

Written by

Shariq Khalid

Founder

Shariq Khalid founded Esco Logics in 2015 and brings more than two decades of experience in IT sales, project management and consulting. He leads teams in three countries that build AI-powered solutions, custom software and e-commerce platforms for clients. He shares insights and lessons about what actually makes technology projects succeed, and what quietly makes them fail.

Table Of Contents


  • 1.Why It Happened So Fast
  • 2.Group 1: Bots That Bring Buyers
  • 3.Group 2: Bots That Take and Give Nothing Back
  • 4.Group 3: Bots That Attack
  • 5.The Hidden Problem
  • 6.A Short Checklist
  • 7.Let the Good Bots In
  • 8.Decide on AI Crawlers in robots.txt
  • 9.Protect Logins and Forms
  • 10.Check Your Analytics Filter
Request a Quote


FAQs

Frequently Asked Questions

Got any Questions?

Let us know! Reach out and our team will get right back to you.

Contact Us
  • Look at your server logs rather than your analytics dashboard. Bots show up as visits with identical timing, odd user-agent strings, or hundreds of pages read in seconds. Most analytics tools miss a large share of them. Esco Logics runs this comparison as the first step of any traffic review, and the gap is usually bigger than owners expect.
  • It depends on what your site is for. If you sell services or products, blocking the assistants that recommend businesses means losing a growing source of buyers. If your content is the product, blocking training crawlers protects it. Block by crawler, not all at once, and remember that robots.txt is a request some bots ignore.
  • In U.S. retail, yes. Adobe's data shows AI-referred visitors converting 42% better than other channels by March 2026. Outside retail the data is thinner, but the pattern is the same: people who arrive from an AI recommendation have already compared options and are closer to buying.
  • Because the attacks are automated and cheap. A bot trying stolen passwords does not pick targets by size. It tries everything with a login page. Small sites are hit because they are often the least protected.
  • Structured data that labels your products, services, prices and locations; plain pages that say what you do and what it costs; and fast load times. Esco Logics audits sites for this and typically finds the pricing and service pages are where most of the gaps sit.
  • Partly. GA4 excludes bots on a known list automatically, and you cannot turn that off or see how much it is removed. It misses many AI agents and anything that mimics a human. The check is to compare a month of raw server logs against your analytics numbers. If you want someone else to do that, Esco Logics can run it in a few days.
Discover Technology You’ll Love to Use

Let’s build tomorrow
together

Technology should be simple, powerful, and future‑ready. That’s what we create at Esco Logics.

Book a Free Consultation
  • logo
  • logo
  • logo
  • logo