Google’s AI is getting a free ride on the web
Live Events as a Reliable and Trusted News Source Addas a Reliable and Trusted News Source Add Now! Google became the king of internet search
Live Events as a Reliable and Trusted News Source Addas a Reliable and Trusted News Source Add Now! Google became the king of internet search and created a free cornucopia of knowledge by striking a clever bargain with anyone who made a website: Let the company’s crawler index your pages and Google would send human readers back. That trade looks very different today thanks to Google’s strategy of using its search infrastructure to strengthen its artificial-intelligence products.The technique, which has largely flown under the radar, gives the company an unfair advantage over other AI firms while also accelerating the rise of automated traffic on the internet at the expense of humans. Google is now meeting some resistance from a plucky British regulator and a crusading industry player, which may finally bring some much-needed change.Googlebot, the firm’s automated crawler, has for many years scoured the Internet and downloaded web pages to feed its massive index so that the freshest and most relevant pages could appear on Google’s search results. More recently, though, the line between content collected for search and content used to generate AI answers has become blurred, as the bot is used to help feed both search and Google’s flagship AI model, Gemini.Google’s web crawler sees 3.2 times more of the web than OpenAI’s version and 4.8 times more than Microsoft Corp.’s, according to Matthew Prince, chief executive officer of web security company Cloudflare.
Prince’s firm is a traffic gatekeeper for about a fifth of the web, handling traffic and blocking attacks from bad actors, and he’s become a vocal critic of Google for bundling its search and scraping systems. He says the strategy makes it difficult for publishers to block AI training without losing the search traffic they rely on.That advantage also has broader consequences. If AI increasingly reads, summarizes and rewrites the web without sending readers back to the pages created by humans, the economic incentive to produce original information starts to fade.AI agents generated more than 57% of web traffic in 2026 versus about 42% for humans, according to Cloudflare, marking the first time in history that machine activity surpassed humans on the Internet. Prince sees that divergence getting more extreme in the next few years, a trend that echoes the so-called dead internet theory, the idea that machines increasingly create and consume online content. “Humans will be a rounding error on the internet versus what their agents and their bots are doing,” he tells me.Agents are still largely the preserve of software developers and companies; but Mark Zuckerberg said last week that Meta Platforms Inc. would soon launch a consumer version for Facebook Messenger, WhatsApp and Instagram “that just works out of the box and is easy enough for billions of people” to use.There’s nothing wrong with bots doing things that are useful, but the internet will become vapid, less original and ultimately less useful if human contribution is no longer rewarded and peters out.
The BriefWire