It reads your HTML, ignores your design, and tries to complete a task. It does not see your hero animation or your brand colours. It sees text, links, and structure. If the thing it needs is only expressed visually or only appears after a script runs, the agent fails and moves to a competitor.
This is no longer a thought experiment. A large share of the traffic hitting your site today is already software, and some of it is acting on behalf of a person who asked a question.
So the question is worth asking plainly. If an agent came to your site to book, buy, or compare something on a customer''s behalf, could it?
A crawler collects pages. An agent tries to accomplish something. A crawler visits, stores what it finds, and leaves. An agent arrives with an instruction from a user, reads what it needs, and then acts, which might mean filling a form, comparing prices, or starting a checkout.
That difference changes what a site owner should care about. For crawlers, the question has always been whether your content can be read and indexed. For agents, the question is whether your site can be operated without eyes and without hands.
Most sites answer yes to the first and no to the second, and nobody has noticed because until recently nothing tried.
More than half of it, across the internet as a whole. Cloudflare reported in July 2026 that more than 50 percent of traffic on the internet is now non-human, crossing that threshold for the first time. That is not a projection about the future. That is Cloudflare describing what its network saw.
The composition is shifting fast too. Cloudflare found that 52 percent of crawler requests were for AI training as of June 2026, up from 22 percent in spring 2025. Mixed-use crawlers, which blend search, agent activity, and training in one, made up over 36 percent of activity.
The part that should worry publishers is what Cloudflare said about search. Pure search crawling now represents a small and declining share of overall crawler activity, even though it remains critical for visibility. The traffic that historically paid you back with visitors is shrinking as a proportion of the traffic taking your content.
Cloudflare also offered a striking framing of human behaviour: for every hour spent online searching for information, only 15 minutes is now spent on the open web. Whatever you think of where that trend lands, it explains why referral traffic feels harder to earn.
Because some sites put their content in the HTML and some assemble it in the browser afterwards. An agent that does not run JavaScript, or does not wait long enough for it, sees an empty shell. Your content exists, and to that agent it does not.
This is the same problem search engines have had for years, just with less tolerance. Google built enormous infrastructure to render JavaScript. Most agents have not, because rendering is slow and expensive and they are working through a task in real time.
We build sites that put the content in the first response for exactly this reason, and it is why this blog is static HTML rather than a page that fetches its own articles. The full argument, including how to test what a non-rendering client actually receives, is in our piece on why AI crawlers miss JavaScript-rendered content.
The Agentic Commerce Protocol, or ACP, is an open standard for commerce flows between buyers, AI agents, and businesses. It is published under the Apache 2.0 licence and described as community-designed. The idea is that a business implements the specification once, and any compatible agent can then securely start a checkout.
The design decision worth noting is who keeps the customer. ACP''s documentation states that businesses remain the merchant of record, keeping control over which products can be sold, how they are presented, and how orders are fulfilled. That is a deliberate answer to the obvious fear that agents turn every shop into an anonymous supplier.
ACP is described as REST and MCP compatible, so a business can publish its checkout configuration through a traditional API or through the Model Context Protocol. It covers physical and digital goods, subscriptions, and asynchronous purchases rather than just simple one-off transactions.
The Agent Payments Protocol, or AP2, is an open protocol for agent-driven payments, announced on 16 September 2025. It exists because today''s payment systems assume a human is directly clicking buy on a trusted website, and an autonomous agent breaks that assumption completely.
The questions AP2 is trying to answer are authorisation ones. How do you verify that a user actually gave an agent authority for a particular purchase? Card networks and fraud systems were built around a person''s presence, and that signal disappears when software transacts.
AP2 is positioned as an extension of the Agent2Agent protocol and the Universal Commerce Protocol rather than a standalone system, and its documentation notes a v0.2 release and a donation to the FIDO Alliance, which is developing standards for trusted AI agent interactions. Handing a protocol to a standards body is usually a sign the sponsors want it to outlive them.
We would treat both ACP and AP2 as early. They are real, published, and backed, but they are not settled infrastructure, and most businesses reading this do not need to implement either yet.
Yes, because almost everything that makes a site agent-readable also makes it better for people and for search. Clean HTML, real headings, descriptive links, content in the initial response, and honest structured data are not agent-specific work. They are the same fundamentals, with a new reason to take them seriously.
That is the honest version of the pitch. We are wary of anyone selling agent optimisation as a separate discipline with separate deliverables, because most of what they will sell you is technical SEO with a new name on the invoice.
Where we would push back is on granting agents more than reading access without thinking. Letting software read your prices is very different from letting it start transactions, and the second deserves a real decision rather than a default.
Webflow''s own developer documentation is a good working example. Every page can be requested as clean Markdown by appending .md to the URL, a complete documentation index is published at /llms.txt, and there is an MCP server for AI clients to connect to directly. That is three deliberate affordances for non-human readers.
You do not need all of that. For most businesses the useful version is smaller: put your key facts in text rather than images, keep your pricing and contact details crawlable, use proper heading structure, and make sure the pages you care about do not depend on JavaScript to exist.
An llms.txt file is a low-cost addition if you want to be explicit about what matters on your site, though it is a convention rather than a standard anyone is obliged to honour. We set out what it does and does not do in our explainer on llms.txt.
The other half of the job is being worth citing in the first place. Structure gets you read, and substance gets you quoted, which is the distinction we draw in our guide to getting cited by AI search engines.
We do not think so, and we would treat confident predictions either way as speculation. The data shows a real shift in how people find information, but a website is still where your claims live, where trust is built, and where the content agents read comes from. Removing it does not make you more visible to agents. It makes you invisible.
What does seem likely is a change in what your site is for. Fewer visits, each carrying more intent, with agents handling the comparison stage that people used to do across five tabs. That would make persuasion matter less on some pages and clarity matter more on all of them.
We would rather build for that quietly than rebuild for a future nobody can describe yet. The sites that do well under either outcome are the fast, clearly written, well structured ones, which is a comfortable place to be.
Fetch your own most important page with JavaScript turned off and see whether the content is there. If it is not, fix that first, because everything else in this article depends on it. Then check that your prices, contact details, and core claims are real text rather than pictures of text.
Skip the protocols for now unless you sell online at scale, in which case ACP and AP2 are worth reading properly this year rather than next. For everyone else, the work that pays off is the unglamorous kind you should have been doing anyway.
If you want help working out whether your site is readable to the things that are actually visiting it, we are happy to take a look. Reach out at phoenix.studio and we will run the checks with you.
Tell us where you want to go. We'll tell you how we'd get you there.