What Happens When Your AI Agent Hits a CAPTCHA?

What Happens When Your AI Agent Hits a CAPTCHA?

As of September 2026, it usually fails. A CAPTCHA is built to prove a human is present, and an agent working on a user's behalf cannot prove that. The industry answer is Web Bot Auth, where the agent signs its requests with a key instead of solving a puzzle.

This question comes up constantly now. A team builds an automation that checks a supplier portal or pulls a report, it works for a week, then it starts returning a challenge page. Nothing in the automation changed. The site's bot defences simply noticed it.

The fix is not a better browser trick. It is identity. Below is what the standard actually does, how far along it really is, and what to do on both sides of the request.

Why Do CAPTCHAs Block AI Agents That Users Asked For?

Because the site cannot tell the difference. A traditional bot rule sorts traffic into human or not human. An agent acting for a real person fails that test even though the request is legitimate. The old categories simply do not have a slot for it.

Cloudflare made this point directly when it introduced signed agents in August 2025. It described a new class of traffic, agents that are directed by an end user rather than by a single company or entity, and noted that existing verified bot handling did not fit them.

That distinction matters more than it sounds. A search crawler belongs to one company, always behaves the same way, and can be checked by reverse DNS. An agent might be a browser extension one minute and a scheduled job the next, running for a different person each time.

What Is Web Bot Auth?

Web Bot Auth is a way for an automated client to cryptographically sign its HTTP requests so a server can confirm who sent them. Instead of guessing from behaviour, the server checks a signature against a published public key. Identity is proven in the request itself, with no puzzle involved.

The underlying work is an IETF effort by Thibault Meunier and Sandor Major. Their architecture draft was titled "HTTP Message Signatures for automated traffic Architecture" and describes letting automated HTTP clients cryptographically sign outbound requests so servers can authenticate their identity.

The important word is authenticate, not authorise. A signature tells you which agent is knocking. It does not tell you whether to let it in. That decision stays yours, and we think most teams will get that part wrong first.

Where Does the Standard Actually Stand Right Now?

It is real but unfinished, and honest reporting on it is rare. The original architecture draft reached revision 05, was last updated on 2 March 2026, has expired, and was replaced. The protocol draft that followed reached revision 02 on 18 August 2026 and has also expired as an individual submission.

The encouraging part is what replaced it. The individual protocol draft was succeeded by draft-ietf-webbotauth-httpsig-protocol, and that "ietf" in the name signals a working group document rather than one person's proposal. That is the normal path a serious standard takes.

So the accurate summary is this. The mechanism is deployed in production by major infrastructure vendors, and the specification behind it is still moving through the IETF. Anyone telling you it is a finished web standard is ahead of the evidence.

Which Headers Does a Signed Agent Send?

Three, according to Cloudflare's own documentation. Signature-Input carries the metadata, Signature carries the cryptographic signature over the chosen components, and Signature-Agent points at the bot's key directory as a quoted HTTPS URL.

Signature-Input is where the detail lives. It includes a tag set to web-bot-auth, a keyid holding the JWK thumbprint, created and expires timestamps, and optionally a nonce. The timestamps are what stop a captured signature from being replayed later.

Cloudflare also documents what does not work, which is the kind of thing you only discover at two in the morning otherwise. The @query-params and @status components are unsupported, as are the sf, bs, key and req parameters.

If you already handle signed webhooks, this will feel familiar. The shape is the same as other request-signing schemes, and the reasoning matches what we wrote about in our guide to telling a real AI agent from a scraper.

How Does a Bot Publish Its Keys?

At a well-known URL on its own domain. The agent hosts a key directory at /.well-known/http-message-signatures-directory that serves a JSON Web Key Set containing its Ed25519 public key. That directory response must itself be signed using HTTP message signatures.

The self-signing requirement is elegant. It closes the obvious hole, where an attacker serves a fake key directory and then signs requests with the matching private key. The directory has to prove it is the directory.

On Cloudflare specifically, publishing keys is not enough on its own. A bot operator also registers through the Bot Submission Form in the Cloudflare dashboard, under Manage Account and then Configurations. There is a human review step in the middle, and you should plan for it to take time.

What Do Cloudflare and AWS Do With a Signed Agent?

Both now treat a valid signature as grounds to let the request through. Cloudflare validates the signature and sets the cf.bot_management.verified_bot field, which your firewall rules can then read. Unverified bots fall back to other methods such as reverse DNS validation.

AWS moved in November 2025. On 21 November it announced Web Bot Auth support in AWS WAF, which automatically permits verified Web Bot Auth bots and refines its AI category so that verification is respected rather than every unverified bot being blocked. At announcement, the feature was limited to CloudFront distributions and carried no additional fee beyond standard AWS WAF charges.

Two vendors is not the whole internet, but these two sit in front of an enormous amount of traffic. When both adopt the same mechanism within fifteen months, the direction is not really in doubt.

Does This Mean You Should Let Every Verified Agent In?

No, and this is where we see teams overcorrect. A signature proves identity, not intent. An agent can be perfectly verified and still scrape your entire pricing table every hour, or submit forms you would rather a human submitted.

The useful mental model is an allow list, not a gate. Decide which agents you actually want, and what you want them to reach. Cloudflare's enterprise tier lets customers act on signed agents as a group when configuring security rules, which is a starting point rather than a finished policy.

Ask the commercial question too. If an agent is reading your content to answer questions elsewhere, that is a different transaction than a buyer reading your site. We looked at that trade in our piece on charging AI bots for crawling.

What Should You Do If You Run the Agent, Not the Site?

Sign your requests and stop fighting the defences. Generate an Ed25519 key pair, publish the signed directory at the well-known path, send the three headers, and register with the networks that matter for your targets. Then failures become policy decisions you can appeal rather than mysteries.

Until then, treat challenge pages as a hard signal. If a site is challenging you, it does not want anonymous automation. Routing around that with rotating browser fingerprints is a maintenance treadmill, and it puts you on the wrong side of a conversation you will eventually need to have.

There is also a simple design lesson. Build your automation so a blocked request surfaces clearly rather than silently returning a challenge page as if it were data. We have seen pipelines quietly ingest CAPTCHA HTML for weeks.

What Breaks If You Ignore All of This?

Two things, in opposite directions. If you run a site and block everything unverified, you will start blocking agents your own customers sent, and you will not see it in any dashboard because a blocked request is not a session. If you run agents and stay anonymous, your automations will degrade at random.

The second failure is worse for B2B companies than it looks. Buyers increasingly ask an assistant to check a vendor before they visit the site themselves. A challenge page served to that assistant is a lost evaluation, and it never appears in your analytics.

This is the same problem as crawler access, one layer up. Our guide to controlling AI crawlers with robots.txt covers the layer below it.

What Should Teams Do Next?

Start by looking at what you are already blocking. Pull a week of challenge and block events from your edge, group them by user agent, and see how much of it is agent traffic rather than abuse. Most teams have never looked, and the answer usually surprises them.

Then pick a posture deliberately. Allow the signed agents you want, keep the rest challenged, and write it down so the next person understands the intent. The standard is still moving, so revisit the decision rather than setting it once.

If you want help reading your edge logs or deciding which agents should reach which pages, we are happy to walk through it. You can find us at phoenix.studio.