Should Your Automation Drive a Browser or Call an API?

Should Your Automation Drive a Browser or Call an API?

Call the API if one exists. Drive a browser only when there is no API, no export, and no other way in. A browser gives you access to everything a person can reach, and it pays for that access with slower runs, higher costs, and failures that arrive silently when a button moves.

This comes up constantly now that models can look at a screen and click things. The demo is genuinely impressive. The production system built on that demo is often a mess, and the mess is avoidable.

Here is how we decide, and what the research actually says about how reliable browser-driving agents are today.

What Does "Driving a Browser" Actually Mean?

It means your automation opens a real browser, loads a page, and interacts with it the way a person would: reading the screen, clicking, typing, scrolling. Tools like Playwright and Puppeteer do this deterministically from a script. Agent frameworks do it with a model choosing each step.

Those two are very different animals and people confuse them constantly. A Playwright script is a program: same input, same clicks, every time. An agent driving a browser is making a judgement call at each step about what to click next. One fails loudly when the page changes. The other improvises, which sounds better and is usually worse.

How Reliable Are Browser Agents, Really?

Less reliable than the demos suggest. The WebArena paper, published by its authors and presented as an oral at NeurIPS 2024, built fully functional websites across e-commerce, social forum discussions, collaborative software development and content management, then measured end-to-end task success. Its best GPT-4-based agent completed 14.41% of tasks. Human performance on the same tasks was 78.24%.

TheAgentCompany, from the same research group and presented at ICML 2025, built a simulated software company with internal websites and data, and tested agents on work a digital employee would do. Its finding: "the most competitive agent can complete 30% of tasks autonomously."

Both numbers are baselines from their publication dates, and agents have improved since. Public leaderboards now report much higher WebArena scores, but they disagree with each other depending on the evaluation setup, which is itself a reason to treat any single figure carefully. The honest summary is that browser agents went from bad to partly useful, not from bad to reliable.

Why Is an API Call So Much Cheaper?

Because it skips the expensive part. An API call sends a request and gets structured data back. A browser run launches a browser process, downloads the page and its assets, waits for rendering, and then, if a model is involved, sends screenshots or accessibility trees into a model on every step.

That difference compounds. One API call might be a few hundred tokens of response. One agent step can be a full page of context, and a ten step task means ten of those. Then multiply by every retry. We have seen teams surprised by the bill on an automation that seemed trivial, which is the same failure mode we wrote about in controlling AI costs on a web team.

Latency follows the same pattern. An API integration finishes in under a second. A browser task finishes in tens of seconds at best. If anything downstream is waiting on the result, that gap changes what you can build.

When Is a Browser the Right Answer?

Four cases, in our experience. First, when the system has no API at all: plenty of legacy vendor portals, government sites and insurance systems still do not. Second, when the API exists but excludes the thing you need, which is common with reporting and export endpoints. Third, when you are testing the interface itself, because the whole point is to exercise what a user sees. Fourth, when the task genuinely requires reading a rendered page, like checking whether your own site still shows the right price.

Notice that three of those four are best served by a deterministic script, not an agent. The case for putting a model in the loop is narrower still: when the page layout varies unpredictably between runs and you cannot write a stable selector.

That is a real case, and it is rarer than people think. Most portals are ugly and stable, which is exactly what a script handles well.

How Do You Stop a Browser Script From Being Flaky?

Use the tool's own reliability features instead of sleeps. Playwright's documented model is that it "performs a range of actionability checks on the elements before making actions" and "auto-waits for all the relevant checks to pass and only then performs the requested action." For a click it confirms the locator resolves to exactly one element, and that the element is visible, stable, receives events, and is enabled.

Playwright also ships auto-retrying assertions that, in its words, "remove flakiness by waiting until the condition is met." And it offers a force option that "disables non-essential actionability checks," which is a useful escape hatch and a bad default.

The bigger lever is the locator. Playwright's own best practices guide says tests should "verify that the application code works for the end users, and avoid relying on implementation details" such as the CSS class of an element, and that each run should be "completely isolated" with its own storage, cookies and data. A script that finds a button by its accessible name survives a restyle. A script that finds it by a generated class name breaks on the next deploy.

What Breaks First When a Page Changes?

A script breaks loudly, an agent breaks quietly, and the quiet failure is worse. When a selector disappears, a Playwright run throws a timeout and you get an alert. When a page changes under an agent, the agent often finds something else to click and reports success.

That is the silent failure pattern we keep running into: the automation reports green, the data it produced is wrong, and nobody notices for three weeks. We wrote about it at length in automations that fail without telling you, and it is the single strongest argument for preferring determinism wherever you can get it.

If you do run an agent against a browser, you need an independent check on the output. Did the record actually get created? Does the number match a second source? An agent's own report of what it did is not evidence.

Is There a Middle Path Between the Two?

Yes, and it is usually the right answer. Use a model to interpret and a script to act. Let the model read the messy page content and decide what matters. Let deterministic code do the clicking, the writing and the recording.

In practice that looks like this: a Playwright script navigates a known path and pulls the raw page text, a model turns that text into structured fields, and code validates those fields against a schema before anything is written anywhere. The model never touches the keyboard. You get flexibility where the data is messy and determinism where the consequences are.

This is also easier to test, because you can replay a saved page through the model step without launching anything. That matters more than it sounds. Most of the pain in browser automation is that you cannot reproduce yesterday's failure.

What About Rate Limits and Terms of Service?

Check both before you build, not after. A browser can reach pages an API would gate, and that is sometimes exactly what the gate was for. Automating a third-party interface may breach its terms even where it is technically easy, and the fact that a model can click a button is not permission to click it ten thousand times.

The polite version of this matters commercially too. If a vendor notices sustained automated traffic against their web interface, the likely outcome is a block, and you have now built a dependency that can be switched off without notice. An API gives you a contract, a documented rate limit and a version policy.

For your own systems this is a non-issue, which is why browser automation for internal testing is uncontroversial while browser automation against someone else's portal deserves a conversation first.

How Should You Decide on a Specific Task?

Work down four questions in order. Is there an API or a scheduled export that covers this? If yes, use it and stop. If no, can a deterministic script reach it with a stable locator? If yes, write the script. If no, does the page vary so much that only a model can navigate it? If yes, use an agent for navigation but validate the output independently. And in all cases: what happens if this runs and produces the wrong answer without failing?

That last question decides how much you should invest in checks. An automation that drafts an internal summary can be wrong occasionally. An automation that updates pricing on a live site cannot. We set the tolerance first and pick the tool second, which is the same order of operations we argued for in choosing between an AI agent and a plain script.

If you are staring at a vendor portal with no API and wondering whether this is worth automating at all, that is a good conversation to have with someone who has built both. We are at phoenix.studio.