Should Your AI Automation Use a Small Model or a Big One?

Does Model Choice Actually Matter for Automations?

More than most teams think, and not in the way they expect. The gap between a small model and a frontier model is not mainly about quality on your task. It is about cost per run, speed, and how badly the system fails when it is wrong. Those three things decide the answer.

We see the same pattern on client automation projects. Somebody picks the most capable model available, wires it into a workflow that runs thousands of times a month, and gets a bill nobody budgeted for. Then somebody else overcorrects and puts the cheapest model on a task that needed judgement.

Here is how we actually make the call, with the published numbers behind it and the tests we run before committing.

What Does the Price Gap Look Like?

Wide enough to change architecture decisions. Anthropic publishes its API pricing openly, and the spread across its tiers is roughly ten to one from the smallest model to the most capable. That is not a rounding difference. At volume it is the difference between a line item and a problem.

Here is the published pricing across Anthropic's tiers, retrieved in September 2026, in US dollars per million tokens.

Tier	Input	Output	Cache read
Small	1.00	5.00	0.10
Mid	2.00	10.00	0.20
Frontier	5.00	25.00	0.50
Top	10.00	50.00	2.50 write tier applies

Two things in that table matter more than the headline rates. Output tokens cost five times what input tokens cost at every tier, so a chatty model is expensive twice over. And cached input reads cost a fraction of fresh input, which means a well structured prompt can cut real spend more than a model downgrade would.

Anthropic also documents a 50 percent saving on batch processing across input and output tokens. If your automation does not need an answer within seconds, that discount is available before you touch model choice at all.

Where Do Small Models Genuinely Win?

On bounded tasks with a clear right answer. Classification, extraction, routing, tagging, short summaries and format conversion all fall into this group. The task has a small output space, the input is structured enough to constrain it, and a human could check the result in seconds.

These are also the tasks that run most often. A workflow that classifies every inbound form submission runs far more times than one that drafts a proposal. Cost per run compounds exactly where small models are strongest, which is a convenient coincidence.

The other win is boring and real. A cheaper model lets you afford to run the same input twice and compare, or to run a second model as a checker. Two cheap passes with disagreement detection often beat one expensive pass with no verification at all.

Where Do They Fall Over?

On anything that needs to hold several constraints at once. Long multi step reasoning, code that has to compile, decisions that depend on subtle context, and any task where the correct output is open ended. Small models do not fail loudly here. They produce something plausible that is quietly wrong.

Tool use is the sharpest edge. When a model has to choose between several tools, build correct arguments and recover from an error, capability differences show up fast. An automation that calls three APIs in sequence is a much harder task than it looks on a diagram.

Context length is the other practical limit. Smaller models often ship with smaller context windows, so a workflow that stuffs a long document into every prompt may simply not fit. Check the vendor's model page before you design around a context size.

How Do You Decide Without Guessing?

Test both on your own data before you commit. This is the whole answer, and almost nobody does it. Public benchmarks tell you about general capability. They tell you nothing about whether a small model can classify your support tickets correctly.

Build the eval first, then run the same set through each candidate. Compare accuracy, cost per run and time per run side by side. The decision usually makes itself once those three columns exist, and it is often not the answer either side of the argument expected.

Independent benchmarks are still useful for narrowing the field. Artificial Analysis publishes comparisons across intelligence, quality, performance and price, and defines its measures precisely: output speed as average tokens received per second after the first token, and time to first token as the duration from request submission until the first token arrives. Use that to build a shortlist, not to pick a winner.

What Should Your Eval Set Look Like?

Small, real and adversarial. Fifty examples drawn from your actual production inputs beats five hundred synthetic ones. Include the awkward cases: the empty field, the wrong language, the input that is technically valid but nothing like the others, the one that broke the workflow last month.

Grade the thing you care about, not the thing that is easy to score. If the automation writes a CRM field, grade whether the field value is correct and usable. Do not grade whether the prose sounded good. We wrote about building this properly in our guide to evaluating AI automation in production.

Then keep the set. The reason to own an eval is not this decision. It is the next model release, when you want to know in an hour whether switching is safe.

Is Latency a Better Reason Than Cost?

Often, yes, and it gets ignored. If a model sits in front of a person waiting for a page to load, speed is a product requirement rather than an optimisation. A cheaper model that responds in a fraction of the time can be the right choice even when the more capable one is slightly more accurate.

Artificial Analysis separates these two measures deliberately, because they behave differently. Time to first token governs whether an interface feels responsive. Output speed governs how long a long answer takes to finish. A chat style interface cares about the first. A background job cares about the second, if at all.

For work that nobody is waiting on, latency should not enter the decision. Overnight enrichment, nightly summaries and scheduled reports can take as long as they like, which is exactly when the batch discount and the more capable model both become affordable.

Should You Route Between Two Models?

Sometimes, but later than people think. Routing means classifying the incoming task and sending easy cases to a small model and hard cases to a larger one. It works, and it is the standard answer at real scale. It also adds a classifier that can itself be wrong, plus two code paths to maintain.

The prerequisite is a working eval, because routing is only safe if you can measure what the router sends where. Without that, you have built a system that fails in a new way and cannot tell you it is failing.

Our rule is to ship on one model, measure for a month, and only introduce routing when the data shows a clear split in task difficulty and the volume justifies the complexity. Premature routing is the most common over engineering we find in inherited automation stacks. We covered the wider spend question in our piece on AI cost control for web teams.

What Do Most Teams Get Wrong Here?

They treat it as a one time choice. Model pricing and capability both move, sometimes within a quarter. A decision made carefully in March can be wrong by September, and the team that has an eval set can find out cheaply while the team without one keeps paying.

The second mistake is optimising the model before optimising the prompt. Cached input reads cost a fraction of fresh input at every tier, so restructuring a prompt so the stable part is cacheable often saves more than dropping a tier, and costs nothing in quality.

The third is measuring cost per request instead of cost per completed task. A cheaper model that needs two retries and a human correction is not cheaper. Count the whole path, including the human time at the end of it.

How Should You Approach Your Next Automation?

Start on a capable model, get it correct, then push down. Correctness first is the cheaper order, because you learn what the task actually requires before you start trading capability away. Trying to save money on day one usually means debugging a quality problem and a cost problem at the same time.

Once it works, take the fifty example eval, run the smaller tier against it, and look at the three columns. If accuracy holds, take the saving. If it does not, you now know exactly which cases need the bigger model, which is the input to a routing decision later.

And decide whether you are building this at all before optimising it. We laid out that question in our piece on building versus buying AI automation.

If you want help sizing a model decision, or you have an automation bill that grew faster than the automation's value, we are happy to look at it with you. Find us at phoenix.studio.