Reducing AI costs

How I use a cheap model for most of an agent task and save the expensive model for decisions that actually matter

I’ve been using strong AI models for coding, and they are great, but paying one to do every tiny step feels like a waste, especially when most of an agent’s time is spent reading files, calling tools, waiting for tests, renaming things and cleaning up its own output, none of which needs the smartest model available, I only really need that model for the few decisions where a bad answer can ruin the rest of the task.

That is why Anthropic’s advisor tool caught my attention, I liked the idea, but I wanted to use it inside Pi without locking both models to the same provider, so I rebuilt the pattern as a Pi extension.

The setup I normally use is DeepSeek V4 Flash for the actual work and GPT-5.6 Sol as the advisor.

What I built

My extension has two roles, an executor and an advisor.

The executor owns the task, it talks to me, reads files, edits code, runs tests and writes the final answer, while the advisor is a stronger model it can call whenever it reaches a decision worth double-checking.

I think of it like asking a more experienced developer for a second opinion, I am not handing over the keyboard or starting another agent in a separate workspace, I am just showing it what has happened, asking what it thinks and giving control straight back to the original model.

That difference matters, I didn’t want an expensive manager model sitting above a group of cheap workers and reviewing everything they do because that would just move the cost somewhere else, in my setup the cheap model handles the job from start to finish, and Sol only appears when its judgment is actually useful.

The advisor is exposed to the executor as a normal tool, when the executor calls it my extension takes the current Pi session and creates a separate read-only request for Sol, including the system prompt, conversation, tool calls, tool results and whatever progress the executor has made so far, and while the executor can add a question to focus the review, it doesn’t have to waste tokens writing a giant summary first.

I also don’t send the entire raw session forever because that would eventually make the advisor just as expensive as the main agent, so before each call the extension cleans up the context:

• The latest eight messages stay complete
• Older reasoning is removed
• Large old tool calls and results are shortened, while keeping their beginning and end
• Tool names and descriptions are included, but the full JSON schemas are not

The advisor gets no tools, which means it cannot open another file, run a command, edit the repo, call another advisor or reply to me directly, all it can do is look at the evidence and return a recommendation, and I keep that visible advice under 120 words with a 2,048-token hard limit on the whole response.

I was pretty strict about this because giving the advisor tools ruins the whole point, the moment it starts its own agent loop it keeps collecting more context and more tool output, and now I am paying for a second worker instead of one useful opinion.

I also cap how many consultations can happen in one turn and across the whole session, if the advisor provider fails the executor keeps going instead of crashing the task, repeated calls use the same cache identity, and Pi asks for approval before any context is sent to a different provider.

The rough cost difference

I don’t have enough clean production telemetry to call this a benchmark yet, the numbers below are just an estimate using public API prices, and since my actual setup uses subscription-backed providers, this is not a copy of my bill either.

For a deliberately large example, I assumed one agent task used 10 million input tokens and 500,000 output tokens, then used a 99% cache hit rate because agent sessions repeat most of their existing context on every turn, leaving 9.9 million cached input tokens and 100,000 uncached ones, I also added two Sol consultations with 20,000 compressed input tokens each and charged both for the full 2,048-token output limit, which is probably harsher than what I would normally see, but I would rather overestimate it.

At the current GPT-5.6 Sol list price of $5 per million input tokens and $30 per million output tokens, using Sol for the entire task comes out to:

(100,000 × $5 / 1M) + (9,900,000 × $0.50 / 1M) + (500,000 × $30 / 1M) = $20.45

Using the current DeepSeek V4 Flash peak rates of $0.014 per million cached input tokens, $0.44 per million uncached input tokens and $1.32 per million output tokens, the executor costs about $0.8426, then the two Sol consultations add $0.32288, bringing the total to $1.16548.

That is about 94% cheaper with these assumptions, although I am not claiming everybody will save exactly 94%, shorter advice, cache hits and off-peak DeepSeek pricing can make it cheaper, while too many advisor calls or bloated context can make it more expensive. The exact percentage is not really the part I care about, what matters to me is that I am paying the expensive model to make a few important decisions instead of paying it to read every file and run every command.

How I use it

The actual flow ended up being pretty small:

I started with one consultation after the executor understood the task but before it committed to an expensive decision, and I log when the advisor gets called, how much context it receives, what the call costs and whether the recommendation changes anything, because if the advice doesn’t change the plan or catch a risk, the call was probably just expensive reassurance.

These are the guardrails I settled on:

I treat every advisor response as untrusted advice, so the executor still has to test the recommendation before acting on it, context does not cross provider boundaries without permission, an unavailable advisor does not stop the task, and simple lookups or mechanical edits do not need a consultation at all.

I will only add a second call near the end of a task if the first one proves useful, I don’t want the executor asking Sol for approval every five minutes because at that point I might as well have used Sol for the whole task.

What surprised me

I tested DeepSeek V4 Flash with GPT-5.6 Sol first, then GPT-5.6 Luna with Sol, I kept the advisor exactly the same, but changing the executor made a bigger difference than I expected.

DeepSeek V4 Flash is extremely cheap and capable, but it produces a lot of long thinking slop, by the time it finally asks the advisor it has often explored too much and half-committed to a plan already, Sol can still correct it, but I would rather get that advice before DeepSeek burns tokens going down the wrong path.

Luna handles the relationship better, it seems to understand that asking the advisor is part of the workflow rather than an emergency button, so when it reaches a real decision it pauses, asks, and then continues with the answer instead of trying to solve everything alone first.

In my tests, Luna asked earlier and wasted fewer tokens before each consultation, which taught me that price and tool use are not the only things that matter here, the executor also needs decent stopping instincts, because a cheap model that wanders around for ages before asking for help can eat into the savings pretty quickly.

Is it worth it?

This setup does not magically turn a cheap model into a frontier model, and if every step of a task needs frontier-level reasoning, I would just use the strong model from the start.

That is not what most of my coding tasks look like though, usually most of the work is routine and only a few decisions determine whether the result is solid or complete garbage, so DeepSeek can keep the loop moving for almost nothing, and I can save Sol for the moments where its judgment is actually worth the money.