Cutting cost: shorter prompts, smaller models and prompt caching

The bill that keeps growing. Meet Sara. She runs online bookings for a hair salon, and last month her support agent answered about three thousand customer messages. It changes appointments, answers questions about prices, and sends booking links. Customers love it. Then the monthly bill arrived: about thirty four dollars, and it grows every time the salon gets busier. That's not a disaster today, but Sara also wants agents for invoices and sales reports, and she sells this setup to other salons. So what if the same agent could do the same job for about a fifth of the price? That's exactly what this lesson is about, and none of it needs any code, just a little arithmetic.

What you will learn. Here's the plan for the next few minutes. First, we'll break one agent run into its parts, so you can see exactly where the money goes. Then we'll use three levers, one at a time. The first lever is shorter prompts, meaning less text sent with every message. The second is smaller models, meaning cheaper models for the easy work. The third is caching, where text you send again and again gets a big discount. We'll apply all three to Sara's salon agent and watch the monthly bill drop, with real numbers at every step.

Where the money goes. Quick reminder from the last lesson. A token is a small chunk of text, roughly three quarters of a word. Every agent run has two sides. Input tokens are everything you send: the system prompt, the customer's message, and any history. Output tokens are the reply. Each side has its own price, quoted per million tokens, and output usually costs several times more than input. Here's the part people miss. The model remembers nothing between runs, so your whole system prompt travels along with every single message. A long prompt times three thousand messages is where most of Sara's bill hides.

Lever one: shorter prompts. Sara's system prompt grew the way most prompts grow. Every time something went wrong, she pasted in another paragraph. Now it's a long scroll: the same rule said three ways, five full example conversations, and the entire price list for every service. Trimming it into a short page is mostly editing. Say each rule once. Keep one short example instead of five. And move the price list into a tool the agent calls only when someone actually asks about prices. The one thing you never cut is a rule that protects quality, like when to hand a customer over to a human.

Worked example: the trimmed prompt. Let's put numbers on it. These are example prices: three dollars per million input tokens and fifteen per million output. Each run carries the system prompt, about three hundred tokens of customer message and history, and a two hundred token reply. First, one run with the long prompt, per million: two thousand eight hundred input tokens times three, plus two hundred output tokens times fifteen, makes eleven thousand four hundred. For three thousand runs, long prompt, we multiply by three thousand and divide by a million: thirty four dollars twenty a month. Now one run with the short prompt, per million: fifteen hundred times three, plus three thousand, is seven thousand five hundred. Three thousand runs, short prompt, come to twenty two fifty. Price questions now cost a little extra for the tool call, but that's small next to sending the whole list on every run. One afternoon of editing saved about a third.

Lever two: smaller models. Think about how a real salon works. The receptionist answers most calls: opening hours, prices, moving an appointment to Thursday. Only the hard cases reach the manager, like a refund request or an angry customer. You'd never pay a manager's salary to read out opening hours. AI models work the same way. Small models are cheaper, often around a quarter of the price of a big one, and they handle simple, clear questions very well. When Sara looked at her messages, about seventy percent were easy ones like these. That's a huge share of runs that can move to the cheaper model.

Routing messages to the right model. To split the work, you add a router. A router is just a first step that reads the message and decides where it goes, like the receptionist deciding whether to fetch the manager. In a no-code tool, that's a simple branch: a small model labels the message easy or hard, and each label follows its own path. Using our example prices, with the small model at eighty cents input and four dollars output, the two thousand one hundred easy runs cost four dollars twenty, and the nine hundred hard ones cost six seventy five. The router is a model call too: about four hundred tokens in and a one word answer, on every message, which adds about a dollar a month. The total is eleven dollars ninety seven. That's about half of the trimmed version.

Lever three: prompt caching. Now the third lever. Imagine a regular at a coffee shop. The barista already knows her usual order, so she only has to say what's different today. Prompt caching works like that. The provider stores the beginning of a prompt you send often, and when the next run starts with exactly the same text, that part is billed at a small fraction, often around a tenth of the normal input price. The key is order. Role, rules, tools and the example never change, so they go first. The customer's message changes every time, so it goes last. One catch: providers only cache prompts above a minimum length, often about a thousand tokens and higher on some small models, so check the limit for your model.

Worked example: the cached bill. Here's the working for the cached version. Take an easy run on the small model, per million: the twelve hundred cached tokens cost eight cents per million instead of eighty, so the run comes to one thousand one hundred thirty six. Two thousand one hundred easy runs cost about two dollars thirty nine. Now a hard run on the big model, per million: cached tokens cost thirty cents instead of three dollars, so the run comes to four thousand two hundred sixty. Nine hundred hard runs cost about three eighty three. Add the router's dollar and two cents, which is too short to cache, and the month comes to seven dollars twenty four. This assumes every run reads from the cache, and it leaves out the small extra charge for writing the cache in the first place.

The whole picture. Let's line the versions up. Here's where Sara started: thirty four dollars twenty a month. After trimming the prompt, twenty two fifty. After sending easy messages to the small model, with the router included, eleven ninety seven. And after caching the stable start of the prompt in both models, about seven dollars twenty four. That figure assumes every run reads from the cache, which happens when messages arrive close together, and it leaves out the small charge for writing the cache. Still, the same agent now costs about a fifth of what it did, and customers can't tell the difference.

Common mistake: breaking the cache. Here's the trap that quietly cancels all your caching savings. Someone adds a friendly line at the very top of the prompt, like today's date, or "You are helping Maria". It feels harmless. But the cache only reuses text that's identical from the very first token. One changed word at the top is like a crooked brick at the bottom of a wall: everything above it no longer counts. The same happens if you tweak the rules on every run. Keep the top of the prompt frozen, and put the date, the customer's name and their message at the end.

Recap. Let's wrap up. Every agent run costs its input tokens and output tokens times their prices, and the system prompt is paid for on every single run. Shorter prompts and routing easy work to smaller models attack the biggest parts of the bill, as long as you count the router's own small cost. Prompt caching then discounts the text you repeat, as long as the stable part comes first and the changing details come last. Together, these took Sara's agent from about thirty four dollars a month to about seven, with the same answers for her customers. And those savings grow with every new client you build for.

Your exercise. Now it's your turn. Take the support agent prompt you've been building in this module and give it a trim. Count its tokens before and after, using your provider's token counter or a free online one. Then collect twenty realistic customer messages for a small business you know, and label each one easy or hard. Finally, estimate a month of three thousand messages for the original, the trimmed, the routed and the cached version. Use the same steps as our worked examples, by hand or in a spreadsheet with one row per version: input tokens times the input price, plus output tokens times the output price, times the number of runs, divided by a million. Don't forget the router. Bring your numbers to the next lesson, where we'll test the prompt against ten tricky messages.