Running free local models with Ollama for zero per-call cost

The surprise AI bill. Picture a hair salon you've built an automation for. Every booking request, every "are you open Sunday?" goes to a cloud AI model and gets classified and answered. It works beautifully. Then the owner forwards you a bill that grows every month, because every single message is a paid call. Now imagine you run that same workflow for ten clients. Per-call pricing quietly eats the margin you were hoping to keep. Today we fix that by running the model on your own laptop or a small server, where each extra message costs you almost nothing.

What you will learn. Here's the plan for this lesson. First, you'll understand what a local model is, and why running it yourself removes the per-call charge. Second, you'll install Ollama, a free tool that runs AI models on your own machine, and plug it into an n8n workflow that sorts customer messages. Third, you'll learn a simple rule for deciding when a small local model is good enough and when a message should still go to a stronger cloud model. Keep this checklist in mind, because each slide ticks off one piece of it.

Renting versus owning. Let's look at two ways to use an AI model. A model is the trained program that reads text and writes an answer. On the left is the cloud: a company runs the model on their servers and you send it your text over the internet. Their meter counts tokens, which are small chunks of words, roughly three quarters of a word each, and you pay per token. That's like taking a taxi: easy, but the fare keeps running. On the right is your own laptop running the model itself. That's like owning a bike. You pay once in hardware and a bit of electricity, and then every extra ride is free.

Installing Ollama. Ollama is a free app for Windows, Mac and Linux that downloads models and runs them for you. Install it from ollama dot com, then open a terminal. A terminal is the text window where you type commands: Terminal on a Mac, PowerShell on Windows. The pull command downloads a model; llama three point two is a small one, about two gigabytes, that runs on most modern laptops. The run command opens a chat so you can test it right away. Try asking it to write a polite reply to a late customer. The list command shows every model you've downloaded. Once you see an answer appear, you have your own private AI with no meter running and no account needed.

Picking a model size. Models come in sizes, measured in parameters. Parameters are the numbers the model learned during training, a bit like the settings on a giant mixing desk. A model labelled three B, which means three billion of them, is a small one. A small model, like a three or eight billion one, is quick and handles sorting messages or drafting short replies well. A large model, like seventy billion, reasons better but needs far more memory and is slower. Start small. For most small business tasks, such as tagging a lead or summarizing an email, a small model is more than enough, and it keeps replies quick on ordinary hardware.

Plugging Ollama into n8n. Now let's swap it into a real workflow. Take the message-sorting flow from lesson three: a webhook receives a contact form, a Basic LLM Chain node classifies it, a Switch node routes it, and an email goes out. The only change is the model underneath. Attach an Ollama Chat Model node to the chain, and create a credential with the base URL http colon slash slash localhost, port one one four three four. That's where Ollama listens. Pick the model name you pulled. That's it: the same workflow, the same prompts, and now every classification costs you nothing per call.

Worked example: sorting messages. Here's roughly what happens behind the scenes, so you can see there's no magic: n8n sends a request like this to Ollama's chat endpoint. The body names the model, turns streaming off so the answer arrives in one piece, and asks for JSON, which is structured text your next node can read reliably. The messages list carries the prompt, which asks for one of three labels. For "Can I move my cut to Friday at four?" the model should answer with the label booking, and your Switch node sends it to the booking reply. A message like "my colour came out wrong" lands in the complaint tray instead, and gets flagged for the owner.

Worked example: what you save. Let's put real numbers on it. Say each client gets three thousand messages a month, and you serve ten clients. Each message uses about eight hundred tokens going in and out. The loop prices that at three example cloud rates per million tokens: a budget model, a mid model and a premium one. That's fourteen dollars forty, seventy two dollars, or three hundred sixty dollars a month. Locally, if each answer takes four seconds on a laptop drawing an extra sixty watts, that's about thirty three hours of work, so two kilowatt hours, and at fifteen cents per kilowatt hour the electricity comes to about thirty cents. On top of that is the laptop or small server you already own or buy once, which has to be switched on while messages arrive. Those savings grow with every client you add.

Local first, cloud when needed. Local doesn't have to mean local only. Small models make more mistakes on unusual or tricky messages. So use a simple pattern: local first, cloud when needed. Ask the local model for a label plus a confidence score from zero to one. An If node checks it. Above, say, zero point eight, the workflow replies on its own. Below that, the message goes to a stronger cloud model, or straight to the owner. A small model's opinion of itself is not always reliable, so treat the score as a rough hint and spot check the replies it marks as confident. What share of messages do you think are tricky? In a typical small business it's only a small slice, so you pay for that slice and nothing else.

Common mistake: localhost in Docker. Here's the trap almost everyone hits. Docker is a tool that runs n8n inside a sealed box called a container. If you run n8n that way, the word localhost points to the container, not to your laptop. So n8n knocks on its own wall, finds no Ollama, and the node fails with a connection error. The fix is to use host dot docker dot internal, port one one four three four, which means "the computer this container runs on". On Linux, add the option shown on screen, add host, when you start the container, so that name works. Always press the Test button on the credential before building the rest of the flow.

Recap. Let's wrap up. A local model runs on your own machine, so like owning a bike instead of hailing a taxi, each extra message costs almost nothing. Ollama downloads and runs the model, and in n8n you only swap the chat model node and point it at port one one four three four, or at host dot docker dot internal when n8n lives in Docker. And you don't have to choose all or nothing: let the local model handle the everyday bulk, and route the low confidence cases to a cloud model or a person.

Your exercise. Now it's your turn. Install Ollama and pull llama three point two. Open the message-sorting workflow you built in lesson three and swap its model for an Ollama Chat Model node. Then write ten realistic customer messages for a small business you know: bookings, complaints, questions about prices. Run all ten through the flow. In your notebook, record how many labels came out right and roughly how many seconds each took. Bring those numbers to the next lesson, where we'll make sure a workflow like this never fails silently.