Why does inference cost more than the project we were quoted?

The quote is usually the build. Inference is the running cost, and it grows with every question, image, or document you send. A pilot can look cheap because few people use it. Price the busy month before you launch, and put a cap on how much one request can consume.

Does inference happen in the cloud or on our own servers?

Either. Most companies call a vendor's service, so the text leaves your building for that request. If the data cannot leave, you run the model on machines you control, which costs more to set up and to staff. The answer quality and the bill both change with that choice.

Foundations

What is AI inference?

Inference is what you pay for every time a user interacts with your AI feature. A feature that is cheap to prototype can still be expensive to run at scale.

In plain terms

Inference is the act of running a trained AI model to get an answer, as opposed to training, which is the earlier work of building the model.

Inference is the moment the system answers. Training is the practice. Inference is the live use: a customer asks a question, a photo is checked, a ticket is summarized. That is the step you pay for every time, and it is the step a user waits on.

When you plan a project, split the money in two. One amount to build it. Another amount, every month, to run it. A demo on a quiet afternoon does not show the bill of a busy Monday. Ask how many requests you expect, how long each answer may be, and what happens when a campaign doubles the volume.

Think of it this way: Training is learning to drive. Inference is every trip you take after passing the test. The exam happens once; the road charges you per mile.

A support team runs every inbound message through a model so it can sort what the customer wants. The build was a one-time project. The bill arrives again every month, and it grows with the number of messages.

An insurer adds a what does my policy cover box to the customer app. In the pilot, fifty people a day try it and the bill is small. After the email launch, four thousand people try it the same morning. The feature still works. The monthly run cost is now the larger line.

Inference is always involved when users interact with an AI feature. The question is whether you have designed for the running cost from day one. You cannot avoid inference if you want a live AI feature. But you can cut cost by caching frequent answers, shrinking prompts, and routing simple tasks to smaller, cheaper models. A recurring cost per use, not a one-time build cost.

You do not need to pick a model from this page. RaftLabs starts from the job: the documents, the decision, and what a wrong answer costs. You see a working version before a large build. The related work on our side is MLOps.

This sits with the other foundations terms on the glossary. The words under every AI conversation, from the model itself to the data it learned from. Worth reading next: Artificial Intelligence (AI), Machine Learning (ML), and Large Language Model (LLM).

Common questions

: The quote is usually the build. Inference is the running cost, and it grows with every question, image, or document you send. A pilot can look cheap because few people use it. Price the busy month before you launch, and put a cap on how much one request can consume.
: Either. Most companies call a vendor's service, so the text leaves your building for that request. If the data cannot leave, you run the model on machines you control, which costs more to set up and to staff. The answer quality and the bill both change with that choice.

Work with us

Tell us what's broken.

Tell us what's not working in your business. We'll find the real problem and tell you exactly what it would take to fix it.

Scope and cost agreed before work starts. No surprises. No obligation.
Working prototype within 3 weeks of kickoff.
Pay by milestone. You see progress before each invoice.
60-day post-launch warranty. Bug fixes, UI tweaks, and deployment support. No retainer.
All conversations are NDA-protected.