LLM Fine-Tuning

Fine-tuning is the process of continuing to train a pre-trained language model on your specific data so it adapts to your task, domain, and output requirements. Use fine-tuning when prompt engineering alone cannot produce consistent output format, when you need significant inference cost reduction at scale (a fine-tuned smaller model can outperform a larger model with a long system prompt), or when domain-specific vocabulary significantly degrades base model performance. Fine-tuning is not always the right answer; start with prompt engineering and RAG first.

OpenAI fine-tuning API supports GPT-4o mini and GPT-3.5 Turbo (hosted fine-tuning, no infrastructure required). Open-source models include Llama 3 (8B, 70B), Mistral 7B, Phi-3, and Gemma (require GPU infrastructure for training). Google Gemini fine-tuning is available via Vertex AI. The right model depends on your budget, data privacy requirements, and accuracy needs. Open-source models eliminate per-token costs and run on your own infrastructure.

The number depends on the task and the base model. Quality matters more than quantity: consistent, representative examples beat a large noisy set. For open-source fine-tuning with LoRA or QLoRA adapters, larger adaptation needs more examples. We assess your existing data and help curate or generate training examples if your dataset is thin.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that trains a small set of adapter weights rather than the full model. It is cheaper in compute and memory than full fine-tuning while achieving comparable results for most tasks. QLoRA extends this with quantization for even lower memory requirements. LoRA is the standard approach for fine-tuning open-source models on modest GPU infrastructure. We use LoRA and QLoRA for open-source model fine-tuning and full fine-tuning only when the task requires it.

We establish a benchmark before fine-tuning. A representative set of inputs with expected outputs is evaluated on your task-specific metrics (accuracy, format compliance, domain terminology usage, output length consistency). The fine-tuned model is evaluated against this benchmark on a held-out test set. We only recommend production deployment when benchmark improvement is statistically significant. Fine-tuning that does not improve over the baseline prompt-engineered base model is not worth the cost.

The single sharpest qualification question. RAG fixes knowledge failures; fine-tuning fixes behavior failures. Order to try: evaluation set first (30 to 100 real inputs with known-good outputs), then prompt, then a bigger model, then RAG for knowledge failures, then fine-tuning only if the failure is behavioral. Most projects stop before the final step. Tell us what the model is getting wrong, in the words your users would use, and we will tell you which of these three it is. A short engagement frequently ends with a recommendation cheaper than the planned project.

The sharpest lock-in question in the set: can we export the weights and host elsewhere? Three costs never quoted: the dataset ages (stale examples equal confident wrong behavior), you are pinned to a base model (switching cost on deprecation), and evaluation becomes mandatory rather than optional. Fine-tuning attaches to a specific model version; a prompt has no switching cost. When does fine-tuning genuinely win? Format compliance on structured outputs, style and voice consistency, domain vocabulary, and distillation to smaller models. Never fine-tune for facts that change.

Project cost covers training data curation, fine-tuning run costs, evaluation, and deployment. The biggest cost driver is usually data curation and evaluation work, not the training run itself. Request a 30-min call and we will scope your project and give you the number.