> **TL;DR.** Fine-tuning a model means updating its weights on your own data so it learns your domain, tone, or task format — not just general knowledge. LoRA and QLoRA make this cheap enough for solo developers. Whether you need fine-tuning at all depends on your use case; often good prompting or RAG is the right answer first.
When Fine-Tuning Actually Makes Sense
Most teams jump to fine-tuning before they've exhausted cheaper options. Before spending compute, verify which problem you actually have.
**Fine-tuning is the right call when:**
- The model consistently fails a *format* requirement — structured output, a specific code dialect, a strict response template — even with detailed prompting.
- You need consistent *style or voice* across thousands of outputs and can't enforce it via system prompts reliably.
- Inference latency or token cost matters — a smaller fine-tuned model can outperform a larger base model on narrow tasks.
- Your domain has vocabulary, abbreviations, or conventions the base model clearly doesn't know (proprietary protocols, internal tooling, niche legal terminology).
**Fine-tuning is NOT the right call when:**
- The base model gives correct answers if you refine the prompt. Try [AI Prompts for Coders 2026](/en/rehberler/ai-prompts-coders-2026) patterns first.
- You just want the model to access fresh data — that's a RAG problem, not a weights problem.
- Your dataset is under a few hundred examples. You'll overfit.
- You need the model to "know more facts." Fine-tuning teaches patterns and formats, not long-tail factual recall.
Fine-Tuning Methods Compared
**LoRA (Low-Rank Adaptation)** is the default choice for most product teams. It inserts small trainable rank decomposition matrices into the transformer layers while keeping base weights frozen. You get a small adapter file (often under 1 GB) that snaps onto the base model at inference.
**QLoRA** adds quantization on top — the base model loads in 4-bit precision, so you can fine-tune a 13B parameter model on a single 16 GB GPU. Quality loss is minimal for most tasks.
**DPO (Direct Preference Optimization)** is worth knowing: it lets you train preference into the model using pairs of "good" and "bad" responses without a separate reward model. Simpler than RLHF, increasingly practical.
The LoRA Workflow: Step by Step
This is a concrete walkthrough using the Axolotl framework, which handles most of the configuration overhead.
**1. Prepare your dataset**
Format your data as instruction-response pairs in JSON:
```json
{"instruction": "Summarize this commit message professionally.", "input": "fix stuff", "output": "Resolved a regression in the authentication flow introduced in the previous release."}
```
Aim for 500–5,000 high-quality examples. More isn't always better — noisy data is worse than clean small data.
**2. Install and configure Axolotl**
```bash
pip install axolotl
axolotl fetch examples/llama-3/lora.yaml
```
Edit the YAML: set `base_model`, your dataset path, `lora_r` (rank, typically 8–64), and `output_dir`.
**3. Run training**
```bash
accelerate launch -m axolotl.cli.train your_config.yaml
```
On a single A100 80GB with a 7B model and 2,000 examples, expect 1–4 hours.
**4. Merge and export**
```bash
python -m axolotl.cli.merge_lora your_config.yaml --lora_model_dir="./lora-out"
```
You now have a full merged model you can serve or push to Hugging Face Hub.
**5. Evaluate — don't skip this**
Run your eval set through the fine-tuned model *and* the base model. Compare on your actual task metrics. If the gap is under 5%, reconsider whether you needed to fine-tune at all.
Tools and Infrastructure
**Frameworks:**
- **Axolotl** — the most batteries-included LoRA/QLoRA trainer. Config-driven, supports Llama, Mistral, Gemma, Phi families.
- **Hugging Face PEFT** — the underlying library Axolotl wraps. Use directly if you want more control.
- **Unsloth** — optimized kernels that cut training time and memory by 30–50% compared to vanilla PEFT. Worth using.
- **LLaMA-Factory** — web UI option if you prefer not to touch config files.
**Managed training APIs:**
- **Together AI** — submit a JSONL file, specify a base model, get back a fine-tuned endpoint. No GPU management.
- **OpenAI fine-tuning** — works for GPT-4o mini and compatible models. Straightforward API, limited model choice, costs add up fast at scale.
- **Replicate** — can host and serve custom LoRA adapters alongside base models.
- **Modal / RunPod / Lambda Labs** — rent GPU compute for self-managed training jobs when you want model ownership without cloud platform lock-in.
**For serving the result**, see [AI Model Deployment 2026](/en/rehberler/ai-model-deployment-2026) — the inference infrastructure decisions are as important as the training ones.
Data Quality: The Part Most Guides Skip
Training data quality is the single biggest predictor of fine-tuning success. Teams routinely underestimate the effort here.
**What makes training data good:**
- **Consistent format.** Every example should follow the same instruction/input/output schema. Mixed formats confuse the gradient signal.
- **Representative distribution.** If 80% of your examples cover one sub-task and 20% cover four others, the model will specialize hard on the majority case.
- **Edge case coverage.** Include examples that are at the boundary of correct behavior — these are where the base model tends to fail, and where your examples matter most.
- **No leakage of base model outputs.** If you generate training data with the same model you're fine-tuning, you're likely teaching it its own existing behavior. Use human-curated examples or outputs from a different, better model.
**Practical data prep tools:**
- **Argilla** — open-source annotation interface, good for human review of AI-generated drafts.
- **LabelStudio** — more general annotation, works for mixed media.
- **DataTrove / Dolma** — if you're working at scale with web data and need filtering pipelines.
Minimum viable dataset: 300–500 clean, diverse, correctly formatted examples. Beyond 10K examples, returns diminish rapidly unless the task complexity warrants it.
Cost and Resource Reality
Cost scales with: model size × dataset size × number of epochs × hardware rental rate.
**Rough ballpark for LoRA on 7B model:**
- A100 40GB on RunPod: ~$1.50/hr
- 2,000 examples, 3 epochs: roughly 2–4 hours
- Total: $3–$10 per training run
**QLoRA on consumer hardware (RTX 4090):**
- 7B model: feasible with 24 GB VRAM
- 13B model: tight but doable with gradient checkpointing
- 34B+: you'll need QLoRA + 2× 4090 or move to cloud
**Full fine-tuning a 70B model**: needs multi-GPU setups (4–8× A100s), takes days, and costs hundreds to thousands of dollars per run. Only justified when you need production-quality behavior change on a large model.
**Managed API pricing** (Together, OpenAI): typically charged per token trained. Estimate your dataset token count and multiply by their per-token rate. Convenient, but you don't own the weights in the same way.
Alternatives to Consider First
Before committing to ai fine-tuning, evaluate these in order:
1. **Better prompting** — system prompts, few-shot examples in context, chain-of-thought. Costs nothing, iterate in minutes.
2. **RAG (Retrieval-Augmented Generation)** — inject domain knowledge at inference time. Better for factual recall, easier to update.
3. **Structured outputs / constrained decoding** — if your issue is output format, JSON mode or grammar-constrained generation often solves it without training.
4. **Prompt caching** — if you're paying for long system prompts at scale, cached tokens are far cheaper than a training run.
5. **Model switching** — sometimes a different frontier model handles your task natively. Evaluate across providers before training.
If you've ruled these out, fine-tuning is the right move.
Next Steps
- If you're building an AI-powered product and deciding where fine-tuning fits in the stack, [AI for Startup Founders 2026](/en/rehberler/ai-startup-founders-2026) covers the product-level decisions.
- For deploying your fine-tuned model to production — inference servers, quantization for serving, cost optimization — see [AI Model Deployment 2026](/en/rehberler/ai-model-deployment-2026).
- If you're evaluating whether the effort is worth it for your specific idea, [AI Startup Idea Validation 2026](/en/rehberler/ai-startup-validation-2026) has a useful framework for assessing custom model ROI against simpler approaches.