Fine-tuning & Transfer Learning for NLP
Fine-tuning Strategy
Fine-tuning adapts a pretrained model to a specific task with a small labeled dataset. Three main approaches: 1. Full Fine-tuning — Update all weights. Requires GPU, more data, risk of catastrophic forgetting. 2. Feature Extraction — Freeze all layers, train only a new task head. Fast, works with tiny datasets. 3. Parameter-Efficient Fine-Tuning (PEFT) — Modify only a small fraction of parameters (LoRA, Adapters, Prefix Tuning).
LoRA — Low-Rank Adaptation
LoRA freezes the original model and injects trainable low-rank matrices: W' = W₀ + ΔW = W₀ + B·A Where A ∈ R^{r×d}, B ∈ R^{d×r}, r << d Trainable parameters: only A and B (e.g., 0.1% of original params). Full-quality results at tiny fraction of compute. Standard for LLM fine-tuning (Alpaca, Vicuna, RLHF).
QLoRA — Quantized LoRA
Combine 4-bit quantization of base model + LoRA adapters. Enables fine-tuning 7B–70B parameter models on a single GPU (16–24GB VRAM). The key breakthrough that democratized LLM fine-tuning.
LoRA Fine-tuning with PEFT
For 7B+ models, use QLoRA (4-bit + LoRA). Use r=8 or r=16 to start. Larger r = more capacity but more parameters.
Finished reading? Mark it complete to earn your XP.