zucchetti-llm-course

Lab 08: LoRA Fine-Tuning

Lecture: 9, Efficient Fine-Tuning (LoRA and QLoRA) Duration: 45 minutes Difficulty: Advanced

Objectives

• Format an instruction dataset using chat templates (Slide 17)
• Configure LoRA adapters with rank, alpha, and target modules (Slide 10-11)
• Understand how LoRA reduces trainable parameters by >99% (Slide 8)
• Use practical tricks: packing, NEFTune, gradient accumulation (Slides 15-19)
• Compare a base model vs fine-tuned model (before/after)
• Merge and export an adapter for deployment (Slide 27)

Prerequisites

Requirement	How to Check
Python 3.10+	python --version
Dependencies	pip install transformers peft datasets trl torch rich accelerate
Hardware	Colab T4 GPU (recommended) or Apple Silicon (MPS)

Colab Recommended: This lab is designed for Google Colab with a T4 GPU (free tier). It also works on Apple Silicon or CPU, but training will be slower. We use Qwen2.5-0.5B (tiny model) so it runs everywhere.

Quick Start

Option A: Google Colab (Recommended)

1. Upload lab_08_lora.ipynb and data/ folder to Colab
2. Select Runtime → Change runtime type → T4 GPU
3. Run all cells

Option B: Local

cd lab_08_lora_finetuning
pip install transformers peft datasets trl torch rich accelerate
python lab_08_lora.py

Files

File	Description
lab_08_lora.ipynb	Main notebook — Colab-ready, step-by-step
lab_08_lora.py	Python script version (same content)
solutions.py	Solutions for TODO 1 (chat format) and TODO 2 (LoRA config)
data/medical_instructions.jsonl	10 medical Q&A pairs for training

Tasks

Task 1: Format the Dataset with Chat Templates

Find the format_dataset() function. Convert raw instruction data into the messages format:

messages = [
    {"role": "system", "content": "You are a medical assistant..."},
    {"role": "user", "content": instruction + context},
    {"role": "assistant", "content": response}
]

Why? Models expect conversations in a specific chat template format. Using the wrong format = broken model (Slide 17).

Task 2: Configure LoRA

Find the get_lora_config() function. Return a LoraConfig with:

• r=16 — rank (adapter matrix size)
• lora_alpha=32 — scaling factor (alpha/r = 2.0)
• target_modules=["q_proj", "k_proj", "v_proj", "o_proj"] — all attention projections
• lora_dropout=0.05
• task_type=TaskType.CAUSAL_LM

Why? These parameters control how much the adapter affects the model’s behavior (Slide 10-11).

What You’ll See (No Code Required)

The lab also demonstrates these concepts automatically:

• Packing analysis — how much compute is wasted on padding (Slide 16)
• Parameter breakdown — trainable vs frozen params with real numbers
• Before/After comparison — same questions, different responses
• Adapter merging — save and merge the adapter for deployment (Slide 27)

Expected Output

✅ GPU: Tesla T4 (15.0 GB)
✅ Model loaded! Total parameters: 494,033,920

═══════════════════════════════════════════════════
BEFORE Fine-Tuning (Base Model)
═══════════════════════════════════════════════════
Q1: A 45-year-old male presents with crushing chest pain...
A:  [base model response — may be generic or verbose]

trainable params: 1,572,864 || all params: 495,606,784 || trainable%: 0.3173

🚀 Starting Training...
Step 5:  loss=2.1423
Step 10: loss=1.8912
Step 50: loss=0.9234
✅ Training Complete! (120.5 seconds)

═══════════════════════════════════════════════════
AFTER Fine-Tuning (LoRA Adapted Model)
═══════════════════════════════════════════════════
Q1: A 45-year-old male presents with crushing chest pain...
  BEFORE: [generic response]
  AFTER:  [more focused, medical-style response]

✅ LoRA adapter saved to ./medical_lora_adapter/
   Adapter size: 6.2 MB

Discussion Questions

After completing the lab, discuss with your team:

1. What would happen if we increased rank from 16 to 64?
2. How would you detect overfitting? (Hint: Slide 22)
3. Should we fine-tune or use RAG to add Zucchetti’s medical knowledge?
4. For a production deployment, what would you change?

🆘 Troubleshooting

Issue	Solution
Missing dependencies	Run pip install transformers peft datasets trl torch accelerate
Killed: 9 (OOM Error)	Reduce per_device_train_batch_size to 1
Very slow training	Ensure GPU is selected (Colab: Runtime → Change runtime type)
SFTConfig not found	Update trl: pip install -U trl>=0.8.0
Chat template error	Ensure you’re using Qwen2.5-0.5B-Instruct (not the base version)