{ "module": "FT08 — LoRA & QLoRA", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 45, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "State the LoRA decomposition equation and the initialization of each matrix.", "options": [ "W = W0 + BA, where W0 is trainable, B is random-init, A is zero-init.", "W = W0 + BA, where W0 is FROZEN (pretrained), B is ZERO-init (d×r), and A is random-Gaussian-init (r×k). At step 0, BA=0 so the model behaves exactly like the base.", "W = W0 - BA, where both B and A are random-init and W0 is updated by gradient descent.", "W = BA, where the pretrained W0 is discarded and replaced by the low-rank product." ], "answer_index": 1, "rationale": "LoRA: W = W0 + BA. W0 is the frozen pretrained weight; B (d×r) is initialized to ZERO and A (r×k) to random Gaussian, so at step 0 BA=0 and the adapter contributes nothing — the model is identical to the base. This makes LoRA a residual steer: it cannot degrade the base at initialization, a guarantee full fine-tuning lacks." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Name QLoRA's three innovations (Dettmers et al., 2023).", "options": [ "FP8 training, gradient checkpointing, FlashAttention.", "NF4 (NormalFloat 4-bit) quantization, double quantization, and paged optimizers.", "INT4 quantization, mixed precision, and CPU offloading of weights.", "GGUF quantization, KV-cache compression, and speculative decoding." ], "answer_index": 1, "rationale": "QLoRA's three innovations: (1) NF4 — 4-bit quantile bins matched to the normal distribution, information-theoretically optimal for Gaussian weights; (2) double quantization — quantize the quantization constants themselves to 8-bit, saving ~0.37 bits/param; (3) paged optimizers — NVIDIA Unified Memory pages optimizer states to CPU to avoid OOM spikes during checkpointing. All three compose; drop any one and the method degrades." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the modern default for LoRA target modules, and how does it differ from the original LoRA paper?", "options": [ "Original: all MLP modules. Modern: attention only (q_proj, v_proj).", "Original and modern both use attention-only (q_proj, v_proj).", "Original (2021): attention only, typically ['q_proj','v_proj']. Modern default (2024+): ALL attention + MLP projections — q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.", "Modern default: only the embedding layer and lm_head." ], "answer_index": 2, "rationale": "The 2021 LoRA paper targeted attention only (q_proj, v_proj) — cheap but weaker. The modern 2024+ default is ALL linear projections: the four attention modules (q,k,v,o_proj) PLUS the three MLP modules (gate,up,down_proj). All-linear is stronger because behavior steering touches the MLP feature pathway, not just attention. It is the single biggest quality lever in the LoRA config." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You are configuring a LoRA adapter for a Qwen2.5-7B SFT. You want the modern production default. Which LoraConfig is correct?", "options": [ "r=8, lora_alpha=8, target_modules=['q_proj','v_proj'], lora_dropout=0.3.", "r=16, lora_alpha=32, target_modules=['q_proj','k_proj','v_proj','o_proj','gate_proj','up_proj','down_proj'], lora_dropout=0.05.", "r=128, lora_alpha=16, target_modules=['q_proj','v_proj'], lora_dropout=0.", "r=16, lora_alpha=128, target_modules=['embed_tokens'], lora_dropout=0.5." ], "answer_index": 1, "rationale": "The modern production default: r=16, alpha=32 (the alpha ≈ 2×r convention), ALL seven linear target modules (q,k,v,o_proj + gate,up,down_proj), dropout=0.05. Option (a) is attention-only with alpha==r (too quiet) and excessive dropout. Option (c) is attention-only at r=128 (overfit + memory). Option (d) targets embeddings only and has alpha 8×r (overcorrection/forgetting)." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "Your QLoRA run fits comfortably in steady-state VRAM (~12 GB on a 24GB card) but OOMs partway through training, always around a checkpoint save. What is the cause and the fix?", "options": [ "Activations too large; reduce context length.", "The OOM SPIKE during checkpointing. The optimizer pages state to CPU/disk causing a momentary memory spike exceeding steady-state. Fix: use optim='paged_adamw_8bit' (bitsandbytes) — NVIDIA Unified Memory transparently pages optimizer states under pressure, absorbing the spikes. This is QLoRA innovation 3.", "Learning rate too high; reduce it 10×.", "LoRA rank too high; reduce r to 4." ], "answer_index": 1, "rationale": "This is the OOM-spike problem paged optimizers solve. Steady-state fits, but checkpointing causes a transient memory spike that exceeds the budget and crashes the job. The fix is paged_adamw_8bit, which uses NVIDIA Unified Memory to page optimizer states to CPU under pressure and back in when needed — QLoRA innovation 3. It is NOT an activation or learning-rate problem; the timing (always at checkpoint) is the diagnostic." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You trained a QLoRA adapter and want to deploy it as a single self-contained model that vLLM can load with no PEFT dependency. What do you do?", "options": [ "Save the adapter with save_pretrained and load it via PeftModel.from_pretrained at serve time.", "Call model.merge_and_unload() to bake the adapter into the base (W0 := W0 + (α/r)BA), then save the merged model. This produces one self-contained model with no adapter runtime overhead — load directly with AutoModelForCausalLM.from_pretrained. Re-quantize (GGUF/AWQ) afterward for deployment.", "Export the adapter to ONNX and serve via TorchServe.", "Concatenate the adapter weights onto the base's safetensors file manually." ], "answer_index": 1, "rationale": "merge_and_unload() computes W0 := W0 + (α/r)BA for every adapter and produces a single self-contained model with no PEFT dependency at serve time — best for deployment. Option (a) is the hot-swap path (keep adapter separate), not the single-artifact deploy path. Note: on a 4-bit base the merge dequantizes, so the merged model is larger than the 4-bit base — re-quantize (GGUF/AWQ) afterward for production." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You call model.print_trainable_parameters() after get_peft_model on a 7B QLoRA at r=16, all-linear. Which result is healthy and expected?", "options": [ "trainable params: 7,000,000,000 || all params: 7,000,000,000 || trainable%: 100.0", "trainable params: ~40,000,000 || all params: ~7,000,000,000 || trainable%: ~0.5–1.0", "trainable params: 0 || all params: 7,000,000,000 || trainable%: 0.0", "trainable params: ~40,000,000 || all params: ~40,000,000 || trainable%: 100.0" ], "answer_index": 1, "rationale": "A healthy 7B QLoRA at r=16, all-linear shows ~40M trainable params out of ~7B total, i.e. ~0.5–1.0% trainable — the LoRA promise. If you see 100% (option a), you forgot to attach the adapter or the base was not frozen. If you see 0% (option c), the adapter attached but nothing is trainable (misconfigured task_type or target_modules). Option (d) means only the adapter params exist, which is impossible if the base loaded." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You are writing a 4-bit loading config for QLoRA. Which BitsAndBytesConfig correctly sets NF4 + double quantization + bf16 compute?", "options": [ "BitsAndBytesConfig(load_in_8bit=True, llm_int8_skip_modules=['lm_head']).", "BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16).", "BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type='fp4', bnb_4bit_use_double_quant=False, bnb_4bit_compute_dtype=torch.float32).", "BitsAndBytesConfig(load_in_16bit=True, bnb_4bit_quant_type='nf4')." ], "answer_index": 1, "rationale": "The correct QLoRA base config: load_in_4bit=True, bnb_4bit_quant_type='nf4' (NormalFloat 4-bit, innovation 1), bnb_4bit_use_double_quant=True (innovation 2), bnb_4bit_compute_dtype=torch.bfloat16 (compute dequantized in bf16). Option (a) is INT8 (not QLoRA). Option (c) uses fp4 (uniform, not NF4) and disables double quant — worse quality and more overhead. Option (d) is 16-bit (plain LoRA, not QLoRA)." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "Your 500-sample style-steering SFT loss plateaus at ~1.4 and the target style never quite lands in the model's output — it is weakly present but unreliable. Your config is r=4, all-linear, α=8. What is the most likely cause and the fix?", "options": [ "Overfitting; raise dropout to 0.3.", "Rank TOO LOW (underfit). r=4 gives the adapter too few directions of change to express the style shift; loss plateaus high because the adapter cannot represent the needed perturbation. Fix: raise r to 16 (and alpha to 32, keeping α≈2×r) and compare.", "Learning rate too high; reduce to 1e-5.", "Wrong optimizer; switch to full AdamW (32-bit)." ], "answer_index": 1, "rationale": "This is the underfit arm of the rank anti-pattern. r=4 is too few directions; the adapter literally cannot express the required change, so loss plateaus high (~1.4) and the style is weakly present. Fix: raise r (4 → 16) and alpha accordingly (8 → 32, keeping the α≈2×r convention). The overfit arm is the opposite (r=128 on small data, loss crashes too low). The diagnostic is the high plateau + weak style, pointing to underfit, not overfit." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does the FT00 thesis ('fine-tuning steers behavior; it does not teach knowledge') imply that a LoRA adapter at <1% of parameters can match full fine-tuning on a style/persona SFT task?", "options": [ "Because LoRA uses a higher learning rate than full FT, so it converges faster.", "Because style/persona steering is a LOW-RANK operation. The base already possesses the vocabulary and capability from pretraining; fine-tuning only redirects probability mass to use it reliably and in the target format. The intrinsic dimension hypothesis (Aghajanyan) predicts the useful change lives in a small subspace, which B·A captures. Full FT's extra capacity is unused on such tasks.", "Because LoRA targets only the most important layers, skipping unimportant ones.", "Because the 4-bit quantization in QLoRA removes redundant parameters that full FT would waste time updating." ], "answer_index": 1, "rationale": "The FT00 thesis + intrinsic dimension hypothesis together explain LoRA's effectiveness. Steering (format, persona, style) redirects probability mass the base already has — it is intrinsically low-rank. Aghajanyan showed the useful changes live in a small subspace, so B·A at r=16 captures them. Full FT's extra capacity goes unused on steering tasks; you pay 10× VRAM for no observable gain. Had the task required full-FT-scale updates, that would be evidence of knowledge injection (teaching), not steering — and you would need RAG or continued pretraining, not a bigger adapter." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "A colleague sets r=128 'to get closer to full fine-tuning quality.' Based on Shuttleworth et al. 2024, why is this reasoning flawed, and what should they do instead?", "options": [ "It is correct; higher rank always moves LoRA closer to full FT.", "Shuttleworth et al. showed LoRA and full FT produce STRUCTURALLY DIFFERENT weight matrices — LoRA finds a different low-rank solution via different geometry, NOT an approximation of full FT. Raising r to 128 does NOT converge toward full FT; it gives a poorly-regularized, high-memory LoRA. If they genuinely need full-FT-style updates, they should DO full FT (Module FT10). Otherwise, start at r=16.", "They should use r=256 instead; that is the threshold where LoRA equals full FT.", "They should switch to attention-only targeting to compensate for the high rank." ], "answer_index": 1, "rationale": "Shuttleworth et al. 2024 (arXiv:2410.21228) found LoRA and full FT produce structurally different weights — they reach similar behavior via different geometry, not approximation. So 'more rank = closer to full FT' is a category error: r=128 is not a better approximation of full FT, it is a worse LoRA (overfit + memory, paying full-FT cost for LoRA's inherently different solution). If full-FT-style updates are genuinely needed, do full FT. Otherwise start at r=16 and escalate only with measured underfit evidence." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Two teams train adapters on the same SFT task with the SAME trainable-param count. Team A uses r=16, all-linear (7 modules). Team B uses r=56, attention-only (2 modules). Both report similar loss. Team A's model generalizes better on unseen prompts. Why?", "options": [ "Team A used a better learning rate.", "Team B's data was lower quality.", "Capacity distribution matters more than total param count. All-linear distributes capacity across attention AND the MLP feature pathway, where behavior steering actually lives. Attention-only concentrates capacity in attention alone, missing the MLP features. Even at comparable param budgets, all-linear matches or beats attention-only because it targets the modules that matter for steering. This is why all-linear is the modern default.", "Team A's rank is lower, which always generalizes better." ], "answer_index": 2, "rationale": "The comparison isolates target-module choice (the params are matched). All-linear (Team A) distributes capacity across attention + MLP; attention-only (Team B) concentrates it in attention. Behavior steering touches the MLP feature pathway, so all-linear targets where the change actually lives. Team B compensates with higher rank but still misses the MLP modules — worse generalization at the same param budget. This is the empirical basis for the all-linear modern default (and the FT08 claim that target modules are the biggest quality lever)." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "You trained a QLoRA adapter on a 4-bit base, called merge_and_unload(), and the resulting merged model is ~14 GB (FP16) — larger than the ~3.5 GB 4-bit base you trained on. Is this a bug? What happened, and what is the correct production path?", "options": [ "Yes, it is a bug; merge should preserve 4-bit. Re-run with a newer bitsandbytes.", "Not a bug. merge_and_unload() DEQUANTIZES the merge: it computes W0 := W0 + (α/r)BA and writes the result at full precision (FP16/BF16), so the merged model is larger than the 4-bit base. The correct production path is to RE-QUANTIZE the merged model afterward (GGUF for Ollama / AWQ for vLLM). Merge bakes in the steer; re-quantize compresses for deploy. This two-step is the standard Layer 2 → Layer 4 path.", "Not a bug; the merged model should be discarded and the adapter served separately.", "It is a bug; you should have merged before quantizing the base." ], "answer_index": 1, "rationale": "Not a bug — expected behavior. merge_and_unload() on a 4-bit base dequantizes: the merged weights are stored at FP16/BF16, so the model is larger than the 4-bit training base. The production path is merge THEN re-quantize: bake in the steer (merge), then compress for deployment (GGUF/AWQ). This is the standard Layer 2 (adapter) → Layer 4 (export) path of the FT00 steering stack. Serving the merged FP16 model directly is fine if you have the VRAM; otherwise re-quantize." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "A team claims: 'QLoRA is just a compromise for people who cannot afford real fine-tuning. For production quality we always do full FT.' Evaluate this claim against the FT00/FT08 framework.", "options": [ "Correct — full FT is always higher quality; QLoRA is only for prototyping.", "Wrong on two counts. (1) For STEERING tasks (FT00: format, persona, preference, reasoning activation), QLoRA is NOT a compromise — it is the correct tool, because steering is low-rank and full FT's extra capacity goes unused. (2) Shuttleworth 2024 shows LoRA/full-FT find DIFFERENT solutions, not approximations — so 'full FT is strictly better' is false for steering. Full FT is the exception (Module FT10), justified by specific evidence of knowledge injection, not the production default. Reaching for full FT by default pays 10× for unobservable gains.", "Correct for models over 7B; QLoRA only works for small models.", "Correct — QLoRA always degrades quality by at least 5%." ], "answer_index": 1, "rationale": "The claim is backwards. For steering tasks (the vast majority of fine-tuning), QLoRA is the correct tool: steering is low-rank (intrinsic dimension), so <1% params suffices and full FT's extra capacity is wasted. Shuttleworth 2024 shows LoRA is not a degraded approximation of full FT — it is a different solution. Full FT is justified only by specific evidence (knowledge injection, Module FT10), not as a default. Treating QLoRA as inferior is the 'reaching for full FT by default' anti-pattern from FT01 — paying 10× VRAM for gains you cannot observe in the output." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "You need a single model that can be a JSON API, a pirate persona, AND a medical-triage assistant — three styles, hot-swapped at inference without reloading the base. Architect this with the FT08 tools.", "options": [ "Train one full-FT model on all three styles merged together.", "Train ONE base, THREE separate LoRA adapters (JSON, pirate, triage), keep each as adapter_model.safetensors (<100 MB each), and hot-swap at runtime via PeftModel.from_pretrained(base, adapter_path). One base in memory; swap adapters like loading config files. This is the hot-swap path (NOT merge_and_unload, which bakes one adapter in).", "Train three separate full-FT models and load whichever is needed.", "Use a single r=256 adapter trained on all three styles simultaneously." ], "answer_index": 1, "rationale": "This is the hot-swap use case that LoRA's swappability was designed for. One base, three adapters (each <100 MB, trained via QLoRA), swapped at runtime via PeftModel.from_pretrained. The base stays loaded; only the tiny adapter changes — like loading a config file. Option (a) blends the styles (interference). Option (c) triples base VRAM. Option (d) risks interference and uses a poorly-justified r=256. merge_and_unload (the deploy path) bakes ONE adapter in and is wrong here — you need the adapter kept SEPARATE for hot-swapping." } ] }