Lab Specification — Deep-Dive FTDD-03: Feel the Unsloth Win (QLoRA: Unsloth vs TRL baseline)

# Unsloth (CUDA only)
pip install "unsloth[cu121-torch240]" --no-deps
pip install --no-deps "trl<0.9" "peft" "accelerate" "bitsandbytes"
pip install "transformers" "datasets" "xformers" "triton"

# Verify Unsloth sees the GPU
python -c "from unsloth import FastLanguageModel; print('Unsloth OK')"

# baseline_trl.py
import torch, time
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer

model_id = "openbmb/MiniCPM5-1B"  # or a 7B if you have 24GB
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, load_in_4bit=True, device_map="auto"
)
model = get_peft_model(model, LoraConfig(
    r=8, lora_alpha=16, target_modules=["q_proj","k_proj","v_proj","o_proj"],
    lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",
))

ds = load_dataset("AI-ModelScope/UltraChat-200k", split="train_sft").select(range(1000))

t0 = time.time()
trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, dataset=ds,
    max_seq_length=512,
    args=TrainingArguments(
        output_dir="./trl-baseline",
        num_train_epochs=1, per_device_train_batch_size=4,
        gradient_accumulation_steps=2, learning_rate=1e-4,
        logging_steps=10, save_strategy="no",
    ),
)
trainer.train()
elapsed_trl = time.time() - t0

# Peak memory
mem_trl = torch.cuda.max_memory_allocated() / 1e9
print(f"TRL:    {elapsed_trl:.1f}s, peak VRAM {mem_trl:.2f} GB")

# unsloth_run.py
import torch, time
from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer
from transformers import TrainingArguments

model_id = "openbmb/MiniCPM5-1B"  # SAME model as baseline
model, tokenizer = FastLanguageModel.from_pretrained(
    model_id=model_id, max_seq_length=512, dtype=None, load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
    model,
    r=8, lora_alpha=16, target_modules=[
        "q_proj","k_proj","v_proj","o_proj",
    ],
    lora_dropout=0.05, bias="none",
)

ds = load_dataset("AI-ModelScope/UltraChat-200k", split="train_sft").select(range(1000))

t0 = time.time()
trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, dataset=ds,
    max_seq_length=512,
    args=TrainingArguments(
        output_dir="./unsloth-run",
        num_train_epochs=1, per_device_train_batch_size=4,
        gradient_accumulation_steps=2, learning_rate=1e-4,
        logging_steps=10, save_strategy="no",
    ),
)
trainer.train()
elapsed_unsloth = time.time() - t0

mem_unsloth = torch.cuda.max_memory_allocated() / 1e9
print(f"Unsloth: {elapsed_unsloth:.1f}s, peak VRAM {mem_unsloth:.2f} GB")

speedup = elapsed_trl / elapsed_unsloth
mem_reduction = (mem_trl - mem_unsloth) / mem_trl * 100
print(f"Speedup:        {speedup:.2f}x")
print(f"Memory reduction: {mem_reduction:.1f}%")

# Quick inference check — both adapters, same prompt
prompt = "List three planets. Respond as JSON."
# (load each adapter and generate — details depend on your merge path)

# Unsloth's GGUF export (Dynamic 4.0)
model.save_pretrained_gguf(
    "minicpm-unsloth-dynamic4",
    quantization_method=["dynamic_4.0"],  # intelligent per-layer
)

ollama create minicpm-ftdd03 -f Modelfile  # pointing at the dynamic_4.0 GGUF
ollama run minicpm-ftdd03 "List three planets. Respond as JSON."

# Lab Specification — Deep-Dive FTDD-03: Feel the Unsloth Win (QLoRA: Unsloth vs TRL baseline)

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Deep-Dive**: FTDD-03 — Unsloth
**Duration**: 30–45 minutes (a single-GPU benchmarking lab)
**Environment**: A machine with a single NVIDIA CUDA GPU (≥12GB VRAM recommended; ideally 24GB RTX 4090/3090 for the full 7B comparison). This lab is CUDA-focused — see the Apple Silicon note at the end. Python 3.10+.

---

## Learning objectives

By the end of this lab you will have:

1. **Run the same QLoRA fine-tune twice** — once through Unsloth, once through a TRL baseline — on the same model, the same data, and the same hyperparameters.
2. **Measured the speed and memory difference** — the Unsloth win, felt directly on your own GPU rather than asserted from a benchmark table.
3. **Confirmed the "same result, faster" property** — that Unsloth produces an equivalent adapter, not a different one.
4. **Exported a Dynamic 4.0 GGUF** (if time permits) — Unsloth's intelligent per-layer quantization, for local serving.

This lab is the empirical anchor for the deep-dive's central claim: Unsloth is ~2x faster and ~60% cheaper in memory, with zero algorithmic change. You will feel the win, not just read about it.

---

## Phase 0 — Set up (5 min)

Install both frameworks. Use a fresh venv to avoid dependency conflicts.

```bash
# Unsloth (CUDA only)
pip install "unsloth[cu121-torch240]" --no-deps
pip install --no-deps "trl<0.9" "peft" "accelerate" "bitsandbytes"
pip install "transformers" "datasets" "xformers" "triton"

# Verify Unsloth sees the GPU
python -c "from unsloth import FastLanguageModel; print('Unsloth OK')"
```

> **Version note:** Unsloth's install commands evolve. Check `github.com/unslothai/unsloth` for the current instruction matching your CUDA (cu121/cu124) and PyTorch version. The lab uses QLoRA, which Unsloth supports out of the box.

Confirm you have a model to fine-tune. **MiniCPM5-1B** (FTDD-01's base) is the recommended choice for a fast lab; if you have a 24GB GPU, you can use a 7B model (e.g., Qwen2.5-7B or Llama-3.1-8B) to see the win at the scale where it matters most.

---

## Phase 1 — The TRL baseline (10 min)

Run a short QLoRA fine-tune via TRL (the substrate library). This is your baseline — the "without Unsloth" measurement.

```python
# baseline_trl.py
import torch, time
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer

model_id = "openbmb/MiniCPM5-1B"  # or a 7B if you have 24GB
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, load_in_4bit=True, device_map="auto"
)
model = get_peft_model(model, LoraConfig(
    r=8, lora_alpha=16, target_modules=["q_proj","k_proj","v_proj","o_proj"],
    lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",
))

ds = load_dataset("AI-ModelScope/UltraChat-200k", split="train_sft").select(range(1000))

t0 = time.time()
trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, dataset=ds,
    max_seq_length=512,
    args=TrainingArguments(
        output_dir="./trl-baseline",
        num_train_epochs=1, per_device_train_batch_size=4,
        gradient_accumulation_steps=2, learning_rate=1e-4,
        logging_steps=10, save_strategy="no",
    ),
)
trainer.train()
elapsed_trl = time.time() - t0

# Peak memory
mem_trl = torch.cuda.max_memory_allocated() / 1e9
print(f"TRL:    {elapsed_trl:.1f}s, peak VRAM {mem_trl:.2f} GB")
```

**Record:** `elapsed_trl` (seconds) and `mem_trl` (GB). Note these are your baseline numbers.

---

## Phase 2 — The Unsloth run (10 min)

Now run the SAME fine-tune through Unsloth — same model, same 1000 examples, same LoRA rank (8), same hyperparameters.

```python
# unsloth_run.py
import torch, time
from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer
from transformers import TrainingArguments

model_id = "openbmb/MiniCPM5-1B"  # SAME model as baseline
model, tokenizer = FastLanguageModel.from_pretrained(
    model_id=model_id, max_seq_length=512, dtype=None, load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
    model,
    r=8, lora_alpha=16, target_modules=[
        "q_proj","k_proj","v_proj","o_proj",
    ],
    lora_dropout=0.05, bias="none",
)

ds = load_dataset("AI-ModelScope/UltraChat-200k", split="train_sft").select(range(1000))

t0 = time.time()
trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, dataset=ds,
    max_seq_length=512,
    args=TrainingArguments(
        output_dir="./unsloth-run",
        num_train_epochs=1, per_device_train_batch_size=4,
        gradient_accumulation_steps=2, learning_rate=1e-4,
        logging_steps=10, save_strategy="no",
    ),
)
trainer.train()
elapsed_unsloth = time.time() - t0

mem_unsloth = torch.cuda.max_memory_allocated() / 1e9
print(f"Unsloth: {elapsed_unsloth:.1f}s, peak VRAM {mem_unsloth:.2f} GB")
```

**Record:** `elapsed_unsloth` and `mem_unsloth`.

---

## Phase 3 — Compute the win (5 min)

Calculate the speedup and memory reduction:

```python
speedup = elapsed_trl / elapsed_unsloth
mem_reduction = (mem_trl - mem_unsloth) / mem_trl * 100
print(f"Speedup:        {speedup:.2f}x")
print(f"Memory reduction: {mem_reduction:.1f}%")
```

**Record the numbers.** The expected result is roughly 1.5–2.5x speedup and 40–70% memory reduction — the exact figures depend on your GPU, the model size, and the sequence length. The point is the direction and magnitude, not a precise match to the headline 2x/60%.

If your numbers are far off (e.g., Unsloth is slower), the most likely causes are: (a) version mismatch — reinstall Unsloth per the current repo instructions; (b) the baseline run benefited from a warm cache while Unsloth's was cold — run both twice and take the second; (c) you're on an unsupported GPU architecture.

---

## Phase 4 — Confirm "same result, faster" (5 min)

The deep-dive's claim is that Unsloth produces an *equivalent* adapter, not a different one. Confirm this qualitatively: run the same inference prompt against both adapters (or the merged models) and compare.

```python
# Quick inference check — both adapters, same prompt
prompt = "List three planets. Respond as JSON."
# (load each adapter and generate — details depend on your merge path)
```

The expected finding: both adapters produce qualitatively similar outputs (both steer format, both built on the same base). The difference between them is within the noise of a 1000-example, 1-epoch LoRA — not a structural difference. This confirms Unsloth changed the *speed*, not the *result*.

---

## Phase 5 — Dynamic 4.0 GGUF export (optional, 10 min)

If time permits, export your Unsloth-fine-tuned model to GGUF using Dynamic 4.0 quantization — the intelligent per-layer export (Layer 4 of the stack).

```python
# Unsloth's GGUF export (Dynamic 4.0)
model.save_pretrained_gguf(
    "minicpm-unsloth-dynamic4",
    quantization_method=["dynamic_4.0"],  # intelligent per-layer
)
```

Then serve it via Ollama (FT20):

```bash
ollama create minicpm-ftdd03 -f Modelfile  # pointing at the dynamic_4.0 GGUF
ollama run minicpm-ftdd03 "List three planets. Respond as JSON."
```

Observe: the Dynamic 4.0 GGUF is the same approximate size as a uniform 4-bit GGUF but should produce slightly better quality (cleaner JSON, fewer artifacts). This is the Layer-4 extension of Unsloth's optimization.

---

## Apple Silicon note

This lab is CUDA-focused. On Apple Silicon (M-series Mac), Unsloth's kernels do not apply — you will not see the speedup. Instead:

1. Run the **TRL baseline** (Phase 1) under MLX or `transformers` (Apple Silicon path) to get a baseline time.
2. Skip the Unsloth run (Phase 2) — it will not accelerate on Metal.
3. Read the Phase 3 numbers from a CUDA-equipped peer or a hosted run, and internalize the win conceptually. The point of the lab — feeling the Unsloth advantage — requires CUDA. Apple Silicon users should complete the MLX equivalent in FT20 for the Metal-optimized path.

---

## Deliverables

Submit `ftdd03-unsloth-benchmark.md` containing:

- [ ] The Phase 1 TRL baseline numbers: elapsed time (s), peak VRAM (GB).
- [ ] The Phase 2 Unsloth numbers: elapsed time (s), peak VRAM (GB).
- [ ] The Phase 3 computed win: speedup (x), memory reduction (%).
- [ ] The Phase 4 qualitative confirmation: did both adapters produce equivalent outputs? (2–3 sentences.)
- [ ] (Optional) The Phase 5 Dynamic 4.0 GGUF: file size and a one-line note on quality vs. a uniform quant.
- [ ] A 2–3 sentence reflection: given your hardware, when would you choose Unsloth vs. Axolotl vs. TRL?

---

## Solution key

The expected finding is a Unsloth speedup in the 1.5–2.5x range and a memory reduction of 40–70% versus the TRL baseline, with qualitatively equivalent adapter outputs (Phase 4). The exact numbers depend on GPU/model/seq-length, but the direction is stable: Unsloth is faster and cheaper, and it does not change the result.

The reflection should name the decision heuristic: single GPU + speed/memory/GGUF export → Unsloth (this lab's tool); multi-GPU + production configs → Axolotl; custom training loop + maximum control → TRL (the baseline in this lab). If the student has a single GPU, Unsloth is their default; the TRL baseline they ran is the "substrate" path for when they need control Unsloth's abstractions don't expose.

If the speedup is absent or negative, check (in order): Unsloth install matches CUDA/PyTorch version; the GPU is NVIDIA (not Apple Silicon/AMD without ROCm support); both runs used the same model/data/hyperparameters (apples-to-apples); the cache was warm for both (run twice, take the second).

---

## Stretch goals

1. **Run at 7B.** If you have a 24GB GPU, repeat the comparison on a 7B model (Qwen2.5-7B or Llama-3.1-8B). The Unsloth win is often more dramatic at 7B — this is the scale where Unsloth's "fits on an RTX 4090" claim is load-bearing. Observe whether the TRL baseline even fits in 24GB for 7B; if it doesn't, that IS the Unsloth win (it enables a run the baseline cannot do at all).
2. **Compare Dynamic 4.0 to uniform GGUF.** Export the same model both ways (Dynamic 4.0 and a standard uniform 4-bit) and run a small quality comparison (e.g., a set of 10 prompts, rated by hand or by an LLM judge). Quantify the quality-per-byte advantage Dynamic 4.0 provides.
3. **Profile where the time goes.** Use `torch.profiler` on the TRL baseline to identify which operations dominate (attention? linear layers? optimizer step?). Then confirm Unsloth's hand-written kernels target exactly those operations — this makes the "engineering win" claim concrete: you can see the specific kernels Unsloth rewrote.