Course: Course 3 — LLM Fine-Tuning Masterclass Deep-Dive: FTDD-03 Duration: 45 minutes Level: Senior Engineer and above Prerequisites: FT00 (Steering Stack), FT01 (VRAM Math), FT08 (LoRA/QLoRA)
The single-GPU speed and memory optimizer. Hand-written Triton kernels and manual autograd make a 7B QLoRA fit on a $1,500 RTX 4090 — the tool that democratized consumer-GPU fine-tuning.
After this deep-dive, you will be able to:
Unsloth is a fine-tuning library built around a single thesis: the existing deep-learning frameworks (PyTorch, Transformers, TRL) leave substantial performance on the table because their kernels are generic. Unsloth rewrites the performance-critical paths by hand — specifically, the forward and backward passes — and in doing so achieves roughly 2x training speed and 60% lower memory usage on a single GPU. No algorithmic change. No new fine-tuning method. Pure systems engineering, aimed at the bottleneck that actually matters for the person paying for the GPU.
| Metric | Value |
|---|---|
| Type | Single-GPU speed/memory optimizer |
| Speedup | ~2x vs. baseline (e.g., TRL) |
| Memory reduction | ~60% less VRAM |
| Mechanism | Hand-written Triton kernels, manual autograd, 4-bit optimizers |
| Headline enabler | 7B QLoRA on a $1,500 RTX 4090 |
| Platform | CUDA-focused (NVIDIA) |
| Apple Silicon | Not the target — use MLX (FT20) |
| Recent feature | Dynamic 4.0 GGUF quants (intelligent per-layer quantization) |
Unsloth matters because it is the tool that made consumer-GPU fine-tuning practical at the 7B scale. Before Unsloth, a 7B QLoRA required a more expensive GPU or aggressive compromises. After Unsloth, a single RTX 4090 — roughly $1,500 — runs a 7B QLoRA comfortably. That is the difference between "fine-tuning is something labs do" and "fine-tuning is something I do, on my desk, this afternoon." For a course built on the thesis that fine-tuning steers behavior and that students should feel that thesis by running experiments (FT00), Unsloth is the tool that makes the curriculum executable on a budget.
Unsloth's performance gains come from three engineering decisions, none of which is a new algorithm. This is important: Unsloth does not change what fine-tuning does. It changes how fast and how cheaply the same fine-tuning runs.
PyTorch and Transformers use general-purpose kernels — code written to handle many cases reasonably well. Unsloth replaces the performance-critical kernels (the attention computation, the linear-layer forward and backward passes) with hand-written Triton kernels. Triton is a language for writing GPU kernels at a higher level of abstraction than CUDA C, but with near-CUDA performance. Unsloth's kernels are specialized for the exact shapes and operations that fine-tuning actually performs, eliminating the overhead general-purpose kernels carry.
The gain is concrete: faster forward and backward passes, which is where most training time goes.
PyTorch's autograd — the automatic differentiation system that computes gradients during backpropagation — is general and correct, but it builds a computation graph dynamically and materializes intermediate tensors. For the specific case of fine-tuning (especially with LoRA/QLoRA adapters), much of this generality is wasted: the shapes and operations are known in advance.
Unsloth implements manual autograd for its custom kernels — it computes the gradients directly, by hand, for the specific operations, rather than relying on PyTorch to derive and schedule them. This eliminates the intermediate-tensor overhead and the graph-construction cost. The gradients are mathematically identical; the path to computing them is shorter.
The optimizer (AdamW, typically) maintains a state — momentum and variance estimates — for every trainable parameter. At 32-bit precision, this state is a significant memory cost, often larger than the model itself for full fine-tuning.
Unsloth uses 4-bit optimizers: the optimizer state is stored at 4-bit precision rather than 32-bit. This is an 8x reduction in optimizer-state memory, with negligible effect on convergence (the optimizer state is statistical; it tolerates aggressive quantization). Combined with QLoRA's 4-bit base weights (FT08), this is what brings a 7B fine-tune down to a size the RTX 4090 can hold.
This is the point to internalize. Unsloth does not change the fine-tuning algorithm. A QLoRA fine-tune run through Unsloth produces the same kind of adapter — same LoRA rank, same target modules, same learning dynamics — as one run through TRL. The math is the same. What Unsloth changes is the implementation: the kernels that execute the math, the autograd that derives the gradients, the precision at which the optimizer state is stored.
This means Unsloth is a near drop-in acceleration layer. You do not learn a new fine-tuning method to use it. You learn a slightly different API entry point, and your training runs faster and cheaper. The conceptual content of this course — steering vs knowledge, LoRA, DPO, GRPO — is unchanged. Unsloth just makes it runnable on the hardware you actually own.
The headline: Unsloth made 7B QLoRA viable on a $1,500 RTX 4090. This is the democratization claim, and it is literal.
Before Unsloth, 7B fine-tuning on consumer hardware required either a more expensive GPU (24GB+ VRAM, pushing toward the $2,000-$5,000 tier), or aggressive compromises — tiny batch sizes, short sequences, offloading to CPU (which is slow). After Unsloth, the RTX 4090's 24GB is enough to run a 7B QLoRA with reasonable batch sizes and sequence lengths, at roughly twice the speed.
For this course, that is the difference between a curriculum that is theoretically executable and one that is practically executable. When FT00 says "load MiniCPM5-1B, apply a LoRA, feel the steering thesis," Unsloth (or an equivalent optimizer) is what makes that lab completable on a student's own machine in minutes rather than hours. When FT08 teaches QLoRA, Unsloth is the tool that lets a student run the full worked example on a single GPU.
The democratization is not just economic. It is pedagogical. A student who can run a dozen fine-tuning experiments in a day learns the steering-vs-knowledge distinction (FT00) by feel — they see it, repeatedly, on their own hardware. A student who can afford one experiment per day learns it from a textbook. Unsloth is what makes the former possible.
Unsloth is CUDA-focused. Its hand-written kernels target NVIDIA GPUs. This is a deliberate scope choice: NVIDIA dominates the GPU-ML ecosystem, and the kernels are where the performance lives, so Unsloth optimizes for the platform that most users have.
The consequence: Apple Silicon is not Unsloth's target. On an M-series Mac, Unsloth's CUDA-optimized kernels do not apply. Apple Silicon users should reach for MLX instead — Apple's own ML framework, which is designed for the Metal backend and the unified-memory architecture of M-series chips. MLX is the subject of FT20 (serving) and is the Apple-Silicon path for both inference and fine-tuning on Mac.
This is not a defect in Unsloth; it is a scope boundary. The course's guidance:
Unsloth's recent Dynamic 4.0 release introduced intelligent per-layer GGUF quantization — and this is the feature that extends Unsloth's value from training into the export step (Layer 4 of the FT00 stack).
A uniform quantization applies the same precision (e.g., 4-bit) to every layer of the model. This is simple, but suboptimal: some layers are more sensitive to quantization than others (layers involved in attention, or the output projection, often degrade more when quantized aggressively). Uniform quantization over-quantizes the sensitive layers and under-quantizes the robust ones.
Dynamic quantization does the opposite: it analyzes each layer's sensitivity and assigns a higher precision to the sensitive layers and a lower precision to the robust ones, hitting a target average bit-width while preserving more quality. The result: a GGUF file that is the same size as the uniform version but performs measurably better — Unsloth's Dynamic 4.0 quants beat uniform quants at the same file size.
When this matters:
The pattern: Unsloth optimizes training (Layers 2-3 of the stack), and Dynamic 4.0 extends that optimization to the export (Layer 4). A model fine-tuned with Unsloth and exported with Dynamic 4.0 is optimized end-to-end for the single-GPU / local-serving path.
Three frameworks dominate fine-tuning, and the course references all three. The choice is a real engineering decision, and it is decided by your hardware and your need for control.
| Framework | Best for | Multi-GPU? | Control level |
|---|---|---|---|
| Unsloth | Single-GPU, speed/memory constrained, GGUF export | No (single-GPU focused) | Optimized, opinionated |
| Axolotl | Multi-GPU, production configs | Yes | Config-driven, broad |
| TRL | Full control, custom training loops, research | Yes (via Accelerate) | Maximum (it is the library) |
Unsloth — the single-GPU optimizer. Choose it when you have one NVIDIA GPU, you care about speed and memory (who doesn't), and/or you want clean GGUF export (Dynamic 4.0). Its opinionated nature is the tradeoff: it makes decisions for you, which is great when you want to go fast and less great when you want to experiment with the training loop itself.
Axolotl — the multi-GPU config-driven framework. Choose it when you have multiple GPUs, you want production-grade configuration management (YAML configs, reproducible runs), and/or you need to scale beyond what a single GPU can do. FTDD-05 covers Axolotl in depth.
TRL (Transformers Reinforcement Learning) — HuggingFace's fine-tuning library. Choose it when you want maximum control over the training loop, you are implementing a custom or research fine-tuning method, or you are building on top of a fine-tuning stack rather than using one. TRL is the library Axolotl and (often) Unsloth build on. FTDD-04 covers TRL in depth.
The heuristic:
These are not exclusive. A common pattern is to prototype in Unsloth (fast iteration on one GPU) and scale to Axolotl (multi-GPU) for the production run. And TRL is the substrate underneath — the thing you reach for when neither Unsloth's nor Axolotl's abstractions fit.
Unsloth is CUDA-focused. Its hand-written Triton kernels target NVIDIA GPUs. On Apple Silicon, Unsloth's optimizations do not apply. Do not install Unsloth on an M-series Mac expecting a speedup — use MLX instead (FT20). This is a scope boundary, not a bug.
Unsloth is faster and cheaper, but it does not change the fine-tuning algorithm. A QLoRA run through Unsloth produces the same kind of adapter as one through TRL. Do not expect Unsloth to produce a "better" model — expect it to produce the same model, faster, with less memory. If the result is better, the cause is your data or hyperparameters (the actual steering wheel), not the kernel.
Unsloth is single-GPU focused. If you have multiple GPUs and you choose Unsloth, you are leaving compute on the table — Axolotl or TRL (with Accelerate) will use the additional GPUs and finish faster. Unsloth is the right choice when the constraint is one GPU's speed and memory, not when you have a cluster.
| Term | Definition |
|---|---|
| Unsloth | Single-GPU fine-tuning optimizer; ~2x speed, ~60% less memory via hand-written kernels |
| Triton kernels | Hand-written GPU kernels (in the Triton language) that replace PyTorch's general-purpose kernels for the performance-critical forward/backward paths |
| Manual autograd | Unsloth computes gradients by hand for its custom kernels, bypassing PyTorch's dynamic graph construction and intermediate-tensor overhead |
| 4-bit optimizers | Optimizer state (AdamW momentum/variance) stored at 4-bit precision — 8x memory reduction with negligible convergence impact |
| Dynamic 4.0 | Unsloth's intelligent per-layer GGUF quantization; assigns precision per-layer by sensitivity, beating uniform quants at the same size |
| MLX | Apple's ML framework for Metal/unified-memory; the Apple Silicon alternative to Unsloth (FT20) |
| Axolotl | The multi-GPU, config-driven fine-tuning framework (FTDD-05) |
| TRL | HuggingFace's fine-tuning library; maximum control, the substrate Unsloth/Axolotl build on (FTDD-04) |
See 07-lab-spec.md. The lab runs the same QLoRA fine-tune through Unsloth and a baseline (TRL), and measures the speed and memory difference — the Unsloth win, felt directly on your own GPU.
unslothai/unsloth; the library and documentation.# Deep-Dive FTDD-03 — Unsloth **Course**: Course 3 — LLM Fine-Tuning Masterclass **Deep-Dive**: FTDD-03 **Duration**: 45 minutes **Level**: Senior Engineer and above **Prerequisites**: FT00 (Steering Stack), FT01 (VRAM Math), FT08 (LoRA/QLoRA) > *The single-GPU speed and memory optimizer. Hand-written Triton kernels and manual autograd make a 7B QLoRA fit on a $1,500 RTX 4090 — the tool that democratized consumer-GPU fine-tuning.* --- ## Learning Objectives After this deep-dive, you will be able to: 1. Explain how Unsloth achieves its ~2x speedup and ~60% memory reduction (hand-written Triton kernels, manual autograd, 4-bit optimizers) and why these are engineering wins, not algorithmic changes. 2. Defend Unsloth's role as the consumer-GPU enabler — the tool that made 7B QLoRA viable on a $1,500 RTX 4090. 3. Identify Unsloth's CUDA focus and explain why Apple Silicon users reach for MLX instead (FT20). 4. Describe Dynamic 4.0 GGUF quants — intelligent per-layer quantization that beats uniform quants — and when it matters. 5. Choose between Unsloth (single-GPU, speed/memory constrained, GGUF export), Axolotl (multi-GPU), and TRL (full control) for a given fine-tuning job. --- ## The Subject **Unsloth** is a fine-tuning library built around a single thesis: the existing deep-learning frameworks (PyTorch, Transformers, TRL) leave substantial performance on the table because their kernels are generic. Unsloth rewrites the performance-critical paths by hand — specifically, the forward and backward passes — and in doing so achieves roughly 2x training speed and 60% lower memory usage on a single GPU. No algorithmic change. No new fine-tuning method. Pure systems engineering, aimed at the bottleneck that actually matters for the person paying for the GPU. | Metric | Value | | --- | --- | | Type | Single-GPU speed/memory optimizer | | Speedup | ~2x vs. baseline (e.g., TRL) | | Memory reduction | ~60% less VRAM | | Mechanism | Hand-written Triton kernels, manual autograd, 4-bit optimizers | | Headline enabler | 7B QLoRA on a $1,500 RTX 4090 | | Platform | CUDA-focused (NVIDIA) | | Apple Silicon | Not the target — use MLX (FT20) | | Recent feature | Dynamic 4.0 GGUF quants (intelligent per-layer quantization) | Unsloth matters because it is the tool that made consumer-GPU fine-tuning practical at the 7B scale. Before Unsloth, a 7B QLoRA required a more expensive GPU or aggressive compromises. After Unsloth, a single RTX 4090 — roughly $1,500 — runs a 7B QLoRA comfortably. That is the difference between "fine-tuning is something labs do" and "fine-tuning is something I do, on my desk, this afternoon." For a course built on the thesis that fine-tuning steers behavior and that students should *feel* that thesis by running experiments (FT00), Unsloth is the tool that makes the curriculum executable on a budget. --- ## How Unsloth Achieves the Win Unsloth's performance gains come from three engineering decisions, none of which is a new algorithm. This is important: Unsloth does not change *what* fine-tuning does. It changes *how fast and how cheaply* the same fine-tuning runs. ### 1. Hand-written Triton kernels for forward/backward PyTorch and Transformers use general-purpose kernels — code written to handle many cases reasonably well. Unsloth replaces the performance-critical kernels (the attention computation, the linear-layer forward and backward passes) with hand-written **Triton** kernels. Triton is a language for writing GPU kernels at a higher level of abstraction than CUDA C, but with near-CUDA performance. Unsloth's kernels are specialized for the exact shapes and operations that fine-tuning actually performs, eliminating the overhead general-purpose kernels carry. The gain is concrete: faster forward and backward passes, which is where most training time goes. ### 2. Manual autograd PyTorch's autograd — the automatic differentiation system that computes gradients during backpropagation — is general and correct, but it builds a computation graph dynamically and materializes intermediate tensors. For the specific case of fine-tuning (especially with LoRA/QLoRA adapters), much of this generality is wasted: the shapes and operations are known in advance. Unsloth implements **manual autograd** for its custom kernels — it computes the gradients directly, by hand, for the specific operations, rather than relying on PyTorch to derive and schedule them. This eliminates the intermediate-tensor overhead and the graph-construction cost. The gradients are mathematically identical; the path to computing them is shorter. ### 3. 4-bit optimizers The optimizer (AdamW, typically) maintains a state — momentum and variance estimates — for every trainable parameter. At 32-bit precision, this state is a significant memory cost, often larger than the model itself for full fine-tuning. Unsloth uses **4-bit optimizers**: the optimizer state is stored at 4-bit precision rather than 32-bit. This is an 8x reduction in optimizer-state memory, with negligible effect on convergence (the optimizer state is statistical; it tolerates aggressive quantization). Combined with QLoRA's 4-bit base weights (FT08), this is what brings a 7B fine-tune down to a size the RTX 4090 can hold. ### Why these are engineering wins, not algorithmic changes This is the point to internalize. Unsloth does not change the fine-tuning algorithm. A QLoRA fine-tune run through Unsloth produces the same kind of adapter — same LoRA rank, same target modules, same learning dynamics — as one run through TRL. The *math* is the same. What Unsloth changes is the *implementation*: the kernels that execute the math, the autograd that derives the gradients, the precision at which the optimizer state is stored. This means Unsloth is a near drop-in acceleration layer. You do not learn a new fine-tuning method to use it. You learn a slightly different API entry point, and your training runs faster and cheaper. The conceptual content of this course — steering vs knowledge, LoRA, DPO, GRPO — is unchanged. Unsloth just makes it runnable on the hardware you actually own. --- ## The Consumer-GPU Enabler The headline: **Unsloth made 7B QLoRA viable on a $1,500 RTX 4090.** This is the democratization claim, and it is literal. Before Unsloth, 7B fine-tuning on consumer hardware required either a more expensive GPU (24GB+ VRAM, pushing toward the $2,000-$5,000 tier), or aggressive compromises — tiny batch sizes, short sequences, offloading to CPU (which is slow). After Unsloth, the RTX 4090's 24GB is enough to run a 7B QLoRA with reasonable batch sizes and sequence lengths, at roughly twice the speed. For this course, that is the difference between a curriculum that is theoretically executable and one that is practically executable. When FT00 says "load MiniCPM5-1B, apply a LoRA, feel the steering thesis," Unsloth (or an equivalent optimizer) is what makes that lab completable on a student's own machine in minutes rather than hours. When FT08 teaches QLoRA, Unsloth is the tool that lets a student run the full worked example on a single GPU. The democratization is not just economic. It is pedagogical. A student who can run a dozen fine-tuning experiments in a day learns the steering-vs-knowledge distinction (FT00) by *feel* — they see it, repeatedly, on their own hardware. A student who can afford one experiment per day learns it from a textbook. Unsloth is what makes the former possible. --- ## The CUDA Focus and the Apple Silicon Caveat Unsloth is **CUDA-focused**. Its hand-written kernels target NVIDIA GPUs. This is a deliberate scope choice: NVIDIA dominates the GPU-ML ecosystem, and the kernels are where the performance lives, so Unsloth optimizes for the platform that most users have. The consequence: **Apple Silicon is not Unsloth's target.** On an M-series Mac, Unsloth's CUDA-optimized kernels do not apply. Apple Silicon users should reach for **MLX** instead — Apple's own ML framework, which is designed for the Metal backend and the unified-memory architecture of M-series chips. MLX is the subject of FT20 (serving) and is the Apple-Silicon path for both inference and fine-tuning on Mac. This is not a defect in Unsloth; it is a scope boundary. The course's guidance: - **NVIDIA GPU (CUDA)** → Unsloth for fine-tuning speed/memory optimization. - **Apple Silicon (Metal)** → MLX for fine-tuning and inference. - **No local GPU** → a hosted CUDA instance (Colab, cloud) with Unsloth, or a smaller model (MiniCPM5-1B) that runs on CPU. --- ## Dynamic 4.0 GGUF Quants Unsloth's recent **Dynamic 4.0** release introduced intelligent per-layer GGUF quantization — and this is the feature that extends Unsloth's value from training into the export step (Layer 4 of the FT00 stack). A **uniform quantization** applies the same precision (e.g., 4-bit) to every layer of the model. This is simple, but suboptimal: some layers are more sensitive to quantization than others (layers involved in attention, or the output projection, often degrade more when quantized aggressively). Uniform quantization over-quantizes the sensitive layers and under-quantizes the robust ones. **Dynamic quantization** does the opposite: it analyzes each layer's sensitivity and assigns a higher precision to the sensitive layers and a lower precision to the robust ones, hitting a target average bit-width while preserving more quality. The result: a GGUF file that is the same size as the uniform version but performs measurably better — Unsloth's Dynamic 4.0 quants beat uniform quants at the same file size. When this matters: - **GGUF export for local/Ollama serving** (FT19, FT20). If you are shipping a fine-tuned model to run locally via Ollama, Dynamic 4.0 gives you better quality at the same footprint — or the same quality at a smaller footprint. - **Constrained edge devices.** On a phone or a small edge device where every megabyte matters, dynamic quantization extracts more quality per byte. The pattern: Unsloth optimizes training (Layers 2-3 of the stack), and Dynamic 4.0 extends that optimization to the export (Layer 4). A model fine-tuned with Unsloth and exported with Dynamic 4.0 is optimized end-to-end for the single-GPU / local-serving path. --- ## Unsloth vs Axolotl vs TRL — The Decision Three frameworks dominate fine-tuning, and the course references all three. The choice is a real engineering decision, and it is decided by your hardware and your need for control. | Framework | Best for | Multi-GPU? | Control level | | --- | --- | --- | --- | | **Unsloth** | Single-GPU, speed/memory constrained, GGUF export | No (single-GPU focused) | Optimized, opinionated | | **Axolotl** | Multi-GPU, production configs | Yes | Config-driven, broad | | **TRL** | Full control, custom training loops, research | Yes (via Accelerate) | Maximum (it is the library) | **Unsloth** — the single-GPU optimizer. Choose it when you have one NVIDIA GPU, you care about speed and memory (who doesn't), and/or you want clean GGUF export (Dynamic 4.0). Its opinionated nature is the tradeoff: it makes decisions for you, which is great when you want to go fast and less great when you want to experiment with the training loop itself. **Axolotl** — the multi-GPU config-driven framework. Choose it when you have multiple GPUs, you want production-grade configuration management (YAML configs, reproducible runs), and/or you need to scale beyond what a single GPU can do. FTDD-05 covers Axolotl in depth. **TRL (Transformers Reinforcement Learning)** — HuggingFace's fine-tuning library. Choose it when you want maximum control over the training loop, you are implementing a custom or research fine-tuning method, or you are building on top of a fine-tuning stack rather than using one. TRL is the library Axolotl and (often) Unsloth build on. FTDD-04 covers TRL in depth. The heuristic: - **One GPU, go fast, export to GGUF** → Unsloth. - **Multiple GPUs, production configs** → Axolotl. - **Custom training loop, maximum control** → TRL. These are not exclusive. A common pattern is to prototype in Unsloth (fast iteration on one GPU) and scale to Axolotl (multi-GPU) for the production run. And TRL is the substrate underneath — the thing you reach for when neither Unsloth's nor Axolotl's abstractions fit. --- ## Anti-Patterns ### Expecting Unsloth on Apple Silicon Unsloth is CUDA-focused. Its hand-written Triton kernels target NVIDIA GPUs. On Apple Silicon, Unsloth's optimizations do not apply. Do not install Unsloth on an M-series Mac expecting a speedup — use MLX instead (FT20). This is a scope boundary, not a bug. ### Assuming the speedup changes the result Unsloth is faster and cheaper, but it does not change the fine-tuning algorithm. A QLoRA run through Unsloth produces the same kind of adapter as one through TRL. Do not expect Unsloth to produce a "better" model — expect it to produce the same model, faster, with less memory. If the result is better, the cause is your data or hyperparameters (the actual steering wheel), not the kernel. ### Picking Unsloth for a multi-GPU job Unsloth is single-GPU focused. If you have multiple GPUs and you choose Unsloth, you are leaving compute on the table — Axolotl or TRL (with Accelerate) will use the additional GPUs and finish faster. Unsloth is the right choice when the constraint is *one* GPU's speed and memory, not when you have a cluster. --- ## Key Terms | Term | Definition | | --- | --- | | **Unsloth** | Single-GPU fine-tuning optimizer; ~2x speed, ~60% less memory via hand-written kernels | | **Triton kernels** | Hand-written GPU kernels (in the Triton language) that replace PyTorch's general-purpose kernels for the performance-critical forward/backward paths | | **Manual autograd** | Unsloth computes gradients by hand for its custom kernels, bypassing PyTorch's dynamic graph construction and intermediate-tensor overhead | | **4-bit optimizers** | Optimizer state (AdamW momentum/variance) stored at 4-bit precision — 8x memory reduction with negligible convergence impact | | **Dynamic 4.0** | Unsloth's intelligent per-layer GGUF quantization; assigns precision per-layer by sensitivity, beating uniform quants at the same size | | **MLX** | Apple's ML framework for Metal/unified-memory; the Apple Silicon alternative to Unsloth (FT20) | | **Axolotl** | The multi-GPU, config-driven fine-tuning framework (FTDD-05) | | **TRL** | HuggingFace's fine-tuning library; maximum control, the substrate Unsloth/Axolotl build on (FTDD-04) | --- ## Lab Exercise See `07-lab-spec.md`. The lab runs the same QLoRA fine-tune through Unsloth and a baseline (TRL), and measures the speed and memory difference — the Unsloth win, felt directly on your own GPU. --- ## References 1. **Unsloth** — `unslothai/unsloth`; the library and documentation. 2. **Triton** — OpenAI's Triton language for GPU kernel authoring; the substrate Unsloth's kernels are written in. 3. **Course 3 FT01** — VRAM math; why the memory savings matter. 4. **Course 3 FT08** — LoRA/QLoRA; the adapter method Unsloth accelerates. 5. **Course 3 FT19/FT20** — Quantization and serving; where Dynamic 4.0 GGUF export lands. 6. **Course 3 FTDD-04** — TRL; the library Unsloth builds on and the full-control alternative. 7. **Course 3 FTDD-05** — Axolotl; the multi-GPU alternative.