Most fine-tuning guides assume CUDA. If you're on Apple silicon, the advice quietly stops applying somewhere around step three, usually without telling you. Here's what actually happens on an M-series Mac, and how to choose a method.
The four methods
| Method | What it does | On Apple silicon |
|---|---|---|
| LoRA adapter | Trains a small tuned delta over the frozen base | Works well. The default. |
| QLoRA (4-bit) | Same, with a 4-bit quantized base for lower VRAM | Needs CUDA. Falls back to plain LoRA. |
| Full fine-tune | Updates every weight | Possible, usually a bad trade |
| Continued pretraining | LoRA over full text, for domain adaptation | Works well |
Why QLoRA isn't the answer here
QLoRA's advantage is memory: quantizing the base to 4 bits cuts the largest allocation dramatically, which is what lets a 13B model fit on a consumer NVIDIA card. That path depends on CUDA kernels that don't exist for Metal.
Orinth detects this. Select QLoRA on an MPS or CPU machine and the run trains as plain LoRA, with the configuration you see updated to match. It doesn't silently do something different from what the form says, and it doesn't fail an hour in.
On a Mac, unified memory changes the calculus anyway. You're less likely to hit a hard VRAM wall and more likely to hit throughput limits, which quantization does not fix.
Why full fine-tuning is usually the wrong call
Full fine-tuning updates every weight, which means optimizer state for every weight — typically several times the model's own size in memory, plus a much slower step. On a laptop you'll usually get a worse result than LoRA, slower, for a much larger artifact.
The cases where it's worth it: a small base (under ~1B), a large and genuinely distribution-shifted dataset, and a reason to believe the adapter bottleneck is what's limiting quality. That's a narrower set of situations than the discourse suggests.
Start with LoRA. Move up only when you can point at a measurement saying it isn't enough.
A realistic loop
- Build the dataset. A Data Recipe turns your own documents into instruction pairs — deterministic by default, no model in the loop unless you opt in.
- Freeze a version. The run records the version it consumed. Skip this and in three weeks you won't be able to reproduce the result.
- Run LoRA. Orinth reads the accelerator and defaults to MPS. Watch the loss curve for the first few hundred steps; if it's flat, the problem is the data, not the hyperparameters.
- Export to GGUF. Only GGUF models are servable. Adapter → GGUF merges and quantizes in one step.
- Serve and chat. Sampler controls, an optional system prompt, and throughput reported under each reply.
Reading throughput honestly
The chat surface reports completion tokens, tokens per second, and time-to-first-token. Two things worth knowing:
Time-to-first-token is mostly prompt processing. A long system prompt moves it. It is not a measure of generation speed.
Tokens per second depends on quantization. A Q4_K_M model is faster than Q8 and worse at the margins. Whether that trade is acceptable is a question about your task, and the only way to answer it is to score both on a held-out split — which is what the Testing surface is for.
The thing that matters most
None of the above matters as much as the dataset. A LoRA run on 500 well-constructed examples beats a full fine-tune on 5,000 sloppy ones, and it finishes while you're still making coffee.
If your fine-tune isn't working, look at your data first. It's almost always the data.