Train a Decision Model with Unsloth: Qwen3.5-0.8B Benchmarks Jump from 20.7% to 74.3%.

When a task has a fixed set of possible answers, like classification or routing, does it really make sense to run a large language model's full text generation every time? Unsloth has published a training workflow that fine-tunes an existing LLM such as Qwen or Gemma with LoRA and turns it into a decision model that picks among candidates directly and returns probabilities.

The guide's headline experiment uses Qwen3.5-0.8B. It trained for 42 minutes on 4GB of VRAM, and combined accuracy across three decision benchmarks rose from 20.7% to 74.3%. The figures of 10 minutes on 2.5GB and 77%, however, come from a separate case where Laya was trained further, not from this model. Let's keep the two setups apart and walk through a training path you can actually follow.

Turning a Text-Generating LLM into a Model That Chooses

A typical LLM takes an input and generates the next token. A decision model takes a state, a question, and a set of options, then returns an answer and a probability for each question. For example, it can read a customer inquiry and pick the right team, decide true or false on whether a message is a refund request, or score an item against a rubric you define.

The Unsloth guide describes applying LoRA (rank 64) and a Clef-family decision head to a base LLM. The published comparison was run for 1 epoch, and Qwen3.5-0.8B changed as follows on the three benchmarks.

The average is a simple mean of the accuracy on the three benchmarks. A separate holdout accuracy table lists Qwen3.5-0.8B at 78%, so the 74.3% combined figure and the 78% holdout figure should not be read as the same measurement. Unsloth says it decontaminated the test sets so they do not overlap with the training data.

Training It Yourself in Unsloth Desktop

With the GUI, install Unsloth Desktop, pick a model on the Train tab, and choose Train as → Decision model. You can start from the guide's recommended defaults.

1. Install Unsloth Desktop. The official instructions support macOS, Windows, and Linux. For a manual install, use curl -fsSL https://unsloth.ai/install.sh | sh on macOS, Linux, and WSL, or irm https://unsloth.ai/install.ps1 | iex in Windows PowerShell.

2. On the Train tab, select a model and set the training type to Decision model. The settings for reproducing the results are 2 epochs, LoRA rank 16, and a learning rate of 2e-4.

3. Choose a Hugging Face dataset or a file. When you use typed-decisions, set Subset to all.

4. Run training, then check the held-out decision accuracy and the confidence calibration results. A finished model can be connected through Use in Decision API.

The official experiment write-up used rank 64 for 1 epoch, while the reproduction steps for Desktop and Python call for rank 16 and 2 epochs. These are different experiment and workflow settings, so it is more accurate not to merge the numbers into a single configuration.

Controlling Data and Training Settings Directly in Python

In code, you use FastDecisionModel and DecisionTrainer. The official example loads Qwen3.5-4B, but the guide says you can apply the same pattern to Llama 3.2 3B or Gemma 4 E4B by changing the model name.

from unsloth import FastDecisionModel, DecisionTrainer, is_bfloat16_supported
from datasets import load_dataset
from transformers import TrainingArguments

model, tokenizer = FastDecisionModel.from_pretrained(
    model_name="unsloth/Qwen3.5-4B",
    max_seq_length=2048,
    load_in_4bit=True,
)
model = FastDecisionModel.get_peft_model(
    model, r=16, lora_alpha=16, lora_dropout=0,
    use_gradient_checkpointing="unsloth", random_state=3407,
)

dataset = load_dataset("LocalLLaMA/typed-decisions", "all", split="train")
items, report = FastDecisionModel.build_dataset(dataset, tokenizer, model)
print(f"Skipped {report['skipped']} of {report['total']} decisions")
train_items, eval_items = FastDecisionModel.split_holdout(items, seed=3407)

trainer = DecisionTrainer(
    model=model, processing_class=tokenizer,
    train_dataset=train_items, eval_dataset=eval_items,
    args=TrainingArguments(
        per_device_train_batch_size=8,
        gradient_accumulation_steps=4,
        num_train_epochs=2,
        learning_rate=2e-4,
        lr_scheduler_type="cosine",
        warmup_steps=10,
        weight_decay=0.01,
        bf16=is_bfloat16_supported(),
        fp16=not is_bfloat16_supported(),
        eval_strategy="epoch",
        logging_steps=10,
        output_dir="outputs",
        report_to="none",
        seed=3407,
    ),
)
trainer.train()
metrics = FastDecisionModel.calibrate(model, tokenizer, eval_items)
print(metrics)  # holdout accuracy, ECE, loss

The training data consists of a state, the question to judge, and the correct answer. build_dataset converts examples into the model's input format, and split_holdout sets aside evaluation data. Check the skipped count in the report to see how many items were dropped during input conversion. If memory is tight, the guide recommends halving the batch size and doubling gradient accumulation.

VRAM and Time Vary by Model

The figures in the guide are examples for each model and evaluation setup. Don't assume how much GPU memory you need from model size alone; check the headroom with your own data length, batch settings, and hardware.

Laya's 2.5GB, 10 minutes, and 77% are not Qwen3.5-0.8B results. That is a separate case in which Laya, which is already a decision model, was trained further. The 78% in the Qwen3.5-0.8B row, by contrast, is the test accuracy from the guide's table, and the 20.7%→74.3% in the title is the average across the typed-decisions, BANKING77, and CLINC150 benchmarks.

Calibrating and Deploying the Model

In decision-making, what matters is not just the answer but how far you can trust its probability. The official example calibrates the model on holdout data and checks accuracy, ECE (calibration error), and loss. Keep the holdout split intact so that items used for calibration don't leak into training.

You can run inference with FastDecisionModel.predict(model, tokenizer, state, questions). A choice question returns the candidate keys with a probability for each, a noul question returns true/false, and a score question returns the number of the level you defined. For saving, it supports save_pretrained, which keeps the adapter and head together, or save_pretrained_merged, which merges everything into a full 16-bit model.

To load a merged model in Studio, specify the model path as shown below. The first GPU load can take a few minutes.

UNSLOTH_SYSTEMONE_MODEL=/path/to/qwen-decisions-merged unsloth studio -H 0.0.0.0 -p 8888

The model reads the state together with the question and options and scores each option. It can handle structured decisions like classification and routing without generating natural-language answers. In real use, build your own evaluation set that resembles the questions, rubrics, and options you will use in production, not just public benchmarks, and check accuracy and probability calibration separately.

To set up an environment for training existing models yourself, see Getting started with a local training environment in Unsloth Studio. You can pick up the local inference workflow in Steps to run DeepSeek locally with Unsloth, and an example of running a decision model through an API is covered in the Unsloth Laya local run guide.

Official X Posts

Unsloth's announcement of the Qwen3.5-0.8B results on three benchmarks and 4GB of VRAM.

Source: X @UnslothAI

A separate case where Laya was trained further on 2.5GB of VRAM.

Source: X @UnslothAI

References

• Unsloth official guide to training a decision model

• Unsloth GitHub repository

• LocalLLaMA/typed-decisions dataset