Want to run a 27B model on your own PC, but hesitate when the download alone can take tens of gigabytes? Underdog Saluki 27B is an IQ2-mix GGUF based on Qwen3.8-27B, with a main model file sized at 7.89GB. That is about one-sixth the size of the 54GB full-size model.
7.89GB is the model file size, not the total memory needed to run it. The model weights are only part of the requirement: the KV cache and llama.cpp runtime also use memory. This does not mean the model will run on an 8GB computer with no other memory pressure. Requirements also vary with context length and concurrent work.
A Qwen3.8-27B model in IQ2-mix GGUF
Saluki 27B is a low-bit IQ2-mix quantization based on Qwen3.8-27B. Underdog publishes the main weights as Underdog-Saluki-27B-1.0-IQ2-mix.gguf, a 7.89GB file. The model is released under the Apache-2.0 license.
Underdog’s official card reports 96% average performance retained across nine benchmarks. This is the company’s summary figure; it does not mean every benchmark was independently reproduced under identical conditions. Detailed results vary by task and evaluation harness, so the average alone should not be taken as a verdict on performance for every use case.
BFCL v4 tool calling: 88 vs. 84 across 120 tasks
For its tool-calling comparison, Underdog used a fixed set of 120 tasks from Berkeley Function Calling Leaderboard v4 (BFCL v4). With thinking off and temperature set to 0, Saluki passed 88 tasks and the full-size Qwen3.8-27B passed 84. This result applies to that 120-task set and those settings; it does not establish that Saluki performs better on every tool-calling workload.
The official launch thread walks through the model size and benchmark summary, the BFCL evaluation conditions, and the tool-calling results relative to model size.
Source: X @UnderdogAI
Source: X @UnderdogAI
Source: X @UnderdogAI
Download and run it with llama.cpp
The official model card shows how to download the GGUF with the Hugging Face CLI and start a llama.cpp server. These commands save the file in the current directory and launch the server with a 32K context setting.
huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir .
llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --jinja -ngl 99 -fa on -c 32768--jinja enables the Qwen3.8 chat template used for tool calling and thinking behavior. Once the server is running, it exposes an OpenAI-compatible chat API on port 8080 by default. Adjust -ngl to match how many layers your hardware can offload to the GPU.
Image input requires a separate projector
The base Saluki GGUF is focused on text. Image input requires a separate vision projector file. The official card lists an F16 projector at 928MB and a Q8_0 version at 629MB; the projector and runtime environment add to memory needs. Downloading only the main model file does not enable image input.
For more on low-memory local LLMs and Qwen-family models, see Unsloth’s Qwen3.8-27B GGUF Dynamic v3.0 for 8GB RAM and our introduction to the Qwen3.8-27B open-weight model.
Saluki 27B is an option for bringing a 27B-class model to a local setup with a smaller download. Just keep the distinction clear: 7.89GB describes the GGUF file, while actual runtime memory must also cover the cache and runtime.