TypeLLM: Open-Source Type-Safe LLM Output With No JSON Parsing Errors.

Summary in 3 lines

• TypeLLM is an open-source library (Apache 2.0) that returns LLM answers on SGLang as strings, integers, numbers, booleans, or choices, exactly in the type you ask for, without changing model weights.
• Choices come only from the candidates you define, and results come back as a Python dictionary, so there are no JSON parsing errors.
• You can keep Thinking turned on, and you can plug it straight into data extraction like receipts and into agent pipelines.

When you wire an LLM into a service, the JSON sometimes breaks, or the answer is a value that is not in the list you set. TypeLLM is an open-source library for setups that serve open models directly with SGLang; it returns each answer in a type such as string, integer, number, boolean, or choice, without changing the model weights. It was released on September 17, 2026 at 9:41 PM (Korea time), and as of September 27 it has 767 GitHub stars and an Apache 2.0 license. The official @TypeLLM account introduced it on September 27 at 1:24 AM (Korea time) with a fair-die experiment.

Source: X @TypeLLM

1. How it keeps to the type

When it takes choices, TypeLLM attaches a one-token label such as A, B, or C to each candidate. It only reads the score for which label the model picks, so an option outside the list can never come back as the answer. It supports up to 24 choices. A choice answer ends in a single token, so it costs almost no output tokens.

A boolean is handled as one of two candidates. Integers and floats allow only valid number tokens, with a default maximum length of 32 digits. Strings are generated as JSON strings, and you can limit their length by setting maxLength. You get the result directly as a Python dictionary. It guarantees types and choices, but whether free-text content is correct depends on the model's ability.

2. You can turn Thinking on

TypeLLM lets the model reason inside its usual <think> block, and applies the type constraint at the point where it produces the final answer. You can turn on the thinking process with thinking=True and set a per-field token cap such as thinking_budget=2048.

3. Install it and run it for the first time

You need Python 3.10 or later and an SGLang server that serves a model on a GPU yourself. TypeLLM is not a way to just connect an SDK to a hosted API model like GPT or Claude; you set up a compatible open-model server and use it. The following is the SGLang launch example from the official Qwen3.8-27B guide. It serves the model split across TP=4.

sglang serve --model-path Qwen/Qwen3.8-27B --port 30000 --tp-size 4 \
  --context-length 262144 --reasoning-parser qwen3 --tool-call-parser qwen3_coder

As the README says, start the server with prefix caching on, then install the SDK.

pip install -U typellm

This example extracts the merchant, total, expense category, reimbursability, and confidence from a receipt.

from typellm import TypeLLMClient

client = TypeLLMClient("http://127.0.0.1:30000", model="Qwen/Qwen3.8-27B")
result = client.generate(
    context="""Receipt from Hilton London
Total: £324.50
Employee travelled to London for a client meeting.""",
    questions={
        "merchant": {"type": "string", "instructions": "Return only the merchant name."},
        "total": {"type": "number", "instructions": "Extract the total amount in GBP."},
        "expense_type": {"type": "string", "enum": ["meal", "travel", "equipment"],
                         "instructions": "What type of expense is this?"},
        "reimbursable": {"type": "boolean", "instructions": "Should this expense be reimbursed?"},
        "confidence": {"type": "number", "enum": [0.0, 0.25, 0.5, 0.75, 1.0], "instructions": "How confident are you?"},
    },
)
print(result)

The result is not a JSON string but a Python dictionary you can use right away.

{'merchant': 'Hilton London', 'total': 324.5, 'expense_type': 'travel', 'reimbursable': True, 'confidence': 0.75}

The officially supported models are as follows.

Qwen3.8-27B also supports image input. In Claude Code or Cursor, you can start the project setup with the instruction below.

Read https://typellm.ai/SKILL.md and follow it to set up TypeLLM in this project.

TypeLLM Quick Start · Qwen3.8 SGLang guide · Official SKILL.md

4. Features for agent pipelines

The shared opening of a long input is reused through the SGLang prefix KV cache, and independent questions can be processed all at once. If the next question depends on an earlier answer, you declare the dependency with depends_on to set the order. With return_probabilities=True, you also get probabilities for each choice, so downstream logic can handle confidence.

The effect where a model's preference shifts with the order of the options is eased with permutation averaging. permutations: "auto" picks balanced orderings and averages the scores. In the fair-die experiment TypeLLM showed, averaging over 8 orderings cut KL error by about 79%, and averaging over all 720 orderings cut it by about 97%.

Source: X @TypeLLM

A developer in Japan (@yasuhiks) showed on September 26 at 10:57 PM (Korea time) how to pull strings, numbers, and choices from receipts and supporting documents in exactly the type you want.

Source: X @yasuhiks

Type-safe workflow · Fair die experiment

5. 98.70% on JevBench with Thinking

On the public JevBench run of 231 questions in the TypeLLM repo, it got 195 right (84.42%) without Thinking and 228 right (98.70%) with Thinking on. Median time per request was 1.3 seconds and 6.7 seconds, respectively. For comparison, Open-Jev 27B v1.1 scored 85.28% and Jev 1.13.0 scored 86.58%. On the TypeLLM side, the public Qwen3.8-27B checkpoint was used without extra training. You can see the comparison setup and evaluation environment in the repo's JevBench description.

JevBench is a benchmark of how accurately a model solves several types of decision problems. TypeLLM says it was inspired by Jev from TypeSafe AI. Jev is a dedicated decision model, while TypeLLM is a separate open-source implementation that uses existing open models on SGLang in a type-matched way.

6. Update timeline and hosted API

• September 18: Integer and number output support
• September 19: Thinking mode added
• September 22: depends_on dependency graph and prefix reuse
• September 23: JevBench results and permutation averaging
• September 24: Qwen3.8-27B image input

If you would rather use a hosted API than your own GPU server, you can sign up at TypeLLM Early Access.

If you serve open models with SGLang, try stripping out your parsing code with the single line pip install -U typellm.

Sources

• TypeLLM repo · Official introduction · Quick Start
• Official Skill · Type-safe workflow · Fair die
• JevBench results and method · Early Access
• X @TypeLLM: introduction · X @TypeLLM: permutation averaging · X @yasuhiks: use case