In the early hours (KST) of August 18, 2026, developer Jacky Kwok (@jackyk02) shared the results of applying a verification technique to the open-source model DeepSeek V4 Flash. The claim is that it beat Anthropic's Claude Fable 5 on Terminal-Bench 2.1 at 11x lower cost.
How Accuracy Went to 88%
The method is called "LLM-as-a-Verifier." DeepSeek V4 Flash generates five candidate answers, and the same model then scores them and picks the best one. According to Kwok, this raised accuracy from 79% to 88%, ahead of closed frontier models on Terminal-Bench.
The Source Is Stanford, Berkeley, and NVIDIA
LLM-as-a-Verifier is a verification scaling framework published by researchers at Stanford, UC Berkeley, and NVIDIA Research, and its official project page (llm-as-a-verifier.com) says it scored 86.5% on Terminal-Bench V2. The DeepSeek V4 Flash result here is Kwok's own experiment with the framework, not an official benchmark from DeepSeek or from all of the paper's authors.
"11x Cheaper" Isn't the Token Price
In a reply, one developer pointed out, "That 11x is just the token price difference, and self-verification uses more tokens because it samples multiple times." Kwok answered that the cost includes the tokens for both generation and verification. The explanation is that open-source model tokens are so cheap that even using them several times, the total cost is still low.
By Official Prices, It's a 45x Gap
By Anthropic's official price list, Claude Fable 5 costs $10 per million input tokens and $50 per million output tokens. According to DeepSeek's official API documentation, V4 Flash pricing is split into peak and off-peak hours starting August 16, 2026, and the off-peak window, which covers most of the day, is $0.22 for input and $0.66 for output. Comparing only the per-token prices, that's about a 45x gap on input and about 76x on output. Kwok's "11x" should be read as a figure based on the real cost of finishing one task, which also reflects the extra token consumption from self-verification.
| Model | Terminal-Bench 2.1 | Input (per 1M tokens) | Output (per 1M tokens) |
|---|---|---|---|
| DeepSeek V4 Flash (official, without self-verification) | 82.7 points | $0.22 to $0.44 | $0.66 to $1.32 |
| Claude Fable 5 | Not officially confirmed | $10 | $50 |
Per the official Hugging Face model card, DeepSeek V4 Flash on its own scores 82.7 on Terminal-Bench 2.1. That is the baseline without self-verification, and Kwok's rise from 79% to 88% came from a separate experiment, so it's hard to compare directly. Claude Fable 5's Terminal-Bench 2.1 score is reported differently by different sources, so this article leaves it as not officially confirmed.
What to Look at to Reproduce This Now
You can find the concrete implementation of combining an open-source model with self-verification in the GitHub repository Kwok published (llm-as-a-verifier). Because the structure increases token consumption, the actual savings depend on task difficulty and the number of samples.