The "DeepSWE 80%" rumor around ox-alpha was not a full-run result. The full run of all 113 tasks that Henry Zhang published ended at 66 solved, or 58.4%.
This result is less a rebuttal meant to talk down the model than a signal to tell partial tests apart from the full benchmark.
66 solved in the full 113-task run
The public run repository records that ox-alpha was called through OpenRouter as stealth/ox-alpha and that DeepSWE's 113 tasks were run using Docker and mini-swe-agent.
The run repository's summary gives 66/113, 58.4%, and a total run time of 20 hours 39 minutes. So the 80% figure shouldn't be cited as ox-alpha's final full score.
A gap in the score created by tool-call failures
The run records explain that some tasks did not fail from the model's code reasoning alone. In some cases, repeated format errors in which the agent failed to produce a tool call ended the task.
The repository records that in the first 28 runs, 13 had a tool call missing at least once, and 2 of those ended because of repeated format errors. These results are an important signal in real agent work, but they should be read separately from pure code-generation ability.
DeepSWE measures long-horizon work
The DeepSWE paper introduces 113 long software engineering tasks across 91 open-source repositories and five languages. Each task uses a dedicated evaluator that verifies the required functionality.
As the original DeepSWE paper explains, this benchmark looks at long workflows that include exploring a repository, making changes, and running tests, rather than simple code completion. So when choosing a model, you need to look at the harness and tool connections as well as a single score.
Measure it again on your own repository
The conclusion to take straight from this result is not "ox-alpha is good enough for real work." It's that after confirming the 58.4% baseline and the pattern of tool-call failures, you should re-verify with your own team's repository and approval policy.
If you're adopting long-running agent work in particular, record failures in three kinds: code reasoning failures, test or container failures, and tool-call failures. With that split, you can tell whether swapping models or improving the harness is needed first.