Early on August 20, Elon Musk posted a short message on his X account. It was just two words: "Grok Voice."
The post he quoted was a benchmark table shared by the anonymous account X Freeze. It claimed Grok Voice Think Fast 2.0 beat GPT Realtime in both text and voice.
We checked the raw data to see what this table actually measured.
Where does the table Musk shared come from?
The table comes from "Technical Report No. 11" of the open-source benchmark project VulcanBench. It's the result of "Voice Eval Suite v1," published on July 31.
It's not official material released by xAI or OpenAI. VulcanBench is an open-source benchmark published on GitHub and run independently by a third party.
The X Freeze account that first shared the table isn't affiliated with xAI either. According to a Washington Post report, it's known only as an anonymous account Musk engages with unusually often, and the operator's identity hasn't been disclosed.
99% vs. 97.5%: Grok wins on text
The VulcanBench report asked the same 200 questions once as text and once through text-to-speech (TTS), then scored them. The scoring criteria were identical in both modes.
| Model | Text accuracy | Voice accuracy | Voice conversion loss |
|---|---|---|---|
| Grok Voice Think Fast 2.0 | 99.0% | 95.7% | -3.3 pts |
| GPT Realtime | 97.5% | 93.5% | -4.0 pts |
Grok edged ahead in both text and voice. The accuracy lost when switching from text to voice (the report calls it a "voice tax") was also 0.7 percentage points smaller for Grok.
On arithmetic-style questions, both models' accuracy dropped by about 10 percentage points. That means it's a weak area for both. Conversely, on number-extraction questions, Grok actually scored higher in voice mode than in text mode.
Speed is different: GPT speaks first
On accuracy alone, Grok has a slight edge, but speed is another story. In the same report, GPT Realtime's time to first response (first token) was 0.53 seconds, versus 1.22 seconds for Grok.
The total time to finish a conversational turn flips, though. Grok finishes faster at 1.43 seconds, versus 2.70 seconds for GPT Realtime.
That means GPT starts speaking first, while Grok finishes the exchange first. The report itself states that "the first-token figure is not comparable with xAI's officially announced 'time to first audio.'" It's a warning against mixing different measurement methods.
Results also split under noise. When 10 dB of background noise was added, Grok's score fell by 5 points, while GPT Realtime barely changed.
A different flavor from the official benchmarks
The benchmarks xAI published in its official July 29 announcement look a bit different. On the Artificial Analysis speech-to-speech index, Grok Think Fast 2.0 scored 82.9%, ahead of GPT-Realtime-2.1 (79.1%) and Gemini 3.1 Flash (69.5%).
On agentic task performance (τ-voice), Grok also led GPT-Realtime-2.1 by a wide margin, 56.5% to 45.7%. xAI said Grok's time to first audio response is 0.70 seconds, far faster than GPT-Realtime-2.1's 2.98 seconds.
On the metric that measures conversational flow (Full Duplex Bench), though, Grok trailed GPT-Realtime-2.1 slightly, 95.1% to 95.7%. That means Grok didn't win on every measure.
In sum, the direction of "Grok beat GPT" is broadly confirmed by both VulcanBench and xAI's own materials. But the specific numbers and measurement methods differ by source, and it's worth noting that the table Musk shared is a single-run result from a third party, not from xAI or OpenAI.