When people talk about the performance of medical LLMs, it's easy to think first of the score a model gets by solving problems directly. But a randomized, preregistered study published in Nature Medicine compared the model's standalone score with the situation in which users work with the model.
The study used 10 medical scenarios with 1,298 adults in the UK. Its conclusion was that there is a large gap between a model's medical knowledge and users' actual decisions.
LLM alone: 94.9%. In user conversations: under 34.5%
When the researchers presented the scenarios directly to the models, the rate of identifying the relevant conditions averaged 94.9% across the three models. Accuracy on disposition (what level of care to seek) averaged 56.3%.
It was different when ordinary users worked with the same models in conversation. The rate of identifying relevant conditions was under 34.5%, and for disposition it was under 44.2%. Neither result was better than the control group.
These numbers don't mean the models have no medical knowledge. They mean performance wobbled in the process of interpreting input, exchanging the needed information, and turning it into a final decision.
1,298 people and 10 scenarios separated the conditions behind the scores
Three physicians set the appropriate direction of care for the ten scenarios, and four physicians built the lists of relevant conditions. Participants were assigned to use one of GPT-4o, Llama 3, or Command R+, or to a control group that looked up information the way they normally would.
Participants looked at a symptom scenario and answered what kind of medical service they would use and which conditions they had considered. So it was not a simple multiple-choice test but measured the whole process of users entering information and interpreting the model's answers.
Data collection took place in 2024, and the paper was published in February 2026. It should be read not as a result about the performance of all the latest models, but as methodological grounds for evaluating user interaction separately.
What benchmarks missed was the link between conveying information and deciding
The researchers observed cases where users entered information incompletely or didn't properly follow the model's suggestions. Even when the model suggested a relevant condition, there were cases where the user failed to reflect that information in the final decision.
This is where the gap between evaluating a model alone and real use comes from. A benchmark hands over a well-organized question, but a real user has to decide first how to describe their symptoms.
The researchers explained that standard medical knowledge benchmarks and simulated-user evaluations did not adequately predict failures in interaction with real people.
For medical AI, user testing comes before model scores
The practical standard this study suggests is clear. Before deploying a medical tool, don't just check the model's test score; also test the conditions under which real users leave out information or misread it.
However, this study used controlled medical scenarios, not clinical care. It did not reproduce the urgency, anxiety, language barriers, or clinical setting of real patients.
So this paper is not grounds for replacing any particular medical decision. If you're on a team designing a medical service, it's more accurate to read it as a warning to manage model accuracy and user-interaction safety as separate metrics.