Accountants are being outpaced by AI. Accountants who worked with Claude were slower than Claude working alone, and slightly less accurate. In Mercor's experiment, accountants overrode Claude's correct advice or accepted numbers Claude had wrongly changed. On a month-end close task with a 3-hour limit, the AI got a perfect score in under 10 minutes, at 1/49th the cost of the accountants' labor.
12 US CPAs and AI solved the same close tasks
On October 2 (KST), Mercor, an AI training data company, published a study on its blog and as a paper in which 12 US certified public accountants (CPAs) and AI models solved the same accounting tasks. All participants were practicing US CPAs with an average of 5.4 years of accounting experience, and half had worked at a Big Four accounting firm.
On accounting work of moderate length with clear requirements, the latest AI was faster and more accurate than the accountants, and it also beat the participant with the highest score. Mercor considered not publishing the results because of the chance they would be misread, but concluded that releasing them was right so people can prepare for AI.
You can check the experiment and results in the research paper PDF and Mercor's research write-up.
Source: X @mercor
3 hours per task, with 13 to 22 rubric items
The tasks were four month-end close jobs taken from APEX-Accounting, which Mercor released in July, and trimmed so a person could finish them within 3 hours. Participants worked for up to 3 hours per task on a virtual PC loaded with a fictional company's working files. The grading rubrics, written by accountants, had 13 to 22 items per task.
The first task was allocating a nonprofit's shared costs across programs and support departments and comparing December spending to budget. The second was reconciling the differences in December room revenue and online travel agency commissions between a hotel's reservation system and its books. For example, participants had to find an item where $4,000 of parking income had been misclassified as room revenue.
The third was totaling venue revenue by event type and comparing it to budget, and the fourth was updating the lease schedules for three affiliated companies to reflect December lease assets and liabilities.
Accountants averaged 37%, and Opus 5 got 100% in all 20 runs
The accountants' 23 attempts without AI ranged from 0% to around 90%, with an average score of 37%. Most used between 30 minutes and 3 hours. Claude Opus 5, solving them alone with the same files and rubrics, scored 100% in all 20 runs, and each took under 10 minutes.
The accountants who built the tasks expected a junior accountant to average 30% and a mid-level accountant 55%. Most participants submitted before the 3 hours were up. Few lost points because they ran out of time.
The gap isn't the sudden appearance of a recent model. It is the result of closing quickly over about 18 months. GPT-4o scored near 0% on these tasks in mid-2024, and in spring 2025 OpenAI o3 first passed the accountants' 37% average. GPT-5 scored about 69%, Opus 4.8 around 80%, and Opus 5 and the latest frontier models came close to 100%. We have also covered how Claude Opus 5 performed in other model evaluations.
Mercor's view is that if the same experiment had been run 18 months ago, the average participant would have beaten the AI, but today even the best participant is far behind. Some lower-cost or open-weight models, like Qwen3.5-122B, still scored below the accountant average.
Source: X @aden_barton
Working with Claude took about 15 times longer
The study was originally designed to measure how much AI could help accountants. Participants solved two of the four tasks, chosen at random, together with Claude Cowork (Opus 5).
But the human-and-Claude collaboration was slower than Claude working alone and slightly less accurate. Time per task was about 15 times Claude alone. In three of the four collaborative attempts that fell short of a perfect score, Claude offered the right number during the process but it never made it into the final answer. In two cases, Claude changed a correct number to a different value and the accountant followed the changed value, and in one case the accountant overrode Claude's correct advice. On tasks where AI already gets a perfect score, the time humans spent reviewing the output was added on top.
Cost per rubric item was 1/49th
The cost to get one rubric item right was $0.21 for Claude Opus 5 and $10.35 for an accountant. Using the US median accountant wage, the human side costs about 49 times more. Claude's cost is calculated at API list price with no cache discount, and for the accountants it is based on the US Bureau of Labor Statistics median wage rather than the participation fee for the experiment.
On rubric items where the accountants' average success rate was above 60%, the AI worked 34 times faster. Even if model performance doesn't improve beyond today's level, Mercor believes productivity in accounting work could rise substantially.
The answer was in the files they were first given
Mercor re-checked fairness with the accountants who built the tasks. All the information needed to find the answers was in the files participants received at the start. That said, the tasks had been tuned so AI would get them wrong.
In the hotel task, some late-December stays were billed on January 3 of the following month, so they were missing from December revenue. A different company's profile said it reclassifies leases every month, but most participants working without AI skipped that step. Missing one item could throw off the calculations after it, and participants couldn't ask colleagues or clients.
Seven of the 12 participants said the tasks were similar to realistic work. Those who found them unrealistic mostly pointed to the lack of colleagues as the problem.
On the original APEX-Accounting, the AI missed tasks too
Mercor released APEX-Accounting with Ramp on July 31 (August 1 KST). Across 160 close tasks from 10 fictional companies, Claude Fable 5, the top model at the time, got only 56.4% of rubric items right. Even the most consistent model solved only 2.6% of tasks perfectly in all eight attempts, and no model fully solved 58% of tasks even once.
This human-versus-AI experiment trimmed the original tasks to a size that can be solved in 3 hours and added a file guide. The most common mistake in the original was failing to carry a discrepancy found in an earlier step into the final calculation or journal entry. It is the same kind of error as the cases where accountants working with Claude missed the right numbers. You can see the original benchmark in the APEX-Accounting introduction.
Benchmarks are moving toward tasks only AI can solve
As benchmark tasks keep getting harder so that AI models fail them, more of the tasks become hard for a single person to solve alone. Tasks shift from the area both people and AI can do to the area only AI can do.
A person can't run at 60 miles per hour (about 97 km/h), but a car carries people at that speed and raises productivity. The productivity created by work only AI can do could also be worth a great deal economically. But that kind of work barely happens inside the economy yet, so its value is hard to measure with wage statistics. Mercor proposed that when benchmarks are released, they also measure the ratio of tasks people can do to tasks only AI can do.
The change reaches junior accountants first
Mercor's argument is that benchmark scores have risen quickly, but the effect of AI barely shows up in US productivity statistics. In this experiment, on close work that involves looking through every file and following instructions, AI was faster and more accurate than the accountants and cost 1/49th as much.
The experiment didn't measure talking to clients, finding the questions that need asking, or the ability to use judgment when the rules aren't clear. Mercor expects accountants' work to shift gradually toward those tasks. For another example of handing repetitive work to AI, see how to automate work with Claude Cowork and ChatGPT.
This change reaches junior accountants first, because the month-end close, where AI pulled ahead this time, is work junior accountants do by hand every month.