Security

Are LLMs More Skeptical When a Man Speaks? Fact-Checking a Viral Claim.

We fact-checked the viral tweet claiming LLMs are biased against men. The actual paper tested 6 models, not 13, and found a pattern of more skepticism toward statements attributed to male speakers.

· Huuush

Are LLMs More Skeptical When a Man Speaks? Fact-Checking a Viral Claim

Key takeaways

  1. 1Contrary to the viral claim, the fake news detection study evaluated 6 models, not 13.
  2. 2In the study, changing only the speaker's gender label changed the truth judgment, and a judgment flip rate of up to 23.6% was reported.
  3. 3For work like hiring and moderation where gender is irrelevant to the judgment, it's safer to remove gender information from the input.
Still life photo of two human-shaped wooden blocks on a balance scale against a light gray background, tilted to one side

Do Men Get Treated as Liars?

"Every major LLM is secretly biased against men."

On August 16, 2026, this claim from the X account How To Prompt (@HowToPrompt__) spread, drawing more than 82,000 views.

The claim is that when the same sentence was given to an AI for truth judgments with only the speaker's gender changed, statements labeled as spoken by a man were more often classified as "false."

The Actual Paper Tested 6 Models

The paper behind this claim does exist.

It is "Unequal Verdicts: Investigating Gender Bias in LLM-Based Fake News Detection" by Razieh Chalehchaleh, Reza Farahbakhsh, and Noel Crespi. It was posted to arXiv on August 4, 2026.

The researchers took LIAR, a benchmark for fake news detection, changed the speaker information into three versions (gender-neutral, male, and female), and asked each of 6 models.

For the same sentence, changing only the speaker's gender label shifted the truth judgment.

When comparing the male and female versions, the share of judgments that flipped (the flip rate) reached up to 23.6%.

The direction was also clear. A "male-skeptic" pattern appeared: the same claim, when labeled as spoken by a man, was judged more skeptically and strictly by the models.

The Number of Models Tested Was 6, Not 13

However, the viral tweet's wording, "tested 13 models," isn't accurate.

The paper actually evaluated 6 models.

The study's core conclusion, that every model reacted to gender and was more skeptical of statements from male speakers, matches the paper.

But an inflated citation like this is a classic case of why you shouldn't repeat a claim without checking the source.

A Similar Pattern Showed Up in Hiring

Coincidentally, another study published around the same time also reported gender-based differences in AI judgments.

David Rozado's "Gender and Positional Biases in LLM-Based Hiring Decisions" gave 22 models from Anthropic, Google, Meta, OpenAI, and others pairs of resumes with identical qualifications across 70 job roles and had them judge hiring fit.

The result: every model consistently rated resumes with female names higher. The paper was published in PeerJ Computer Science.

The two studies deal with different tasks (truth judgment vs. hiring evaluation), but both show that changing only the gender label changes the AI's judgment.

Why You Should Strip Gender Fields First

Both papers point to the same implication. For work where you hand judgment to AI, such as resume screening, content moderation, and fact-checking, it's safer to remove gender information from the prompt or input data in the first place if gender shouldn't affect the result.

The Rozado paper also found that replacing names with neutral labels like "Candidate A" and "Candidate B" reduced bias in some models.

Affiliations and Peer Review Are Unconfirmed

The authors' institutional affiliations for the two papers weren't clearly confirmed in this check.

The venue for the fake news detection paper hasn't been confirmed (it's an arXiv preprint as of August 4, 2026), and whether it has completed formal peer review also isn't confirmed.

Frequently asked questions

How many models did the LLM gender bias study test?
The paper behind the article, Unequal Verdicts, evaluated 6 models on a fake news detection task. The viral tweet's claim of 13 models doesn't match the paper.
What is the male-skeptic pattern the study describes?
It refers to a tendency for models to judge a claim more skeptically and strictly when it is labeled as spoken by a man.
How can you reduce gender bias in AI judgment tasks?
The article suggests removing gender information from the prompt or input data when gender shouldn't affect the result. In the hiring study it covers, replacing names with neutral labels reduced bias in some models.

Related articles