Do small organizations really need to test AI for bias?

If AI influences decisions about people, yes, in proportion. A dozen paired tests and a written record take an afternoon and show you took care.

Does the four-fifths rule prove discrimination?

No. It's a screening heuristic from the US EEOC's guidelines: a ratio below 0.8 is a signal to look closer. Small samples, job requirements and other factors all matter, so get advice for anything legally sensitive.

Can generative AI be biased in what it writes?

Yes. Review a sample of its output for stereotypes, uneven tone or missing perspectives, especially in anything public or about people.

Are there laws that require bias audits?

Some. New York City, for example, requires bias audits for automated employment decision tools. Rules differ by place and change often, so check with a qualified advisor.

How to check AI for bias: a practical test any organization can run

By Rafael · Updated November 1, 2026 · 2 min read

The short answer

Check AI for bias wherever it affects people, like hiring, eligibility, pricing or care. Review the data for who's missing, run paired tests that change only a name or trait, compare outcome rates across groups (a ratio below 0.8 is a warning sign), then fix, re-test and write it all down.

When you need a bias check

Any time AI informs a decision about a person or writes content about people: screening applications, approving requests, setting prices, prioritizing cases, drafting performance notes, or answering the public. If AI only helps with internal drafts that a person rewrites, a lighter review is usually enough. Bias testing is one part of AI governance.

Employment, housing, lending, insurance, education and health are legally sensitive. If AI touches those decisions, involve a qualified advisor before relying on any test.

A five-step bias test

1. Scope itWrite down the decision AI influences, who is affected, and which groups could be treated unfairly: sex or gender, age, race or ethnicity, disability, language, religion, location, and proxies for them like names, zip codes or gaps in work history.
2. Check the dataDoes the information the AI uses represent the people it affects? Who's missing? Is any input a stand-in for a protected trait that you could remove? Your AI is only as good as your data.
3. Run paired testsBuild 10 to 30 realistic cases. Make versions that differ in only one thing, like the same résumé with a different name, or the same question in another language. Run each several times and note any difference in outcome, score or tone.
4. Compare outcomes across groupsWhen you have enough real decisions, work out each group's selection rate (selected ÷ considered) and divide the lowest by the highest. Below 0.8, the "four-fifths rule" used by the US EEOC, is a warning sign to investigate, not proof of discrimination. Small numbers can mislead.
5. Fix, re-test, recordRemove proxy inputs, change instructions, add a person's review, or stop using AI for that decision. Re-test, then record what you tested, what you found, what you changed, and who signed off. Repeat yearly and whenever the tool or data changes.

Free bias test record

AI bias test record

AI BIAS TEST RECORD · [Organization] · [date]

AI use and decision: ______
People affected: ______
Groups considered: ______
Data review (who's missing, proxy inputs removed): ______

Paired tests run: [n] · Differences found: ______
Outcome rates by group: ______ · Impact ratio (lowest ÷ highest): ______

Findings: ______
Fixes made: ______
Re-test result: ______

Signed off by: ______ · Next test due: ______

Keep the record with your AI register. It's the evidence that shows you checked, if anyone ever asks.

Common questions

Do small organizations really need to test AI for bias?: If AI influences decisions about people, yes, in proportion. A dozen paired tests and a written record take an afternoon and show you took care.
Does the four-fifths rule prove discrimination?: No. It's a screening heuristic from the US EEOC's guidelines: a ratio below 0.8 is a signal to look closer. Small samples, job requirements and other factors all matter, so get advice for anything legally sensitive.
Can generative AI be biased in what it writes?: Yes. Review a sample of its output for stereotypes, uneven tone or missing perspectives, especially in anything public or about people.
Are there laws that require bias audits?: Some. New York City, for example, requires bias audits for automated employment decision tools. Rules differ by place and change often, so check with a qualified advisor.