_edited.png){kind=logo}
AI Washing: The Importance of Checking AI Performance Metrics
- Sep 12, 2025
- 3 min read
From transcription services to decision-support tools, the sales pitch is often confident: “92% accuracy”, “state-of-the-art performance”, “scales seamlessly”.
But I’m always reminded of the Midlands Police and Crime Commissioner’s Strategic Adviser, who reportedly told The Guardian of his concerns about businesses “pitching algorithms to police forces knowing their products may not be properly scrutinised.”
As every Compliance Lead or DPO knows, numbers without context can mislead. When AI is being considered for procurement, performance metrics aren’t just technical details, they’re core to risk, compliance, and fairness.
Beyond the Headline Numbers
Suppliers may present accuracy figures or latency statistics as proof of quality. But percentages alone rarely tell the whole story.
Speech-to-Text (WER): Some systems we explored reported a 12% error rate. Benchmarks for leading providers are closer to 5–8%, so this gap is significant.
Latency: This is simply the time taken for the system to respond after receiving input. In a speech-to-text tool, it’s the delay between finishing speaking and seeing the transcript. A supplier should be able to state, for example:
“Average latency: 1.2 seconds per request”
“99% of requests completed in under 2 seconds, tested at 500 concurrent users. ”If latency doubles under higher demand, that is a clear red flag for procurement teams, a system that works well in a demo may slow to an unusable crawl in real-world use.
Accuracy for Whom?
Perhaps the most important question a DPO can ask about any claimed performance number is:
“Who is in the percentage - and who is left out?”
A system might claim 92% accuracy. But if it was trained on male voices but is being proposed to support women in menopause, for example, the missing 8% error rate could disproportionately affect women. In practice, that makes the system inaccurate where it matters most.
This isn’t just a technical issue. It links directly to the privacy principles of fairness, transparency, and non-discrimination. DPOs should always push for evidence that metrics apply to the intended user group, not just to a generic population.
What Good Metrics Look Like
To be useful, metrics should be:
Specific: e.g. “System correctly identifies 92% of handwritten letters in pilot testing”, not just “highly accurate”.
Relevant: tied to the real-world task and population the system is meant for.
Transparent: showing both strengths and limitations, including where errors cluster.
Comparable – expressed in ways that let procurement teams weigh one supplier against another.
Why DPOs Should Care
As we see the US Federal Trade Commission establishing to combat and the UK's ASA issues rulings and guidance around misleading AI claims in advertising, DPO's are part of an internal effort to ensure that money and time spent on commissioning critical systems is not wasted as a result of overblown claims.
As the U.S. Federal Trade Commission launches Operation AI Comply to combat AI Washing, and the UK’s Advertising Standards Authority (ASA) issues rulings on misleading AI claims, the message is clear: regulators are paying attention.
For organisations, the DPO is part of the internal safeguard - ensuring that money and time spent commissioning critical systems is not wasted on overblown claims, and that adoption decisions are grounded in evidence rather than hype.
Accountability: forcing suppliers to substantiate claims.
Confidence: allowing boards and regulators to see evidence of due diligence.
Risk management: helping to identify potential harms before adoption.
Equity: checking that protected groups are not disproportionately excluded or misrepresented.
The Role of the DPO in Procurement
As AI adoption accelerates, DPOs have a unique role: to ensure procurement teams don’t just buy into big promises. Asking the right questions about performance evidence is one of the most practical steps a DPO can take to safeguard their organisation.
Without metrics, organisations are buying promises. With metrics, they’re buying assurance. And assurance is what procurement, and data protection, are all about.
Are you a DPO or Compliance Lead who wants more training on assessing AI systems? Contact us for details of our next event!
Comments