3 anti-patterns that destroy QA credibility

Series: QA Leadership · Article 9 of 9 · Finale

I could only write this article because I have personally made every mistake described in it. My first dashboard for the board had twenty-three charts and I was genuinely proud of it. The CEO gave it about nine seconds.

Across eight articles we built a toolkit: five metrics, a decision indicator, a narrative. One last question remains - what can ruin all of it. The answer is short. Three habits that look innocent, and each of them quietly undermines trust in the quality team. I know them well, because for years I cultivated all three at once.

The good news is that none of them comes from bad intentions or lack of competence. They all grow out of an excess of good will. And that is exactly why they are so easy to fall into.

Anti-pattern 1: the report that shows everything

a.k.a. the dashboard with twenty-three charts

The logic seems airtight. Since we collect this much data, let’s show it. More charts means more work, more work means more credibility. That is what I thought while preparing that dashboard, and that is what most QA teams think at a certain stage of maturity.

The listener’s attention, however, works like a budget, not like a well. Every additional chart spends part of that budget and shrinks what is left for the others. With six indicators the room still keeps up. With twelve it starts picking randomly what to look at. With twenty-three it quietly gives up and waits for the next agenda item. Nobody says it out loud, because it is polite to appreciate the effort.

Getting out of this habit hurts, because it requires throwing away things you worked hard on. The rule that worked for me: one slide, one decision indicator, at most four supporting numbers. Everything else lands in the appendix and waits for questions. The paradox is that since I started showing less, people ask about more.

Anti-pattern 2: the number that walks alone

a.k.a. 94% with no answer to "is that good?"

A scene from real life. Sprint review, “DDR: 87%” on the slide, twelve people in the room. Someone from the business raises a hand and asks whether that is a good result. Exactly. Eighty-seven percent of what, relative to what? Without a reference point that number is noise that sounds like information.

A listener deprived of context will do one of two things. Either they will fill in their own interpretation, usually a wrong one, or they will stop listening. Both options work against you. The worst part is that the sender usually does not see the problem, because they carry the context in their head. They know it was 74% a quarter ago. They just forgot that they are the only one who knows.

Instead of explaining further, simply test it on yourself. Three real numbers, three quick decisions.

The cure is cheap. Every number goes to the meeting with one of three companions: a trend (how it was before), a reference point (a benchmark, a target, another measure) or a denominator (per release, per sprint, as in article six). One extra sentence on the slide. That is the price of the difference between information and noise.

Anti-pattern 3: the curse of knowledge

a.k.a. flaky tests, race conditions and the Product Owner's face

Once, during a status meeting with the business, I said a sentence I remember to this day: “Regression is falling apart because we have flaky tests due to a race condition on CI, and coverage dropped after the refactor”. The Product Owner nodded. After the meeting he came up and asked whether that meant there would be a delay. Nothing else from what I said had landed.

Psychology calls this the curse of knowledge. Once we know something, we lose the ability to imagine what it is like not to know it. The shorthand of our world seems obvious, so we use it with people from outside that world too. The listener nods out of politeness, and inside makes a decision to ask someone else next time. That is how the advisor’s position quietly dies.

Below are five sentences that were actually said in meetings. Click each one to see the version that gets through.

Notice what the hidden versions of each pair have in common. None of them explains the mechanism. All of them explain the consequence and say what happens next. The business does not care how a race condition works. It cares whether the release ships on Friday and whether the customer will feel anything. Answer that question, and leave the technical details for conversations with engineers, where they belong.

Self-diagnosis: check yourself before the room does it for you

Six questions, two per anti-pattern. Tick the ones that sound familiar. No cheating, nobody is watching.

Four weeks to your first report that works

The whole series is behind us, so here is a minimum plan to finish with. Tested in practice, no revolution, doable alongside normal work.

Nine articles later

We started with a diagnosis: QA reports activity, the business wants to hear about outcomes. Between those two points we fit five metrics, one decision indicator, a narrative workshop and today’s three warnings. That is the complete set. Not because more indicators could not be added, but because this set is enough, and everything beyond it starts working against you. You already know why - I wrote about it a few screens above.

If one thought were to remain from the whole series, let it be this: metrics are not there to prove that QA is working. They are there so the company makes better decisions. The difference seems subtle, and it changes everything, from the choice of indicators to the layout of the slide.