Don't Rely on QA to Find Mistakes

A few years back, my husband and I moved from California to Florida to start new jobs. With our children in tow, we said goodbye to the mountains and dry heat to a life of flat land, humid weather, and BUGS.

Bugs are everywhere in Florida, and for the first time in our lives, we had to actively work to control the bugs beyond setting up a few ant traps.

First, we hired an extermination company, who would come every other week to spray the property and set up special devices to keep mosquitos and other pests at bay. Although this was helpful, we soon realized we would need to try harder to control the bug population affecting our quality of life.

We then set up various moth catchers in the entryway to catch those who made their way inside, plugged in devices that emitted noise too high for most humans to hear, and added ant traps to vulnerable locations (they are not just for California). We are also training our children not to leave doors open (I'll keep you posted on that progress).

I am happy to report that we have the bug population under control through our various endeavors. It isn't eradicated, but it is under control.

Number of bugs in application != Healthy application

Throughout my years in software development, I have often been asked to report the number of defects in a piece of code or application.

The folks asking for these reports are usually CEOs or CTOs so far removed from the actual development of the application that they feel compelled to understand the app's health by the number of bugs that exist. Questions like these are not only unhelpful; they are unproductive. Bugs in an application are much like bugs in Florida; they are inevitable.

Building an application takes great effort, often from many people. Whenever you build an application, there will be things that are not ideal but probably not catastrophic. The things that are not ideal could be minor bugs that few people see or experience, which might get deprioritized for more critical items that need the attention of your engineering teams.

Should these deprioritized bugs be weighted the same as a catastrophic bug that impacts, for instance, your revenue stream? I think the answer is clear; of course, they shouldn't.

Not all bugs in an application are the same. It's like ants in your home vs. roaches. Ants are annoying but can be tolerated if controlled, whereas roaches are, well....roaches and must be eradicated at all costs.

But who is responsible for finding mistakes in the application? Although the knee-jerk reaction is usually the QA team, the answer is more nuanced.

What is QA?

The term QA, quality assurance, is ironic because you can't test quality into an application. Testing generally only exposes issues you introduced with the latest build, not overarching architectural problems, preexisting scalability issues, or lack of functionality that results in support incidents. Please do not misunderstand me and assume I am saying QA is not necessary; it is essential!

The primary role of QA is to identify problems with the system at a lower cost than having a higher-paid engineer do it. The end goal is to get higher velocity from your engineering team while increasing the rate of finding bugs within the application.

Don't get me wrong; the engineers are not off the hook. They are responsible for quality engineering.

Engineers are accountable for documenting standards, writing good code, implementing unit tests, helping with integration tests, and, depending on the maturity of your application, incorporating testing into the CI/CD pipeline. But having a solid QA team to focus on component, automated, and manual regression testing allows the engineering team to focus on their deliverables while keeping costs in check.

The headcount cost of an efficient QA organization is typically lower than the cost of an engineering organization. You should hire a QA person anytime you could get more than one engineer's worth of productivity from the addition of that QA head.

QA's #1 Benefit

One of the most significant benefits a QA person brings to the table is experience with the application. They see the application end to end, day after day. They know the mistakes that have been made and how often the same mistakes are repeated.

An experienced QA team is incredibly beneficial for an organization that has seen rapid growth within engineering. If an engineering team has grown quickly, the engineering team is usually too busy to be able to properly train new engineers on coding standards and, instead, probably passes on bad coding habits. In these instances, an experienced QA team helps an organization learn from engineering missteps. When a QA team understands that they cannot test quality into a system, they work to identify systemic issues within the application or engineering team that lead to quality issues. This work goes beyond burn down charts and reports on the number of bugs identified; it involves getting to the heart of the issues and identifying themes of problems and their sources. Once these themes have been identified, work can begin to fix them.

Who is responsible for finding mistakes?

So who is responsible for finding mistakes in the application?

The answer is everyone. A mistake isn't simply a bug. A mistake could be limited functionality resulting in customer frustration, a poor-performing database because of missing indexes, or an incorrectly integrated third party because of limited documentation. Finding mistakes start where preventing mistakes does, with formal requirements and definition of done and ends with proper monitoring of the application.

Quality is everyone's job. And although an experienced QA team is probably best at helping strive towards quality in your applications, they are not the sole owners of that metric. The QA team is there to help the team reduce bugs and learn from past mistakes while keeping velocity up and costs down.

Just as my husband and I didn't rely solely on the exterminator to control our bug population, your business cannot rely exclusively on a QA team or process to identify every mistake waiting to make it into your production environment. Quality must be considered every step of the way in the application development lifecycle, starting with ideation down to the execution.