Who is Responsible for Product Quality on a Software Team?
In software development, ensuring high product quality is an often confused responsibility within a team. This article examines the core concept of software quality responsibility, emphasizing the dual phases of building with quality and assuring quality post-build. Drawing parallels to Toyota’s renowned production processes, where quality is ingrained at every stage, it underscores the importance of integrating quality into software from the onset of development.
What is software quality responsibility?
Before we begin exploring who is responsible for software product quality, let’s explain what we mean by responsibility in this context.
Simply said, software product quality responsibility is – making sure that changes deployed to production are free of defects.
In order to accomplish this there are two phases of quality responsibility:
1) Building with quality
2) Assuring the quality is there after the build
This is not much different from a car manufacturer’s production line. Take for example Toyota, known for its high quality products and efficient production processes.
Quality Built into Every Process
At Toyota quality is integrated at every stage of production. The company employs standardized work practices and visual controls to help workers identify issues quickly. Each employee is responsible for maintaining quality and can stop the production line if they notice a problem.
Toyota’s approach to quality, as detailed by Akinori Hyodo, emphasizes ensuring quality at every production stage rather than relying on final inspections. Key elements include treating the next process as the customer, implementing standard work procedures, maintaining equipment rigorously, and keeping a strong quality awareness among employees.
Rigorous Testing and Inspection
Before vehicles leave the factory, they undergo extensive quality checks. Over 2,000 inspections are performed on each vehicle, covering various aspects including bodywork, engine performance, and safety features. Additionally, Toyota conducts collision tests and extreme environmental tests to ensure reliability under diverse conditions.
Building Software with Quality
Therefore, owning the software product quality has to happen in two major phases – during development and after development. Quality cannot be an afterthought. It has to be part of the process from the very beginning of writing code. Quality must be built into the software. It cannot be assured into the software.
Quality cannot be “assured” into the software.
It has to be “built” into it.
Assuring the quality is there after the build
Once software is built with high quality the second phase is to ensure that it is there. If this is the first step a team gives attention to quality, it is already too late. You have heard the phrase “An ounce of prevention is worth a pound of cure”. This concept applies here as well.
Here is another way to think about it:
When you drive your car you want to do everything you can to avoid an accident. You, as the driver, take responsibility to ensure safe driving. Your seatbelt and airbags are there to catch you in the event that your best driving efforts fail and you still get into an accident. Similarly in software development you want to do your best to build with quality, and test and inspect after the building is complete to catch any potential misses.
The role of the Software Developer
So, who is responsible for software product quality? The primary responsibility lies on the software developer. He/she is responsible for building quality into the software they develop.
To use the analogy from above – the developer is the driver of the car and it is their responsibility to drive the car safely. Again, that means that quality cannot be an afterthought. They have to write software with quality to begin with.
There is a reason why Test Driven Development (TDD) is so effective. It “begins with the end in mind”, which is one of the most important habits to develop, according to Stephen Covey‘s book “The 7 Habits of Highly Effective People“.
Thus, the software developer is responsible for testing all of their work, before it is considered “complete” and ready for inspection.
The role of the QA Tester
The QA Tester is simply responsible for assuring that the quality exists in the software product. This responsibility is embedded in their title – Quality Assurance (QA). The meaning of the word “assure” is: to make sure or certain.
Therefore, the QA tester only does an inspection of the to validate and confirm that the engineer built the software with quality to begin with.
Continuing the analogy from above – if the developer is the driver of the car, the QA is the seat belt and airbag.
The role of the SDET
A Software Development Engineer in Test (SDET) plays a role in both the development and testing of software. They are involved in designing, developing, and maintaining automated test scripts, frameworks, and tools to ensure the quality and performance of software applications.
When it comes to software product quality the SDET sits somewhere between the developer and the QA. it is worth noting that:
a) The role and responsibilities of an SDET can widely vary from one company to another. This will be primarily determined by the overall team structure and their maturity when it comes to testing.
b) Not every software organization has SDETs on their team
The role of the Product Manager/Owner
The main role of the Product Manager/Owner (PM/PO) is to ensure that the developer has built the product per the product specifications. That does not mean that they will inspect the quality of the work, nor will they examine every use case of the functionality. Their role is to make sure the main functionality and look and feel of the product are there. They should expect that the software is free of defects, because quality has already been built into it, and the software has already been inspected to ensure the quality is there.
It is important to note that on small teams the PM/PO can assume the additional role of a QA. That can be a long-term role, or a short-term one when trying to meet a fast approaching deadline for example. In either case, this additional role and expectation of the PM/PO should be officially agreed upon by the delivery team.
If you're facing hurdles in launching your software product or scaling your team and processes, you're not alone.
Many tech startups encounter similar roadblocks, which can stall progress and growth. My Fractional CTO services are designed to help you navigate these challenges effectively. By leveraging my expertise, you can ensure your product development stays on track, your team is well-coordinated, and your technology is robust and scalable. Don’t let these obstacles hinder your success.
How much testing should developers do?
Now that we understand who is responsible, and how, for software product quality, you might ask – How much testing should software engineers do then? My simple answer is – as much as necessary to make sure that QA will not find any issues.
The next question is – Should engineers do TDD, Unit Testing, Integration Testing, Regression Testing, Performance Testing, etc.? There is no one right answer to this question and this topic alone can be an article on its own. This might look slightly differently on every team, depending on: team structure, number of people, skill and experience level, and so on. However, regardless of the exact configuration, the main principle remains: the developer is the driver of the car and QA is the seat belt and airbag.
Is it not more economical to have engineers write code and testers test?
I was once asked the question: “Why should we spend engineering time for testing, when QAs can do the testing and engineers can just focus on writing code? Engineers are more expensive anyway.”
There are two flaws with this argument:
1) You cannot assure the quality into the software
If engineers are solely focused on writing code this means they are not focused on quality. Their quality awareness will greatly be diminished as their goal would be to just write some code and throw it over the fence to QA. The code quality will drastically diminish and the number of defects will soon rise to such a level that QAs will not be able to keep up. Ultimately the entire code base will become very brittle and any new changes will be riddled with breaking existing functionality. Maintenance of this code will become extremely difficult.
2) A defect becomes more expensive the further it gets from the developer
Short feedback loops or “failing fast” is an extremely valuable concept in software development and it applies here as well. This principle means that if there is a failure in the system, or process, the sooner we know the better. Imagine you are on a road trip and accidentally take a wrong turn. The sooner you discover that, the less costly in time, gas, etc. it will be.
Consider this scenario:
Developer – focused on just writing code, completes a functionality and tosses it over the fence to QA
QA – takes them a day or two (sometimes more) to get to this new code and test it
QA – finds a bug and sends it back to the developer
Developer – by now has moved on to the next task and working on it
Developer – now puts the next task on hold, has to go back and remember what they did with the previous one, then figure out what is wrong with it, then fix it, then send it back to QA again
QA – now busy with testing other work, needs 2-3 days again to validate the fix
Developer – switches context again to remember where they left off the second task, before they were interrupted by QA who found the defect
QA – …
This type of back and forth interruption and switching of context compounds very quickly and greatly slows down productivity.
Therefore, it is not more economical to have developers just develop and QA do all the testing.
Should a software team have QA Testers to begin with?
Another question arises whether a software team even needs QAs. If the engineers are responsible for “driving the car”, couldn’t they do the assurance testing as well? This might be more of a philosophical question question but here are some thoughts:
Both phases of quality ownership have to be there: during development and inspection after. In most cases, someone who did not write the code is better positioned to do the inspection after, because they come from a different context than the author. For this reason, a QA, a Product Manager, or even another fellow engineer might be better suited to do the inspection. The best approach here will again depend on the team composition, number of people, and their skills and maturity.
It is true that, if developers have to build the quality to begin with, and do the inspection afterwards, that leaves less time for development. Therefore, most software companies choose to have QAs on their teams.
On the flip side, I used to lead the engineering department of a SaaS company which: did not have any QAs (engineers only), had nearly 100% test automation code coverage, and had near zero outstanding defects in a code base that was 10 years old. Let that sink in for a moment.
I cannot take credit for what I described above and in my 22+ years in the software industry this is the only example that I have seen with this type of setup and results. The point is – it is possible to maintain very high quality without QAs, but it takes maturity, discipline and an all team commitment to quality and automation.
Who decides the appropriate level of quality?
Finally, when discussing responsibilities for software product quality, we want to discuss appropriate levels of code quality. In other words, how do we know how much quality is acceptable and expected and whose decision is it? Let’s break it down:
What is an acceptable level of quality?
There are a number of factors that go into this. While the list below is not exhaustive it provides a general overview:
- Building a proof of concept (POC), a minimal viable product (MVP), or a mature product. While we want more rigor and higher quality in a mature product, the same level is not required for an MVP, and even less so for a POC.
- Criticality of the system: Systems where failures could lead to loss of life, significant financial losses, or severe environmental damage (e.g., healthcare, aerospace, financial systems) require higher quality standards.
- Regulatory requirements: Some industries have strict regulatory compliance needs that dictate minimum quality standards.
- Security considerations: Applications dealing with sensitive data or critical infrastructure may need higher quality standards to prevent vulnerabilities.
- Performance requirements: Systems with strict performance needs (e.g., real-time systems) may require higher quality standards to meet these requirements.
Based on these factors (and others) every company will have to decide its own appropriate level of software quality.
Who decides the acceptable level of quality?
If Engineering owns quality, can they decide the minimum level before shipping? The simple answer is yes – if Engineering is responsible and accountable for quality, they should have the final word. However, there are some important considerations:
1) Business-critical situations: There might be edge cases where time is more critical than a certain level of quality for the business. In these situations:
- a) Engineering should be open to a conversation with Product.
- b) Engineering should consider the possibility of shipping something below pre-agreed quality standards.
2) Decision-making process:
- a) Engineering should still have the final word.
- b) A negotiation should be an acceptable approach.
3) Shipping below standard:
If an agreement is made to ship below standard quality: There should be a clear plan for prioritizing and closing the quality gap post-deployment.
While Engineering owns quality, flexibility and communication with other departments are important for balancing quality with business needs.