Definition of Terms
I’ve been working in software development for a while, but I’m still getting used to the usage of the term QA. The activities of the software QA groups that I have seen are centered around proving with some level of confidence that a software works, where works means it-does-what-the-requirement-says-it-should-do.
While this is important, it does not embrace the more important aspects in my understanding of Quality Assurance. So here’s my take on defining QA & QC based on experience in Software & Hardware development as well as Manufacturing management and engineering:
QA - Provides information to upper management regarding the likelihood that the marketplace will accept (buy) the product; and analyzes and improves the process that produce the products.
QC - Measures the outcomes of the process through testing and inspection and passes the results to QA for analysis.
To give this some practical sense, consider a company which designs and manufactures bicycle sprockets. At a high level the product life cycle follows a path like:
- Marketing sniffs about to determine what customers want to buy, and they detail the features, timeframes, and price that would be attractive to customers and profitable to the company.
- Engineering creates a design that achieves the product definition and creates prototypes to prove the correctness of the design.
- Process engineers create manufacturing processes to produce the design at appropriate volumes.
- Production faithfully replicates the design.
- Quality Control measures performance of Production for conformance of each sprocket to the design.
- Sales finds customers and encourages them to trade money for sprockets by taking them to lunch and giving them free pens.
- Service helps customers use, maintain, and repair their sprockets.
Notice that the QC department is mostly concerned with measuring what goes on in production. They inspect, measure and test sprockets produced by production. The results are tallied as “50 type 2 sprockets produced, 1 defect” etc.
The QA group looks at all processes used to get from Marketing’s Big Idea through design, manufacturing, delivery, and customer service; all the way to assessing customer satisfaction. Their focus is on the business sense of all the operations that are the essence of the business. It is not about perfection, but rather cost-risk-benefit balance. What process does the design group use? Is it defined, complete, and sufficient to produce a manufacturable, testable, serviceable product? Is it used? Is it cost effective? What changes can be made to the process to increase profitability and/or quality (that is the market’s perception of quality). What about the processes in manufacturing? In QC?
In short, QC checks whether the results are good; QA worries about whether it all makes sense.
Software Development is Creative
I think the physical product model makes sense to most people in an intuitive way. Even if we haven’t been involved in the development and production of physical products, we use them. We have opinions about which ones are better values, higher quality, more prestigious to own, etc.
Software development, however, is significantly different from physical product design & manufacture. The activities in the sprocket company are part creative and part repetitive. Designing the product, and designing all the processes to produce the product is highly creative. Producing instances of the product is intended to be devoid of creativity; the intent is to do the same thing faithfully, over and over. Make a sprocket. Make another sprocket, just like the previous sprocket, make more sprockets... The single most important metric is uniformity.
Software is almost entirely creative. The “production” element consists of copying the release disc, which is straightforward, simple, and commoditized. It is a tiny fraction of the overall effort, and has very low risk of failure.
Software development is costly and can have significant risks. Benefits, however, are often huge. The physical product model of QC does not offer much to software development. Sure we can test that the requirements are met. And while this is like Quality Assurance because it proves the design works, it resembles QC more than QA; we check off each requirement without thinking about whether it has value to the business or its customers.
The physical product QA model is even more important in software development than physical product development because it gives us a way to manage risk in creative endeavors. Creative endeavor; this is software development.
Identify Problems Sooner
One of the simplest concepts for improvement is the idea that finding a problem sooner is much less costly than finding it later. The idea is simple and potent. Money, especially saving some or making it do more, is a winning argument with any manager worth their salt. This is certainly true for bicycle sprocket production, and we all agree that it is true for software development too. But often, we don’t do much about it.
Cultivate a Quality Improvement Champion
This is probably the most important task you can do to improve quality.
Improving the quality of your development process will involve people outside your group and will have some cost to get started and keep going. The benefits of improving quality should eventually lead to a cash gain; this is the main selling point for a manager to champion the effort.
Start where you can (perhaps that is within your development team), apply these principals, and use your successes to radiate upward through the management structure. Managers love success and ideas with a track record.
The higher in the organization your champion is, the greater the impact QA principles will have.
Identify Your Process
The overall plan here is to see what goes wrong, and figure out what changes to your process could prevent that error from recurring. The implication is that a process exists. This step is where you document your current process so that you have something to improve when an error is discovered.
The level of detail needed here will depend on your situation, but to get started it can be simple. When you “improve” the process later, it will grow more detailed in the correct way. As an analogy, this is like test driven development: each new step refines (refactors) all the previous steps just enough so that the process handles all known problems. If a new problem is discovered, refine some more.
Things will go wrong; don’t obsess with achieving perfection. Instead, look at the problems and trace back through your process to determine the earliest practical point in the process where that error could be identified and corrected. This will (almost) always be the lowest cost option for preventing recurrence of that problem. If you’re lucky, the fix will prevent a class of problems.
Refine the Process
Improving the process should be viewed as a positive event. It indicates that your organization is actively improving. Depending on the change, you will need to involve all the people concerned so that implementation is smooth. Monitor the process to make sure the change doesn’t have unintended side effects.
Applying The Ideas Via Agile Development
Using agile development techniques addresses many of these ideas.
The retrospective at the close of each sprint is a proactive effort to find out what worked well and what didn’t. The obvious take away is to promote ideas that have good outcomes and throw out ideas that had bad outcomes. The fact that this occurs at the end of each sprint (so every 2 - 4 weeks) means that we are actively examining what we’re doing and trying to improve.
We also demonstrate the working software created during the sprint to the business stakeholders. Those folks are the experts at what the software should do. This is great feedback into the QA process, letting us know if we are on the wrong path before the budget is consumed.
We often push this to even higher value by engaging the business stakeholders in our daily stand-ups where they can help refine our understanding of the requirements, often averting errors before they are coded.
Access to Development Builds
Giving business stakeholders access to the developer’s continuous integration (CI) deployments let’s them see the results of requirements discussions within hours. This has a dual benefit of verifying our understanding of the requirements and generating enthusiasm and engagement in the business stakeholders.
Automated Testing and Scripting
Automated tests running on the CI server ensure that errors accidentally introduced after the code is created, cannot go undetected. This aligns with the goal of finding and fixing problems as early in the process as possible.
Scripting tasks where possible rather than relying on manual processes eliminates human errors and usually work much faster. For example deploying to development, test and production servers using a script results in lower cost and consistent quality. The scripts can then be controlled like source code, thus any changes to the script (now a process) are recorded.
All these techniques are orchestrated by the agile project manager, who has the role of removing roadblocks so that the self-organizing team can focus on creating great, useful software products. Monitoring and refining the process is a key activity for the project manager.
There is still a need to provide oversight and consistency across multiple cross functional development teams and multiple product efforts. Agile development process can’t address the entire product lifecycle, although most issues are covered. The next tier up is Program or Product management and scrum-of-scrums where development teams each send a person to a higher level stand-up where those coordination issues can be surfaced.