Beyond Testing: Ensure Built-in Quality

Who ensures project consistency? The answer may seem obvious. It's usually the one with "Quality" in their job title who champions excellence. However, when we delve into the roles of QA engineers, we find that many focus more on testing than quality assurance. Can testing alone guarantee our product's consistency with stakeholder expectations and maintain essential quality standards? Unfortunately, no.

In this article, our Director of QA Office, Ihor Kosheliev, discusses:

• How to create a flawless product without defects, especially when the QA team has yet to expand.
• The importance of establishing criteria before coding to ensure we're on the right track toward a clear, high-quality product.
• Ways to help the team shift their perspective toward meeting user needs.
• The conceptual approach to QA and who bears the responsibility for upholding standards.

Testing vs. Quality Assurance: Understanding the Difference

Let's first clarify the difference between testing and quality assurance.

Testing focuses on finding errors and defects in a product. It primarily deals with one aspect of quality: the absence of issues.

Quality assurance (QA) serves a dual purpose. Like testing, it involves error detection, but QA goes further. Quality assurance ensures that the product aligns with stakeholder expectations and needs. It starts early in the software development process, during requirements gathering, and continues throughout the development lifecycle. This approach helps develop the right features to ensure the delivered product meets initial expectations.

To illustrate this difference, consider road traffic and adhering to speed limits. When a driver exceeds the speed limit, the police issue a fine (similar to finding defects). However, the overall responsibility for quality assurance regarding speed adherence lies with all road users.

Here's a nuance: When the police issue a fine, and the driver pays it and continues to obey the speed limit, the desired quality of road traffic is achieved. However, for the driver, the cost of correction becomes higher than if they had followed the traffic rules from the beginning.

Similarly, while defects can be detected and fixed at any stage in software development, identifying and correcting errors early is less resource-intensive and costly. Early quality assurance helps prevent issues, rather than finding and addressing them later.

However, if these issues go unnoticed by testers, the cost of addressing defects and their impact on the product increases significantly. Such errors can result in audience loss, financial losses, reduced user satisfaction, and longer response times for bug resolution.

Quality assurance cannot rely solely on a tester because it demands a more comprehensive approach. Testers conduct post-factum reviews, but this is only part of the process. 

To understand, consider the speeding driver and police officer analogy again. Testers are like police officers who identify violations (control quality or ascertain compliance) but don't ensure them. For that, we need additional mechanisms, which we'll discuss later.

Let's also consider the scenario where the project doesn't allow for expanding the QA team. QA teamIn this case, error correction becomes more expensive and time-consuming, leading to longer lead times. As the product continues to expand, this issue becomes even more pronounced.

Let's explore the “Built-in Quality” approach to address these challenges.

Built-in Quality 

For those who have traveled through Europe, it's evident that the roads there are often narrower than desired, even when there's ample space for wider ones. These narrower roads serve as built-in speed control mechanisms (i.e., quality control) because they make accelerating more challenging.

Now, how can quality be built into software development?

Requirements Stage

In the requirements stage, ensuring and building quality necessitates transparent acceptance criteria – detailed requirements for the product that serve as the basis for future acceptance testing. Acceptance criteria define the expectations our product must meet. While various team members, including developers, testers, or business analysts, can create these criteria, the ultimate responsibility to comply lies with the product owner. 

For instance, consider designing an economical car for a budget-conscious driver with a history of speeding fines. One of the requirements for this car (acceptance criteria) states that it must achieve 47 MPG when maintaining a constant speed of 60 mph.

To ensure clarity and remove ambiguity from acceptance criteria, teams often use the “three amigos” meeting structure. This meeting involves collaboration between a business representative (Product Owner), a developer, and a tester, although additional participants can be included if necessary. Teams describe features using the Gherkin language during this meeting, which is familiar and understandable to all participants, facilitating mutual comprehension of the functional requirements for the product. 

The Gherkin language employs specific keywords to describe different parts of the script:

• Feature: Describes a functional capability or feature that needs implementation.
• Scenario: Describes a particular scenario or test case.
• Given: Specifies the initial system state or preconditions for the scenario.
• When: Specifies the event or action that triggers a change in the system's state.
• Then: Indicates the expected result following the preceding action.

These Gherkin-written Behavior-Driven Development (BDD) scripts can serve as the foundation for creating automated tests.

Here's a straightforward example of a Gherkin script for our car:

Feature: Economical Car

Scenario: Calculate fuel consumption

Given the car is moving at a constant speed of 60 mph

When one hour passes

Then, the car must have averaged 47 MPG

The "three amigos" meeting is a crucial part of the refinement process and occurs before sprint planning. It establishes a shared understanding of the behavioral requirements for the forthcoming software, which will subsequently be used as the basis for acceptance testing.

Development Stage

We understand that quality assurance is about delivering the right feature and ensuring it's built correctly. The Development stage primarily focuses on the former aspect. Here, we adopt the Test Driven Development (TDD) approach, where we start by writing tests, followed by the code that passes these tests, and then refactor the code for cleanliness. This approach results in minimal code and robust, adaptable systems. It also extends to bug fixing – it's not enough to fix bugs; we must also create tests for them to prevent recurrence.

We can check if the tank capacity meets expectations for our car example. We developed a sensor that indicates when precisely 16 gallons of fuel have filled the tank without overflowing. This serves as our test. Then, as we pour fuel into the tank, we attempt to meet this test by filling the tank to the required volume without overflow.

Acceptance Testing Stage

It's time to validate if our product aligns with the criteria established during the initial phase using Behavior Driven Development (BDD). We also ensure that it's ready for use and behaves as expected.

Once the car passes the Acceptance testing stage, we test its fuel efficiency under real conditions. The necessary steps include:

- Set the car engine's constant speed to 60 mph.

- Drive the car for one hour. 

- Expected result: The car should average 47 MPG.

Demonstration Stage

It involves applying Lean UX practices for continuous and iterative user experience improvement. At this point, we demonstrate that we are delivering the right feature correctly.

We achieve this by rapidly creating a minimum viable product (MVP) containing essential functions, allowing testing on a small user group (e.g., a test car lacking features like air conditioning or parking sensors but consuming an average of 47 MPG).

The team then iteratively improves the product based on customer feedback.

Who Might Not Find Built-in Quality Suitable?

Although Built-in Quality is crucial to ensuring overall quality, it comes with certain nuances.

Built-in Quality largely revolves around a team's readiness to receive and act upon feedback.

In essence, quality is all about feedback. The most dependable way to maintain quality and alignment with stakeholder needs is by collecting feedback from these stakeholders and users. Therefore, a team aiming to "build quality" must be prepared to receive — or actively seek — feedback and adjust accordingly.

For instance, following the Think Test-First approach and a left-shifted model, testing can begin once sufficient code is available. Launching testing, receiving feedback, and adjusting accordingly are part of this process.

Due to the significance of feedback and the potential variability in requirements (expectations), integrating Built-in Quality into Waterfall methodology projects seems unlikely. Regular feedback is typically not incorporated in such projects, and defects and issues are often discovered only at the end of the development process.

Conversely, in Agile projects, particularly Scrum, iterative sprints enable continuous evaluation of whether the product still aligns with stakeholders' expectations, allowing scope adjustments if needed. Within the Agile framework, quality is also solidified through team agreements on the "definition of ready" and "definition of done." These lists of requirements help determine when a task can be started and when it can be considered complete. For instance, the “definition of ready” might include creating scripts for acceptance testing, while the “definition of done” might involve code reviews to ensure compliance with quality standards.

Understanding these concepts encourages us to shift our focus from merely moving tickets to the "Done" column to evaluating the product in terms of how well it meets the requirements and needs of all stakeholders. Of course, every project and industry comes with unique requirements and constraints so Built-in Quality practices may need adaptation based on specific conditions. For instance, small projects with limited functionality may not support Behavior-Driven Development (BDD) implementation.

Time and resources are often constraints in project management. While Built-in Quality is essential, it must be balanced with time, budget, and scope constraints.

For example, implementing Built-in Quality and the Test Driven Development (TDD) approach can lead to an inverted pyramid of testing: numerous unit tests, slightly more integration and End-to-End (E2E) tests, and an increasing number of manual tests as the software grows. This can strain the QA team's time and resources, especially for regression or smoke testing.

The Test Automation Pyramid can help address this challenge, but it also requires a significant time investment to be effective. In short-term or MVP projects with limited budgets, the focus should be on:

• Automating testing for key scenarios rather than attempting to cover all functionality (which can be time-consuming).
• Adopting an iterative approach: Write code, test, gather feedback, adjust, and repeat.
• Leveraging Agile and Lean principles for nimble responses to changes and resource-efficient practices. Tools like kanban boards can aid in organizing work efficiently.
• Promoting a culture where quality isn't just the responsibility of QA but is prioritized by the entire team.

Built-in Quality necessitates collaboration and communication among team members. Without clear communication channels, a transparent and systematic communication culture, and a commitment to prioritizing quality, achieving its benefits may be challenging.

So, What is the Result?

As we can see, all or most elements of Built-in Quality are well-known and likely already practiced. However, combined with the Built-in Quality approach, they provide a fresh perspective on quality assurance and ensure continuous improvement.

Of course, this approach requires patience and a long-term commitment. Built-in Quality should be viewed as a valuable tool rather than a universal solution for all issues, as it has its advantages and disadvantages. The decision to implement it should be made considering your project's specific needs and characteristics. Nevertheless, individual elements of Built-in Quality and its underlying philosophy should be considered.