We’ve decided we are going to do a series based on our seven success factors from our first book Agile Testing: A Practical Guide for Testers and Agile Teams. We’ll bring in some new experiences and show why we still think they are important and what we’d change if we rewrote them.
The seven success factors are:
1. Use the whole-team approach
2. Adopt an agile testing mind-set
3. Automate regression testing
4. Provide and obtain feedback
5. Build a foundation of core agile practices
6. Collaborate with customers
7. Look at the big picture
We start the series looking at #7.
Success Factor #7 – Look at the big picture
In the summary chapter of Agile Testing, we talked about looking at a product from a customer’s perspective and thinking about all the different types of testing that is needed. In our second book, More Agile Testing, we introduce the idea of thinking in layers. The system has many features, a feature has many (user) stories. The stories are testable, but often teams forget that features are really the capabilities that the customer wants. We need to test at the feature level as well as the story level, considering how people will use that functionality in production.
When we think about building quality in, we need to start with understanding the feature. Asking questions like: What problem are we trying to solve for the customer? For the business?” Have you ever experienced creating a great new feature that you are sure the customer will love, only to have them say, “This isn’t what I need?”
A story Lisa likes to share is about one of her teams that worked hard to deliver the first “learning release” of a feature many customers had requested. They tested extensively to make sure the information appearing in the UI was correct. However, once the feature was released, only a handful of customers even gave it a try. From production usage data, it was clear that people couldn’t even understand what the new UI page was supposed to do for them. The team had not adequately tested the usability of the page design. They focused on the functionality and forgot the bigger picture.
Testing activities should also include looking at the system as a whole. Before your team starts working on a new feature, discuss questions such as: What impacts might this feature have on the rest of the system? Do we have existing functionality that we need to change? What quality attributes do we need to consider? Accessibility, or security, or any other constraints your application may have. Start thinking about how you are going to create your test automation at this point – for example, Will you have the right tools? Is there a good way to layer the automation so that much of it can be done by the programmers. Also consider what telemetry you might need. What information do you want to collect when your application is in production? What events need to be logged for monitoring and observability?
Once you understand the feature, then you can slice it into small testable stories, thinking about the constraints with every story. Practice acceptance test-driven development (ATDD) or behaviour-driven development (BDD), guiding your development with examples and testing each story as it is implemented. The developers will be testing as well, using practices like test-driven development (TDD).
After all the stories are completed for a particular feature, don’t forget to go back, and perform exploratory testing on the whole feature. Use tools like the agile testing quadrants to help remind you about what tests you may have forgotten.
Janet had an experience where designers created mock-ups for a website for her to give feedback on. She looked at the content, how the page looked and tried hard to understand how it linked together. Some things were fixed early, but when they were implemented, she realized there were a lot of other issues that she didn’t find until she could touch the different pages.
Once those were fixed, the team thought they were ready for production - until the developer asked her a question about how the users would do a particular action. Janet realized that she had been testing only from her own point of view. She explored the new design some more, using the different customer personas, and found some very interesting bugs.
Summary
Each story is part of a feature, and each feature is part your product with potential impacts on other features. Test the story, test the feature, and explore how that feature works in the context of the entire product. Remember the big picture! This news story is a great example of what happens when you only test the small chunks. https://www.engadget.com/2020/02/29/boeing-starliner-failed-first-flight-report/