In 2009 when we published Agile Testing: A Practical Guide for Testers and Agile Teams, the fifth success factor in our list was “Build a foundation of core practices”. While every team should adopt practices that work for its situation, a team without these core agile practices is unlikely to benefit much from agile values and principles.
The practices we highlighted then were: test automation, test environments, manage technical debt, work incrementally, coding and testing part of one process, and synergy between the practices.
We think they are still important, but we put a slightly different spin on them given today’s environment.
Test automation and test environments
Test automation and working test environments have become intertwined in the DevOps world. Test environments can be reframed in the form of build pipelines. We see teams that use build pipelines, have mastered the stable test environment. They have pipeline stages to spin up environments for the automated test suites. When the pipeline finishes with no errors, it automatically spins up a “preview environment” on a cloud platform and deploys a build artefact for further exploring. Unfortunately, many companies still have not embraced these practices and technology, and struggle to create adequate test or staging environments.
Teams that embrace creating their automation as they develop new features, and make it part of their definition of Done, have significantly better success in their releases. One reason for this is the fast feedback provided. (see the section on working incrementally)
Many teams are building observability (o11y) into their product to proactively fix unanticipated production problems and get detailed data on production use. This is a form of automation at its finest, giving fast feedback to teams directly from customer usage.
Observability also helps with keeping automated tests reliable and trustworthy. One of Lisa’s teams had “flaky” tests and could not figure out the causes. They added some instrumentation to the production and test code to capture more log data while automated tests ran. The additional data helped them analyze and fix the “flakiness”.
Managing Technical debt
Teams may consciously take a short cut to meet an urgent business need. They might skip over a refactoring or choose to not automate a test at this time. As long as they plan and do that refactoring or automation soon after, they can still enjoy good product quality and if desired, successful continuous delivery.
Making bad code design decisions, accumulating a huge backlog of serious bugs, and living with “flaky” automated tests is more like “technical guilt”. You must have trust in the code you write to succeed in the long term. If your team is aiming towards continuous delivery or deployment, your first step might be to manage your technical debt.
Working incrementally
Many teams are not able to deploy changes to production frequently, whether that’s for business reasons or because they’re working with older, monolithic code that makes it more difficult. Even in this context, we can get fast feedback. For example, when we first start looking at a feature, we can ask questions that uncover hidden assumptions and build shared understanding across roles.
As we have these early conversations, we can visualize behavior of the new feature, and how customers will use it with techniques like flow diagrams. This helps us identify the simplest thing we can deliver, and what could be added later. We can slice the feature into small, consistently sized stories. Frameworks like example mapping, which we talked about last month, help us dig into each story to understand its value and behavior. Programmers and testers can collaborate to turn these examples and scenarios into executable tests to guide development. That’s where the magic happens – step by step.
Coding and testing as one process
When teams work well together and understand their contribution to the quality of the product and treat coding and testing as one process, it feels like magic. Good practices like test-driven development, make other testing activities easier to accomplish because it builds testability into the product. This includes practices like programmers performing exploratory testing on their own code, doing a desk check or 'show me' immediately after code is finished, and exploratory testing on those stories that need it as soon as possible.
Janet wrote a blog post on this so we won’t go into more detail here.
Synergy between practices
You may have noticed as we talked about specific practices above, how one practice leads into another, and how closely practices are tied together. Taking one practice in isolation may not have the desired affect. Look at the big picture and consider how all the practices fit together.