Chapter 3: Agile Testing Mindset

Before teaching any techniques or theory we like to help people understand that agile testing is fundamentally a different mindset to traditional testing. This topic presents five key differences in the form of a testing manifesto, which was inspired by the agile manifesto. We often deliver this topic as a stand alone workshop, with the Jenga game at the start or even public talk. It’s a great introduction to the concepts in agile testing. Often the phrases we refer to in this topic like “Testing is an activity not a phase” are the things teams remember most and repeat to each other after we have trained them. Proof that presenting this topic this way help it stick in their minds.

4Cs Training plan

Standing Survey (C1)

This helps everyone in the room get to know a little bit about the testing problems others have. Ask people to stand if any of these are true for them:

• testing is always behind
• automation is even further behind that
• testers can’t work until development is done
• there is pressure at the end of a sprint
• there is blame around bugs (its his fault etc.)

Teach (C2)

Introduce the idea that agile testing is not just testing the same way you always have, except in sprints. Explain that the entire way the team thinks about testing should change. Then present the five mindset points below. Each is presented as the traditional way of thinking about testing and then the agile way of thinking. Contrasting like this can help people understand the differences, but it also helps them reflect which side they are closer to at the moment.

Testing is an activity not a phase

Traditionally people view testing as a phase that happens at the end of development. In agile most have changed it that the chunk of development done is smaller, but the testing still happens last. Nothing has fundamentally changed about how testing is done.

One way to see if this is the case is to ask people about their taskboards. If taskboards have a separate column for testing, it’s a sure sign that testing is still being thought of as a phase.

In contrast in agile, testing is just an activity that needs to happen, along with coding, documentation and everything else. Thinking about it like this makes it possible to consider the idea of doing testing tasks before development work. A great way to visualise this on a taskboard is that instead of having a separate column for test, rather just make testing tasks a different colour sticky note. Now put all the tasks in the “To do” column together. Challenge the team to see how many of the testing tasks they can do before any development tasks happen.

For example you can create test cases before any code is written. That way you know how you are going to test it before you build it. You could even create automated acceptance tests first. These should fail since there is no code yet, but once the code is written and the tests pass, the work is done, and there are no test tasks left. Working this way will remove the hurdle of testing always being behind. For some people this is a huge step, however just breaking the mentality that testing tasks follow development is a great start.

Another useful technique is the “Show Me” column. Put it after the “In Progress” column, before the “Done” column. Most teams do code reviews, documentation reviews or even test case reviews on each story. The idea behind the “Show Me” column is to do a review on every task as soon as that task is done. If tasks are small these are micro reviews that might only take a few minutes, but they ensure that at least two people in the team have seen every piece of work, and this can help catch and fix issues much earlier.

Slides

Prevent bugs rather than finding bugs

Traditionally people think that the goal of testing is to find bugs. In fact some organisations even measure tester productivity based on the number of bugs they find (or don’t find). Once again this mindset is limiting, and helps reinforce the idea that testing is something that happens at the end.

Use the star example to illustrate this point. Show the star slide or draw the star on a flipchart and ask people “How many points are there on this star?” People might offer a few numbers: 5, 10, 20. Ask people to write down the number of points they think it has. The ask people to raise their hands if they wrote anything other than 5.

Now show the next slide, and explain that the point inside the circle with a tick is a point (and only one point), and that the other circle with a cross is not considered a point. Hopefully now everyone can agree that there are only 5 points on a circle.

Explain that anyone who wrote down a number other than 5 created a bug. Ask if they could think of any way that they could have prevented that bug. Hopefully someone will realise that they could have asked you what you mean by a point before writing down their answer. If no one mentions this, you can. Explain that this works exactly the same way in software. Often people make assumptions about requirements and implement those assumptions before clarifying them. The assumptions are only clarified once the software is tested, and the bug is then found. Imagine how much more productive it would be to have a short conversation to clarify assumptions before anyone wrote a single line of code.

This example introduces the second agile mindset principle. Agile testing aims to prevent bugs by seeking to eliminate all assumptions and unknowns before starting to code. The goal is to make sure everyone from the customer to the developer and the tester have exactly the same understanding of how something should work. The best way to prevent bugs is to ask questions, especially stupid questions. Ask questions that everyone thinks the answer to is obvious. Remind them of the star. To some people it was obvious that their were 20 points.

Our favourite example for this was a team that needed to create a report showing the average sales data for the last six months. Everyone thought they understood the requirement perfectly, and it didn’t need much discussion. We happened to be there and we asked some questions:

• If I run the report on 1 February is data from February included or not?
• What about if I run the report on 29 February?
• How exactly should the average calculated, as monthly average, or the average over the six months?
• Does the average need to be stored or is it calculated on the fly?
• Does the report need to be stored, or will it only be created when someone selects it?
• What field in the database is the average calculated from?
• Who would be using the report and why did they need it?

It soon became very clear that no one had considered these items, and that more information was needed before they could build the report. Imagine the rework and bugs that could be created if you built this without the answers to these questions.

Slides

Don’t be a checker, be a tester

Traditional testers often don’t like agile because without detailed specification documents they are suddenly unable to do their jobs. This is because they consider their job being to compare the working system to the specification, and report where there are discrepancies. If you think about this for a second, the only thing they are checking is how closely the developers followed the specification. This actually says nothing about the quality of a product, or more importantly if it is fit for purpose.

We call this work ‘checking’. You know what’s really good at checking? Computers! Checking that 1 + 1 = 2 is easy work for computers to do, and they will get it right every single time. They don’t get bored or tired or distracted. With agile testing simple checking should be automated so that testers can be freed up to do the kind of work computers can’t do. Things like exploratory testing or usability testing.

In agile, testers need to become customer advocates. They need to deeply understand who their users are and what they are trying to achieve with the product. They should be the representative of that customer in every design decision, ensuring that the feature meets the customers actual needs, not just the specification, or even what they asked for.

When a user asks for a feature, ask them: “How would you test that?” or “How will you know if that works?”. This can help understand the real result the customer is looking for. Translating that into acceptance criteria for the team can ensure the product does the right thing.

Slides

Don’t try to break the system, instead help build the best possible system

Testers like to break stuff. Yes that’s a generalisation, but it is certainly true for the majority of testers we meet. The problem with this mindset is that it creates a divide between the developers and testers. Developers build it, then testers try to break it. See how this reinforces the other traditional mindsets like testing as a phase. When this gets really extreme some strange stuff happens, like testers telling developers how they will test the product. We like to share the following story.

Many years ago I (Karen) was working as a project manager on a traditional waterfall project. We were nearing the end of development and preparing for the final test run before user acceptance testing (UAT). I was discussing the upcoming UAT with the client, and here is what they said to me. “We don’t want to share our UAT test cases with you, because then you might just build the system to make those test cases pass”. At the time I agreed and said I understood. Now I literally laugh out loud at the absurdity of this statement. Surely the client wanted software that would make their UAT test cases pass. Wasn’t that in fact our joint goal!

The agile mindset is that testers should be helping to build the best system possible. They shouldn’t be celebrating when they find a bug, they should be celebrating with their whole team, when the product works, and solves a business problem in a simple way. The best way to do this is to figure out how to test the system from a user point of view and then share that with developers before they start coding. Chances are high if you do this, they might actually build a system to make those tests pass.

Slides

The whole team is responsible for quality, not just the tester

Traditionally it is the tester, or the test team that is responsible for quality. They get the final say in whether a product is ready to be released or not. The problem with this mindset is that it implies then that only the tester cares about quality and only the tester spends their time ensuring it happens.

Instead in agile the whole team is responsible for quality. This helps teams realise that testing is an activity they all need to take part in and that it happens throughout the work. If customers find a bug in production, no one should be asking the tester why they missed that. Instead the whole team should be discussing together how they can prevent that from happening again in the future.

Once this mindset is adopted, testers are no longer the only people busy at the end of the release, the whole team is involved.

Slides

Testing Manifesto (C3)

Hand out a pack of Testing Manifesto cards to each group. Ask them to place the cards in this way: we value __ over __ , five times. Walk around and help with reminders if anyone seems stuck. Once all groups are done, show the slide with all five statements and quickly read through it.

Slide

Hands Up (C4)

Ask people to raise their hands if the following statement is true for them.

• Do think it might be possible to prevent bugs BEFORE you write code?