3. Introduction

3.1 Why is testability important?

How can software teams adapt and change the software systems they build and run? How can we get ‘early warning’ of performance, compatibility and integration problems? How can we design software architectures that enable ongoing evolution? The answer is to have a sustainable focus on testability.

Testability is a vital property of modern software. It enables software teams to make changes rapidly and safely. With clear feedback loops to understand the impact of changes. The testability of your software product links to its operability and maintainability. There is a close relationship between the extent to which the software meets customer needs and how testable it is. If you want to drive improvements in both speed and agility, testability is the fuel for organizations delivering modern software.

Team dynamics

Imagine if there was a constant focus on how to observe, control and understand the product within your team. Teams need all kinds of skills, and everyone brings different skills to the table. Developers need it to implement change confidently, testers need it to assess risk and provide actionable information, customers need it to achieve their goals with the product. Creating a focus on testability can help bring teams and their customers closer together.

Predictability

Have you ever been asked if your team could go a little bit faster? Or worse, as an individual on behalf of the team? We know we have. We believe that what your customers are really asking for is a little more predictability, delivering software consistently to a known level of quality. Being able to observe, control and understand a system describes predictability pretty well. If your system struggles with those three characteristics, how can you imagine delivering in a predictable fashion? Testability isn’t the only piece of the predictability puzzle, but it’s a big part.

Supporting your business and customers

A focus on testability helps to support many aspects of your business. Operations can control the system, product management can experiment with features for feedback, marketing and design can get feedback on who is using the product and if customer’s needs are being met. Most products have peak consumption periods when many stakeholders use the system simultaneously. This could be due to a major music or sporting event, for example.

Fast feedback

For us, one of the real questions in software development is: how can we get meaningful feedback on what we have created? Be it quantitative feedback through usage statistics and financial measures, or qualitative interpretation of customer experience, enhancing testability can enrich this feedback.

This is also true for feedback on system failure. Meaningful information on failure modes enhances your ability to test a system and its time to recovery after an outage or error, which is crucial for those who support the system. When things go wrong, being able to observe and control the system will be invaluable to both business and technical operations.

Data, not opinion

Everyone in the team has opinions. However, focusing on testability as a team will help you go beyond opinions and have data to support you. Being able to objectively test the assumptions and hypotheses behind the functional and operational features you build helps establish a dialogue with product stakeholders. Enhancing testability allows quantifying the business and operational impact, while also moving conversations away from those who shout loudest to those with the right data.

3.2 What does hard-to-test feel like?

Take your mind back to ‘that’ product. The one which caused you quite a lot of pain, phone buzzing in the middle of the night, and yet another failed cron job. The product that caused a lot of distress to the poor operations people when you handed them a complete mess.

Do you recognize any of these traits?

• Interacting with a product gives you no feedback. No logs, no audit trail, only mysterious unmonitored dependencies. You don’t know if anything went well. Or badly.
• Interacting with a product gives you vast amounts of feedback. Log files spam uncontrollably, lights twinkle on dashboards, integrated dependencies give strange, opaque answers. You don’t know if anything went well. Or badly.
• You release your product. Scores of utterly baffling issues pop up. Seemingly unrelated but somehow intertwined. Next release makes you twitchy.

3.3 What does testable feel like?

We would like to evoke a different feeling when you talk about your next project or product. Imagine a product where:

• You are in control of the amount of feedback your product gives you, allowing either a deep, detailed view or a wide, shallow view. Rather than trying to parse what your product is doing, it will tell its story.
• The product can be set into a state of your choosing, whether that be data or configuration, allowing you to develop your product with much more certainty.
• After release, you are not dreading the call from support or account management that your customers are unhappy. Any problems are flagged early and can be proactively managed.

3.4 What leads to testability being neglected?

During our careers, we have seen various maturity models of the “‘ilities” of software deployed within organizations. Testability might make an appearance every now and then but it rarely seems to rank as high as scalability or reliability. We hope to give strategies for overcoming apathy and some common constraints to testability:

• No paradigm - if you don’t understand what it is, how can you ask for it? Never mind describing its benefits to those who are paying for the product development.
• Nobody knows who should be responsible for it - product people think it has nothing to do with them, developers think it’s testers’ responsibility, testers don’t communicate the pain of a hard-to-test system to developers.
• There’s no time - “we need to build the thing now”, when the pressure is on, with deadlines looming, there is no time for testability.
• It’s not a functional requirement - testability and other operational features never make it into the backlog. But these features are what turns functionality into a product.

3.5 What is covered in this book?

Testability is a vast topic, which permeates much of software development. Our focus will be on how the team can enhance their systems and interactions with stakeholders through a focus on testability. We will share our experiences, those of others we have encountered along the way and many practical resources and references we can use on our testability journey.

What we will cover includes:

• Improving interactions within and between teams to enhance testability both through conversation and documentation
• Practical approaches for creating testable architectures, addressing common risks and useful solutions
• Taming your environments from local, disposable environments all the way to live ones
• Keeping the focus on testability through the concept to customer and beyond the lifecycle of a product

Although this book touches on some technical aspects of software testing, it doesn’t focus on how to make specific artifacts like legacy code or user interfaces more testable. There is a lot of great guidance out there for those, for example “Working Effectively with Legacy Code” by Michael Feathers (2004). Testability is not only about testers and, by extension, not solely about testing!

3.6 How to use this book

Ideally, each chapter is meaningful independently. You should be able to identify your situation and pick the thinking, tools and techniques that help you most in your current situation. However, like most books, this also represents our story, what we have learned over the years. Hopefully those lessons can help you with your lessons, as a holistic experience.

Concepts and theory are important, but we value practical examples highly in this book and have sourced them from various friends within the world of software development, as well as ourselves. These examples are critical to the book and will hopefully provide value to our readers.

Most of all, we believe the process of developing systems and products is a joyful, collaborative endeavor. We hope that shines through the book!

3.7 Why we wrote this book

As experienced testers, we have seen teams struggling with hard-to-test systems for a long time. This has usually manifested itself in pain for testers of all kinds so we felt it. In our experience, there has been little will do anything about it. Through lack of knowledge, advocacy and an acceptance that hard to test is normal. Team testing cultures are hard to create. Enhancing testability makes this journey much easier.

Looking around at the wider body of knowledge about testability, we wanted to fill some gaps. In particular, creating architectures designed for testability has light coverage. Once locked into a hard to test architecture, retrofitting testability is a challenging journey. We hope the book helps those working on systems of all ages to improve their architectures.

We are both testers who have looked beyond our role into other disciplines, engaged in the whole system, including people, processes, products and much more. Improving testability involves that wider system over optimizing for local gains. It’s time for a new practical focus on testability and this book contains the tools for teams to do that.

3.8 Feedback and suggestions

We’d welcome feedback and suggestions for changes: please contact us at info@confluxbooks.com, via @TestabilityBook on Twitter, or on the Leanpub discussion:
https://leanpub.com/SoftwareTestability/feedback

Ash Winter & Rob Meaney - August 2021