Selection of examples and finalization criteria
One of the most frequent questions when you start doing TDD is how many tests do you have to write until you can consider the development to be finished. The short answer is: you’ll have to do all of the necessary tests, and not one more. The long answer is this chapter.
Checklist driven testing
A good technique would be to follow Kent Beck’s advice and write a control list or check-list in which to annotate all of the behaviors that we want to implement. Obviously, as we complete each behavior, we cross it off the list.
It’s also possible that, during the work, we discover that we need to test some other behavior, that we can remove some of the elements of the list, or that we’re interested in changing the order that we had planned. Of course, we can do all of this as convenient.
The list is nothing more than a tool to not have to rely on our memory so much during the process. After all, one of the benefits of doing Test Driven Development is to reduce the amount of information and knowlege that we have to use in each phase of the development process. Each TDD cycle involves a very small problem, that we can solve with pretty little effort. Smalls steps that end up carrying us very far.
Let’s see an example with the Leap Year kata, in which we have to create a function to calculate if a year is a leap year or not. A possible control list would be this one:
1 Leap Year Checklist
2
3 * Years that aren't divisible by 4 are regular years
4 * Years divisible by 4 are leap years
5 * If they're divisible by 100, they're leap years
6 * If they're divisible by 400, then they're not leap years
Another example for the Prime Factors kata, in which the exercise consists in developing a function that returns the prime factors of a number:
1 Checklist for Prime Factors
2
3 * Numbers that don't have any prime factors
4 * Prime numbers (the only prime factor is the number itself)
5 * Non-prime numbers:
6 * Powers of just one prime number
7 * Product of different prime factors
Example selection
For each behavior that we want to implement, we’ll need a certain amount of examples with which to write the tests. In the following chapter we’ll see that TDD has two principal moments: one related to the establishment of the interface of the unit that we’re creating, and other in which we develop the behavior itself. It’s at this moment when we need examples that question the current implementation and force us to introduce code that produces the desired behavior.
A good idea is, therefore, to take not of several possible examples with which to test each of the items of the control list.
But, how many examples are necessary? In QA there are various techniques to choose representative examples with which to generate the tests, but they have the goal og optimizing the relationship between the number of tests and their ability to cover the possible scenarios.
We can use some of them in TDD, although in a slightly different manner, as we’ll see next. Keep in mind that we use TDD to develop an algorithm, and in many cases, we discover it as we go. For that, we’ll need several examples related to the same behavior, in such a way that we can identify patterns and discover how to generalize it.
The techniques that we’re going to look at are:
Partition by equivalence class
This technique relies on one idea: that the set of all possible conceivable cases can be divided in classes according to some criterion. All of the examples in a class would be equivalent between them, so it would suffice to test with only one example from each class, as all are equally representative.
Limit analysis
This technique is similar to the previous one, but paying attention to the limits or boundaries between classes. We choose two examples from each class: precisely those that lie at its limits. Both examples are representatives of the class, but they lets us study what happens at the extremes of the interval.
It’s mainly used when the examples are continuous data, or we care especially about the change that occurs when passing from one class to another. Specifically, it’s the kind of situation where the result depends on whether the value being considered is larger, strictly larger, etc.
Decision table
The decision table is nothing more than the result of combining the possible values, grouped by classes, of the parameters that are passed to the unit under test.
Let’s take a look the election of examples in the case of Leap Year. For this, we being with the list:
1 Leap Year Checklist
2
3 * Years that aren't divisible by 4 are regular years
4 * Years divisible by 4 are leap years
5 * If they're divisible by 100, they're leap years
6 * If they're divisible by 400, then they're not leap years
Let’s see the first item. We could use any number that’s not divisible by 4:
1 * Years that aren't divisible by 4 are regular years
2
3 Examples: 1997, 2021, 1825
In the second item, the examples should meet the condition of being divisible by 4:
1 * Years divisible by 4 are leap years
2
3 Examples: 1996, 2000, 2020, 1600, 1800, 1900
Let’s pay attention to the next element of the list. The condition of being divisible by 100 overlaps the previous condition. Therefore, we have to remove some of the examples from the previous item.
1 * Years divisible by 4 are leap years
2
3 Examples: 1996, 2020
4
5 * If they're divisible by 100, they're leap years
6
7 Examples: 2000, 1600, 1800, 1900
And the same thing happens with the last of the elements of the list. The examples for this item are the numbers that are divisible by 400. It also overlaps the previous example:
1 * If they're divisible by 100, they're leap years
2
3 Examples: 1800, 1900
4
5 * If they're divisible by 400, then they're not leap years
6
7 Examples: 1600, 2000
This way, the example list ends up like this:
1 * Years that aren't divisible by 4 are regular years
2
3 Examples: 1997, 2021, 1825
4
5 * Years divisible by 4 are leap years
6
7 Examples: 1996, 2020
8
9 * If they're divisible by 100, they're leap years
10
11 Examples: 1800, 1900
12
13 * If they're divisible by 400, then they're not leap years
14
15 Examples: 1600, 2000
On the other hand, the example selection for Prime Factors could be this one:
1 Prime Numbers Checklist
2
3 * Numbers that don't have any prime factor
4
5 Examples: 1
6
7 * Prime numbers (the only prime factor is the number itself)
8
9 Examples: 2, 3, 5...
10
11 * Non-prime numbers:
12 * Powers of just one prime number
13
14 Examples: 4, 8, 9, 16, 27...
15
16 * Product of different prime factors
17
18 Examples: 6, 10, 12, 15, 18, 20...
Using many examples to generalize an algorithm
In simple code exercises such as the Leap Year kata, it’s relatively simple to anticipate the algorithm, so we don’t need to use too many examples to make it evolve and implement it. Actually, it would suffice to have an example from each class, as we’ve seen when we talked about the partition by equivalence classes, and in a few minutes we would be done with the problem.
However, if we’re just starting to learn TDD, it’s a good idea to go step by step. The same when we have to face complex behaviors. It’s preferable to take really small baby steps, introduce several examples, and wait to have sufficient information before trying to generalize. Having some amount of code duplication is preferable to choosing the wrong abstraction and keep constructing on top of it.
A heuristic that you may apply is the rule of three. This rule tells us that we shouldn’t try to generalize code until we have at least three repetitions of it. To do it, we’ll have to identify the parts that are fixed and the parts that change.
Consider this example, taken from an exercise from the Leap Year kata. At this point the tests are passing, but we haven’t generated an algorithm yet.
1 function leapYear(year) {
2 if (year === 1992) {
3 return true;
4 }
5 if (year === 1996) {
6 return true;
7 }
8 if (year === 2020) {
9 return true;
10 }
11
12 return false;
13 }
14
15 describe('Identify Leap Year', () => {
16 it('should be Leap Year', () => {
17 expect(leapYear(1992)).toBeTruthy();
18 expect(leapYear(1996)).toBeTruthy();
19 expect(leapYear(2020)).toBeTruthy();
20 });
21 })
There we have our three repetitions. What do the have in common apart from the if/then structure? Let’s force a small change:
1 function leapYear(year) {
2 if (year === 498 * 4) {
3 return true;
4 }
5 if (year === 499 * 4) {
6 return true;
7 }
8 if (year === 505 * 4) {
9 return true;
10 }
11
12 return false;
13 }
14
15 describe('Identify Leap Year', () => {
16 it ('should be Common Year', () => {
17 expect(leapYear(1997)).toBeFalsy();
18 expect(leapYear(1998)).toBeFalsy();
19 expect(leapYear(2021)).toBeFalsy();
20 });
21
22 it('should be Leap Year', () => {
23 expect(leapYear(1992)).toBeTruthy();
24 expect(leapYear(1996)).toBeTruthy();
25 expect(leapYear(2020)).toBeTruthy();
26 });
27 })
Clearly, the three years are divisible by 4. So we could express it in a different way:
1 function leapYear(year) {
2 if (year % 4 === 0) {
3 return true;
4 }
5 if (year % 4 === 0) {
6 return true;
7 }
8 if (year % 4 === 0) {
9 return true;
10 }
11
12 return false;
13 }
14
15 describe('Identify Leap Year', () => {
16 it ('should be Common Year', () => {
17 expect(leapYear(1997)).toBeFalsy();
18 expect(leapYear(1998)).toBeFalsy();
19 expect(leapYear(2021)).toBeFalsy();
20 });
21
22 it('should be Leap Year', () => {
23 expect(leapYear(1992)).toBeTruthy();
24 expect(leapYear(1996)).toBeTruthy();
25 expect(leapYear(2020)).toBeTruthy();
26 });
27 })
Which is now an obvious repetition and can be removed:
1 function leapYear(year) {
2 if (year % 4 === 0) {
3 return true;
4 }
5
6 return false;
7 }
8
9 describe('Identify Leap Year', () => {
10 it ('should be Common Year', () => {
11 expect(leapYear(1997)).toBeFalsy();
12 expect(leapYear(1998)).toBeFalsy();
13 expect(leapYear(2021)).toBeFalsy();
14 });
15
16 it('should be Leap Year', () => {
17 expect(leapYear(1992)).toBeTruthy();
18 expect(leapYear(1996)).toBeTruthy();
19 expect(leapYear(2020)).toBeTruthy();
20 });
21 })
This has been very obvious, of course. However, things won’t always be this easy.
In summary, if we don’t know the problem very well, it can be useful to wait until the rule of three is fulfilled before we start thinking about code generalizations. This implies that, at least, we’ll introduce three examples that represent the same class before we refactor the solution to a more general one.
Let’s see another example from the same kata:
1 function leapYear(year) {
2 if (year % 100 === 0) {
3 return false;
4 }
5
6 if (year % 4 === 0) {
7 return true;
8 }
9
10 return false;
11 }
12
13 describe('Identify Leap Year', () => {
14 it ('should be Common Year', () => {
15 expect(leapYear(1997)).toBeFalsy();
16 expect(leapYear(1998)).toBeFalsy();
17 expect(leapYear(2021)).toBeFalsy();
18 });
19
20 it('should be Leap Year', () => {
21 expect(leapYear(1992)).toBeTruthy();
22 expect(leapYear(1996)).toBeTruthy();
23 expect(leapYear(2020)).toBeTruthy();
24 });
25
26 it('should be exceptional common year', function () {
27 expect(leapYear(1700)).toBeFalsy();
28 expect(leapYear(1800)).toBeFalsy();
29 expect(leapYear(1900)).toBeFalsy();
30 });
31 })
The duplication that isn’t
The divisible by concept is pretty obvious in this occasion and we don’t really need a third case to evaluate the possibility of extracting it. But, the main thing here isn’t actually duplication. Actually, it would have been enough with one example. We have encountered the idea that the condition that is being evaluated is the fact that the year number must be divisible by a certain factor. With this refactor, we make it explicit.
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(100)) {
7 return false;
8 }
9
10 if (divisibleBy(4)) {
11 return true;
12 }
13
14 return false;
15 }
16
17 describe('Identify Leap Year', () => {
18 it ('should be Common Year', () => {
19 expect(leapYear(1997)).toBeFalsy();
20 expect(leapYear(1998)).toBeFalsy();
21 expect(leapYear(2021)).toBeFalsy();
22 });
23
24 it('should be Leap Year', () => {
25 expect(leapYear(1992)).toBeTruthy();
26 expect(leapYear(1996)).toBeTruthy();
27 expect(leapYear(2020)).toBeTruthy();
28 });
29
30 it('should be exceptional common year', function () {
31 expect(leapYear(1700)).toBeFalsy();
32 expect(leapYear(1800)).toBeFalsy();
33 expect(leapYear(1900)).toBeFalsy();
34 });
35 })
This gets clearer if we advance a bit further.
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(400)) {
7 return true;
8 }
9
10 if (divisibleBy(100)) {
11 return false;
12 }
13
14 if (divisibleBy(4)) {
15 return true;
16 }
17
18 return false;
19 }
We find the same structure repeated three times, but we cannot really extract a common concept from here. Two of the repetitions represent the same concept (leap year), but the third one represents exceptional regular duration years.
In search of the wrong abstraction
Let’s try another approach:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(4 * 4 * 5 * 5)) {
7 return true;
8 }
9
10 if (divisibleBy(4 * 5 * 5)) {
11 return false;
12 }
13
14 if (divisibleBy(4)) {
15 return true;
16 }
17
18 return false;
19 }
If we divide the year by 4, we could propose another idea, since that could help us tell apart the parts that are common from the parts that are different.
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5 year /= 4
6
7 if (divisibleBy(4 * 5 * 5)) {
8 return true;
9 }
10
11 if (divisibleBy(5 * 5)) {
12 return false;
13 }
14
15 if (divisibleBy(1)) {
16 return true;
17 }
18
19 return false;
20 }
It’s weird, but it works. Simpler:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5 year /= 4
6
7 if (divisibleBy(100)) {
8 return true;
9 }
10
11 if (divisibleBy(25)) {
12 return false;
13 }
14
15 if (divisibleBy(1)) {
16 return true;
17 }
18
19 return false;
20 }
It’s still working. But, what use is it to us?
- On the one hand, we still haven’t found a way to reconcile the three
if/thenstructures. - On the other hand, we’ve made the domain rules unrecognizable.
In other words: trying to find an abstraction relying only on the existence of code repetition can lead us to a dead end.
The correct abstraction
As we’ve pointed out before, the concept in which we’re interested is leap years and the rules that determine them. Can we make the code less repetitive? Maybe. Let’s do it again, from the top:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(400)) {
7 return true;
8 }
9
10 if (divisibleBy(100)) {
11 return false;
12 }
13
14 if (divisibleBy(4)) {
15 return true;
16 }
17
18 return false;
19 }
The question is that the “divisible by 400” is an exception to the “divisible by 100” rule:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(100) && !divisibleBy(400)) {
7 return false;
8 }
9
10 if (divisibleBy(4)) {
11 return true;
12 }
13
14 return false;
15 }
Which lets us do this and compact the solution a little bit:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(100) && !divisibleBy(400)) {
7 return false;
8 }
9
10 return divisibleBy(4);
11 }
Maybe we could make it more explicit:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 function isCommonYearExceptionally() {
7 return divisibleBy(100) && !divisibleBy(400);
8 }
9
10 if (isCommonYearExceptionally()) {
11 return false;
12 }
13
14 return divisibleBy(4);
15 }
But now it looks a bit weird, we need to be more explicit here:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 function isCommonYearExceptionally() {
7 return divisibleBy(100) && !divisibleBy(400);
8 }
9
10 function isLeapYear() {
11 return divisibleBy(4);
12 }
13
14 if (isCommonYearExceptionally()) {
15 return false;
16 }
17
18 return isLeapYear();
19 }
At this point, I wonder if this solution hasn’t become too unnatural. On the one hand, the abstraction is correct, but by taking it this far, we’re probably being guilty of a certain amount of over-engineering. The domain of the problem is very small and the rules are very simple and clear. If you compare this:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 if (divisibleBy(400)) {
7 return true;
8 }
9
10 if (divisibleBy(100)) {
11 return false;
12 }
13
14 if (divisibleBy(4)) {
15 return true;
16 }
17
18 return false;
19 }
To this:
1 function leapYear(year) {
2 function divisibleBy(divisor) {
3 return year % divisor === 0;
4 }
5
6 function isCommonYearExceptionally() {
7 return divisibleBy(100) && !divisibleBy(400);
8 }
9
10 function isLeapYear() {
11 return divisibleBy(4);
12 }
13
14 if (isCommonYearExceptionally()) {
15 return false;
16 }
17
18 return isLeapYear();
19 }
I think I would stick with the first solution. That said, in a more complex and harder to understand problem, the second solution might be a lot more appropriate, precisely because it would help us make the involved concepts explicit.
The moral of the story is that we mustn’t strive and struggle to find the perfect abstraction, but rather the one that’s sufficient at that particular moment.