The Testing Infrastructure
This section explains the testing infrastructure of Perl we used in the quick start. We explain how to write basic tests and how to run them.
What are tests anyway?
Tests are part of a test program which itself is a normal Perl program. It can be run like a normal program. The real
magic happens when you call it with prove as we will see in a few seconds.
A test compares a result computed by the code under test with an expected result. In Perl we have many modules with
special comparisons, let’s start with an example using the the most basic one: Test::More.
1 use strict;
2 use warnings;
3
4 use Test::More;
5
6 subtest 'basic comparison of constants' => sub {
7 my $expected = 'success';
8 my $got = 'error';
9
10 is $got, $expected;
11 };
12
13 done_testing;
The test program uses the function is which is imported by the module Test::More to compare an expected result with
a computed one. Traditionally in Perl those two values are put in two variables named $expected and $got. It’s best
practice to follow that tradition.
As you can see, both values differ, so our test should fail. We use the function subtest (which is also imported by
Test::More) to give our test a descriptive name of what it does.
done_testing tells the testing infrastructure that we’re done with our tests. We’ll explain that in a few paragraphs.
Proving that we are wrong
So let’s run the test program with prove and look at it’s output.
Whoa! That’s a lot of output for such a simple test program. Let’s look at it bit by bit.
The output can be divided in two sections: the first section contains the test output. That are the lines starting with the pound symbol. The second and shorter section is headed by it’s name: Test Summary Report. We will discuss it when we have more tests, you can ignore it for now except the last line: FAIL. This meets our expectations.
The test output tells us quite clearly what’s wrong: two values differ, and 1 of 1 tests fail. The output even tells us what line to look at in which file, which is quite handy if our code base gets bigger. And there’s the red coloured eye catcher pointing us at the failed test program.
So nows let’s correct our error to see what prove tells us when all tests pass.
1 use strict;
2 use warnings;
3
4 use Test::More;
5
6 subtest 'basic comparison of constants' => sub {
7 my $expected = 'success';
8 my $got = 'success';
9
10 is $got, $expected;
11 };
12
13 done_testing;
Again, run it with prove:
That’s much better now! And since there are no errors to correct we have much lesser output. And we have a nice green looking line telling us that all things went well.
This simple example is a display of the basic red-green-cycle: First we have a failing test indicated by a symbolic red light, and after we have corrected our code the green light is lit.
The “Test Anything” Protocol
The Perl folks invented a protocol for software testing along the way: The “Test Anything” Protocol, or TAP in short.
Basically it’s a way of collecting the information of the success or failure of tests. We can see it’s full beauty when
we run prove with the -v switch (for verbose output).
The output starts with the name of the test program followed by two dots. Then we see the test name after a pound sign,
and the tests of that subtest indented. The 1..1 and the following lines tell us that prove ran 1 of 1 tests and
everything is “ok”.
The line before the final line holds statistical information.
Usually you’ll look at this verbose information only if you’re curious about the strange output you’re seeing in the
console. It can be quite long, so normally you wouldn’t call prove with the -v switch.
TAP was thought to become a standard and several implementation for different languages exist, but in practice it is rarely used outside of the Perl community. There is a dedicated website testanything.org where you can find the specification, history and other useful information.
Testing a subroutine
Let’s get away from this tiny example and work on a real function that reads the contents of a file. How hard can that be?
We’ll let our development driven by tests, so let’s write the test first. That way we design the API before we have
written a single line of code. Our function reads a file, so we’ll just name it that way: read-file. It gets the file
to read as it’s single argument called $filename. It’s in a module called FileRead.
But what file should it read? It’s surely not a good idea to hard-wire the name of a file that’s in our home directory
since this won’t be there if some other user calls the test program. It’s best practice to either generate the test data
that’s needed or to include it along with the tests. We’ll choose the latter option and put the file containing some
random text in t/data/read-file-happy-path.txt.
So here’s the test:
1 use strict;
2 use warnings;
3
4 use Test::More;
5 use FileRead;
6
7 subtest 'reading an existing file (happy path)' => sub {
8 my $filename = 't/data/read-file-happy-path.t';
9
10 my $expected = 'random content';
11 my $got = read_file($filename);
12
13 is $got, $expected;
14 };
15
16 done_testing;
Let’s run it with prove:
It fails. Quite loudly. This is no surprise since we haven’t written the code yet, so let’s add the subroutine’s
definition in the module ReadFile:
1 package FileRead;
2 use strict;
3 use warnings;
4 use base 'Exporter';
5
6 our @EXPORT_OK = qw(read_file);
7
8 use Path::Tiny;
9
10 use v5.20;
11 use feature qw(signatures);
12 no warnings qw(experimental::signatures);
13
14 sub read_file ($filename) {
15 my $file = path($filename);
16 return $file->slurp;
17 }
18
19 1;
Run again with prove, this time using the switch -l which adds the lib directory to the module search path. This
should work now:
Oh, we forgot the test file. Correct that by creating it and run again with prove -l:
Now for some real world cases. The function is a tool that’s used by others and may be misused. What if the argument is undefined? Or the file doesn’t exist? Or can’t be read by the user calling the function? Or something else goes wrong whilst reading the file? Or function should check for that.
Let’s tickle that one by one. First, what if the subroutine is called with an undefined file name? We expect the
function to return undef in that case, so we’ll add a test that checks for that case and run it with prove -l:
1 use strict;
2 use warnings;
3
4 use Test::More;
5 use FileRead;
6
7 subtest 'reading an existing file (happy path)' => sub {
8 my $filename = 't/data/read-file-happy-path.t';
9 my $expected = 'random content';
10
11 my $got = read_file($filename);
12
13 is $got, $expected;
14 };
15
16 subtest 'trying to read a file with undefined name' => sub {
17 my $filename = undef;
18 my $expected = undef;
19
20 my $got = read_file($filename);
21
22 is $got, $expected;
23 };
24
25 done_testing;
Well, not quite. It dies and throws an expection. (We won’t include the exact error message so you have the call the
example by yourself.) We’ll have to catch that one using Try::Tiny and return undef if it is catched:
1 package FileReadWithTry;
2 use strict;
3 use warnings;
4 use base 'Exporter';
5
6 our @EXPORT_OK = qw(read_file);
7
8 use Path::Tiny;
9 use Try::Tiny;
10
11 use v5.20;
12 use feature qw(signatures);
13 no warnings qw(experimental::signatures);
14
15 sub read_file ($filename) {
16 my $contents;
17 try {
18 my $file = path($filename);
19 $contents=$file->slurp;
20 };
21 return $contents;
22 }
23
24 1;
Run it with prove -l:
To reduce the verbosity of this text we’ll assume that you know how to run a test program from this point on and won’t
include it anymore. Additionally, we hightlight only the interesting parts of prove’s output.
Arrange, Act and Assert
In this example we display the testing pattern called “AAA”: arrange, act, assert. Again, this is a best practice pattern. First we arrange for the preconditions, than we act on the function to test, and then we assert that the expected result has occurred.
Arrange:
my $filename;
my $expected;
Act:
my $got = read_file($filename);
Assert:
is $got, $expected;
Now that we know the AAA concept, let’s try to refactor our test program for the happy path and follow that principle. We hard-wired the file to read from and needed an extra file somewhere else. Our goal is to have no unneccessary hard-wired items in our code, and since test code is just normal code we apply our usual standards to test code.
So let’s fix that: We don’t expect the file to exist but we create it on the fly. We need the file just for the test, so
it’s a temporary file. Guess what, we have a module that does just that: Path::Tiny has a method for that.
The arrage phase of our happy path test creates the temporary file and writes contents to it, the act phase calls
read_file, and the assert phase checks that we read the contents we’ve written to the file:
1 use strict;
2 use warnings;
3
4 use Test::More;
5 use File::Temp qw(tempfile);
6 use FileRead;
7
8 subtest 'reading an existing file (happy path)' => sub {
9 my ( $fh, $filename ) = tempfile( UNLINK => 1 );
10 my $content = 'random content';
11 print $fh $content;
12 close $fh;
13 my $expected = $content;
14
15 my $got = read_file($filename);
16
17 is $got, $expected;
18 };
19
20 done_testing;
Check that the test is still green by running prove.
Key Take Aways
- Tests are part of test programs that are called by the tool
prove. - The functions doing the actual testing produce output adhering to the “Test Anything” Protocol (TAP).
- If your test needs additional data, it’s best practice to generate it in the test itself or bundle it with the test in
a special directory under
t. - It’s a good practice to write a test first, and the code fulfilling the test afterwards. This is the “red-green cycle”.
- It’s best practice to structure tests following the “Arrange, Act, Assert” pattern.
- It’s good practice to treat your test code as normal code and apply your standard coding rules.