What about tests?
In the domain of strong static typing (not necessarily functional) you might hear phrases like “It compiles therefore it must be correct thus we don’t need tests!”. While there is a point that certain kinds of tests can be omitted in favour of strong static typing such stances overlook that even a correctly typed program may produce the wrong output. The other extreme (coming from dynamic typed land) is to substitute typing with testing - which is even worse. Remember that testing is usually a probabilistic approach and cannot guarantee the absence of bugs. If you have ever refactored a large code base in both paradigms then you’ll very likely come to esteem a good type system.
However, we need tests, so let’s write some. But before let us think a bit about what kinds of tests we need. :-)
Our service must read and create data in the JSON format. This format should be fixed and changes to it should raise some red flag because: Hey, we just broke our API! Furthermore we want to unleash the power of ScalaCheck1 to benefit from property based testing. But even when we’re not using that we can still use it to generate test data for us.
Besides the regular unit tests there should be integration tests if a service is written. We can test a lot of things on the unit test side but in the end the integration of all our moving parts is what matters and often (usually on the not so pure side of things) you would have to trick (read mock) a lot to test things in isolation.
Testing the impure service
We will start with writing some data generators using the ScalaCheck library.
Generators
ScalaCheck already provides several generators for primitives but for our data models we have to do some more plumbing. Let’s start with generating a language code.
1 val genLanguageCode: Gen[LanguageCode] = Gen.oneOf(LanguageCodes.all)
Using the Gen.oneOf helper from the library the code becomes dead simple. Generating a UUID is nothing special either.
1 val genUuid: Gen[UUID] = Gen.delay(UUID.randomUUID)
2 )
You might be tempted to use Gen.const here but please don’t because that one will be memorized and thus never change. Another option is using a list of randomly generated UUID values from which we then chose one. That would be sufficient for generators which only generate a single product but if we want to generate lists of them we would have duplicate ids sooner than later.
1 val DefaultProductName: ProductName = "I am a product name!"
2 val genProductName: Gen[ProductName] = for {
3 cs <- Gen.nonEmptyListOf(Gen.alphaNumChar)
4 name = RefType.applyRef[ProductName](cs.mkString)
5 .getOrElse(DefaultProductName)
6 } yield name
So what do we have here? We want to generate a non empty string (because that is a requirement for our ProductName) but we also want to return a properly typed entity. First we let ScalaCheck generate a non empty list of random characters which we give to a utility function of refined. However we need a fallback value in case the validation done by refined fails. Therefore we defined a general default product name beforehand.
1 val genProductName: Gen[ProductName] =
2 Gen.nonEmptyListOf(Gen.alphaNumChar).map(cs =>
3 Refined.unsafeApply(cs.mkString)
4 )
Now that we have generators for language codes and product names we can write a generator for our Translation type.
1 val genTranslation: Gen[Translation] = for {
2 c <- genLanguageCode
3 n <- genProductName
4 } yield
5 Translation(
6 lang = c,
7 name = n
8 )
9
10 implicit val arbitraryTranslation: Arbitrary[Translation] =
11 Arbitrary(genTranslation)
As we can see the code is also quite simple. Additionally we create an implicit arbitrary value which will be used automatically by the forAll test helper if it is in scope. To be able to generate a Product we will need to provide a non empty list of translations.
1 val genTranslationList: Gen[List[Translation]] = for {
2 ts <- Gen.nonEmptyListOf(genTranslation)
3 } yield ts
4
5 val genNonEmptyTranslationList: Gen[NonEmptyList[Translation]] = for {
6 t <- genTranslation
7 ts <- genTranslationList
8 ns = NonEmptyList.fromList(ts)
9 } yield ns.getOrElse(NonEmptyList.of(t))
The first generator will create a non empty list of translations but will be typed as a simple List. Therefore we create a second generator which uses the fromList helper of the non empty list from Cats. Because that helper returns an Option (read is a safe function) we need to fallback to using a simple of function at the end.
With all these in place we can finally create our Product instances.
1 val genProduct: Gen[Product] = for {
2 id <- genProductId
3 ts <- genNonEmptyTranslationList
4 } yield
5 Product(
6 id = id,
7 names = ts
8 )
9
10 implicit val arbitraryProduct: Arbitrary[Product] =
11 Arbitrary(genProduct)
The code is basically the same as for Translation - the arbitrary implicit included.
Unit Tests
To avoid repeating the construction of our unit test classes we will implement a base class for tests which is quite simple.
1 abstract class BaseSpec extends WordSpec
2 with MustMatchers with ScalaCheckPropertyChecks {}
Feel free to use other test styles - after all ScalaTest offers a lot2 of them. I tend to lean towards the more verbose ones like WordSpec. Maybe that is because I spent a lot of time with RSpec3 in the Ruby world. ;-)
1 import com.wegtam.books.pfhais.impure.models.TypeGenerators._
2
3 forAll("input") { p: Product =>
4 val rows = p.names.map(t => (p.id, t.lang.value, t.name.value)).toList
5 Product.fromDatabase(rows) must contain(p)
6 }
The code above is a very simple test of our helper function fromDatabase which works in the following way:
- The
forAllwill generate a lot ofProductentities using the generator. - From each entity a list of “rows” is constructed like they would appear in the database.
- These constructed rows are given to the
fromDatabasefunction. - The returned
Optionmust then contain the generated value.
Because we construct the input for the function from a valid generated instance the function must always return a valid output.
Now let’s continue with testing our JSON codec for Product.
Testing a JSON codec
We need our JSON codec to provide several guarantees:
- It must fail to decode invalid JSON input format (read garbage).
- It must fail to decode valid JSON input format with invalid data (read wrong semantics).
- It must succeed to decode completely valid input.
- It must encode JSON which contains all fields included in the model.
- It must be able to decode JSON that itself encoded.
The first one is pretty simple and to be honest: You don’t have to write a test for this because that should be guaranteed by the Circe library. Things look a bit different for very simple JSON representations though (read when encoding to numbers or strings).
I’ve seen people arguing about point 5 and there may be applications for it but implementing encoders and decoders in a non-reversible way will make your life way more complicated.
1 forAll("input") { s: String =>
2 decode[Product](s).isLeft must be(true)
3 }
There is not much to say about the test above: It will generate a lot of random strings which will be passed to the decoder which must fail.
1 forAll("id", "names") { (id: String, ns: List[String]) =>
2 val json = """{
3 |"id":""" + id.asJson.noSpaces + """,
4 |"names":""" + ns.asJson.noSpaces + """
5 |}""".stripMargin
6 decode[Product](json).isLeft must be(true)
7 }
This test will generate random instances for id which are all wrong because it must be a UUID and not a string. Also the instances for names will mostly (but maybe not always) be wrong because there might be empty strings or an even empty list. So the decoder is given a valid JSON format but invalid values, therefore it must fail.
1 forAll("input") { i: Product =>
2 val json = s"""{
3 |"id": ${i.id.asJson.noSpaces},
4 |"names": ${i.names.asJson.noSpaces} 5 |}""".stripMargin
6 withClue(s"Unable to decode JSON: $json") {
7 decode[Product](json) match {
8 case Left(e) => fail(e.getMessage)
9 case Right(v) => v must be(i)
10 }
11 }
12 }
In this case we manually construct a valid JSON input using values from a generated valid Product entity. This is passed to the decoder and the decoder must not only succeed but return an instance equal to the generated one.
1 forAll("input") { i: Product =>
2 val json = i.asJson.noSpaces
3 json must include(s""""id":${i.id.asJson.noSpaces}""")
4 json must include(s""""names":${i.names.asJson.noSpaces}""")
5 }
The test will generate again a lot of entities and we construct a JSON string from each. We then expect the string to include several field names and their correctly encoded values. You might ask why we do not check for more things like: Are these fields the only ones within the JSON string? Well, this would be more cumbersome to test and a JSON containing more fields than we specify won’t matter for the decoder because it will just ignore them.
1 forAll("input") { p: Product =>
2 decode[Product](p.asJson.noSpaces) match {
3 case Left(_) => fail("Must be able to decode encoded JSON!")
4 case Right(d) => withClue("Must decode the same product!")(d must be(p))
5 }
6 }
Here we encode a generated entity and pass it to the encoder which must return the same entity.
More tests
So this is basically what we do for models. Because we have more than one we will have to write tests for each of the others. I will spare you the JSON tests for Translation but that one also has a helper function called fromUnsafe so let’s take a look at the function.
1 def fromUnsafe(lang: String)(name: String): Option[Translation] =
2 for {
3 l <- RefType.applyRef[LanguageCode](lang).toOption
4 n <- RefType.applyRef[ProductName](name).toOption
5 } yield Translation(lang = l, name = n)
This function simply tries to create a valid Translation entity from unsafe input values using the helpers provided by refined. As we can see it is a total function (read is safe to use). To cover all corner cases we must test it with safe and unsafe input.
1 forAll("lang", "name") { (l: String, n: String) =>
2 whenever(
3 RefType
4 .applyRef[LanguageCode](l)
5 .toOption
6 .isEmpty || RefType.applyRef[ProductName](n).toOption.isEmpty
7 ) {
8 Translation.fromUnsafe(l)(n) must be(empty)
9 }
10 }
Here we generate two random strings which we explicitly check to be invalid using the whenever helper. Finally the function must return an empty Option e.g. None for such values.
1 forAll("input") { t: Translation =>
2 Translation.fromUnsafe(t.lang.value)(t.name.value) must contain(t)
3 }
The test for valid input is very simple because we simply use the values from our automatically generated valid instances. :-)
So far we have no tests for our Repository class which handles all the database work. Neither have we tests for our routes. We have several options for testing here but before can test either of them we have do to some refactoring. For starters we should move our routes out of our main application into separate classes to be able to test them more easily.
Yes, we should. There are of course limits and pros and cons to that but in general this makes sense. Also this has nothing to do with being “impure” or “pure” but with clean structure.
Some refactoring
Moving the routes into separate classes poses no big problem we simply create a ProductRoutes and a ProductsRoutes class which will hold the appropriate routes. As a result our somewhat messy main application code becomes more readable.
1 def main(args: Array[String]): Unit = {
2 implicit val system: ActorSystem = ActorSystem()
3 implicit val mat: ActorMaterializer = ActorMaterializer()
4 implicit val ec: ExecutionContext = system.dispatcher
5
6 val url = ???
7 val user = ???
8 val pass = ???
9 val flyway: Flyway = ???
10 val _ = flyway.migrate()
11
12 val dbConfig: DatabaseConfig[JdbcProfile] =
13 DatabaseConfig.forConfig("database", system.settings.config)
14 val repo = new Repository(dbConfig)
15
16 val productRoutes = new ProductRoutes(repo)
17 val productsRoutes = new ProductsRoutes(repo)
18 val routes = productRoutes.routes ~ productsRoutes.routes
19
20 val host = system.settings.config.getString("api.host")
21 val port = system.settings.config.getInt("api.port")
22 val srv = Http().bindAndHandle(routes, host, port)
23 val pressEnter = StdIn.readLine()
24 srv.flatMap(_.unbind()).onComplete(_ => system.terminate())
25 }
We simply create our instances from our routing classes and construct our global routes directly from them. This is good but if we want to test the routes in isolation we still have the problem that they are hard-wired to our Repository class which is implemented via Slick. Several options exist to handle this:
- Use an in-memory test database with according configuration.
- Abstract further and use a trait instead of the concrete repository implementation.
- Write integration tests which will require a working database.
Using option 1 is tempting but think about it some more. While the benefit is that we can use our actual implementation and just have to fire up an in-memory database (for example h2), there are also some drawbacks:
- You have to handle evolutions for the in-memory database.
- Your evolutions have to be completely portable SQL (read ANSI SQL). Otherwise you’ll have to write each of your evolutions scripts two times (one for production, one for testing).
- Your code has to be database agnostic. This sounds easier than it is. Even the tools you’re using may use database specific features under the hood.
- Several features are simply not implemented in some databases. Think of things like cascading deletion via foreign keys.
Taking option 2 is a valid choice but it will result in more code. Also you must pay close attention to the “test repository” implementation to avoid introducing bugs there. Going for the most simple approach is usually feasible. Think of a simple test repository implementation that will just return hard coded values or values passed to it via constructor.
However we will go with option 3 in this case. It has the drawback that you’ll have to provide a real database environment (and maybe more) for testing. But it is as close to production as you can get. Also you will need these either way to test your actual repository implementation, so let’s get going.
Integration Tests
First we need to configure our test database because we do not want to accidentally wipe a production database. For our case we leave everything as is and just change the database name.
1 api {
2 host = "localhost"
3 port = 49152
4 }
5
6 database {
7 profile = "slick.jdbc.PostgresProfile$"
8 db {
9 connectionPool = "HikariCP"
10 dataSourceClass = "org.postgresql.ds.PGSimpleDataSource"
11 properties {
12 serverName = "localhost"
13 portNumber = "5432"
14 databaseName = "impure_test"
15 user = "impure"
16 password = "secret"
17 }
18 numThreads = 10
19 }
20 }
Another thing we should do is provide a test configuration for our logging framework. We use the logback library and Slick will produce a lot of logging output on the DEBUG level so we should fix that. It is nice to have logging if you need it but it also clutters up your log files. We create a file logback-test.xml in the directory src/it/resources which should look like this:
1 <?xml version="1.0" encoding="UTF-8"?>
2 <configuration debug="false">
3 <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
4 <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
5 <level>WARN</level>
6 </filter>
7 <encoder>
8 <pattern>%date %highlight(%-5level) %cyan(%logger{0}) - %msg%n</pattern>
9 </encoder>
10 </appender>
11
12 <appender name="async-console" class="ch.qos.logback.classic.AsyncAppender">
13 <appender-ref ref="console"/>
14 <queueSize>5000</queueSize>
15 <discardingThreshold>0</discardingThreshold>
16 </appender>
17
18 <logger name="com.wegtam.books.pfhais.impure" level="INFO" additivity="false">
19 <appender-ref ref="console"/>
20 </logger>
21
22 <root>
23 <appender-ref ref="console"/>
24 </root>
25 </configuration>
Due to the nature of integration tests we want to use production or “production like” settings and environment. But really starting our application or service for each test will be quite cumbersome so we should provide a base class for our tests. In this class we will start our service, migrate our database and provide an opportunity to shut it down properly after testing.
Because we do not want to get into trouble when running an Akka-HTTP on the same port, we first create a helper function which will determine a free port number.
1 import java.net.ServerSocket
2
3 def findAvailablePort(): Int = {
4 val serverSocket = new ServerSocket(0)
5 val freePort = serverSocket.getLocalPort
6 serverSocket.setReuseAddress(true)
7 serverSocket.close()
8 freePort
9 }
The code is quite simple and very useful for such cases. Please note that it is important to use setReuseAddress because otherwise the found socket will be blocked for a certain amount of time. But now let us continue with our base test class.
1 abstract class BaseSpec
2 extends TestKit(
3 ActorSystem(
4 "it-test",
5 ConfigFactory
6 .parseString(s"api.port=${BaseSpec.findAvailablePort()}")
7 .withFallback(ConfigFactory.load())
8 )
9 )
10 with AsyncWordSpecLike
11 with MustMatchers
12 with ScalaCheckPropertyChecks
13 with BeforeAndAfterAll
14 with BeforeAndAfterEach {
15
16 implicit val materializer: ActorMaterializer = ActorMaterializer()
17
18 private val url = ???
19 private val user = ???
20 private val pass = ???
21 protected val flyway: Flyway =
22 Flyway.configure().dataSource(url, user, pass).load()
23
24 override protected def afterAll(): Unit =
25 TestKit.shutdownActorSystem(system, FiniteDuration(5, SECONDS))
26
27 override protected def beforeAll(): Unit = {
28 val _ = flyway.migrate()
29 }
30 }
As you can see we are using the Akka-Testkit to initialise an actor system. This is useful because there are several helpers available which you might need. We configure the actor system with our free port using the loaded configuration as fallback. Next we globally create an actor materializer which is needed by Akka-HTTP and Akka-Streams. Also we create a globally available Flyway instance to make cleaning and migrating the database easier.
The base class also implements the beforeAll and afterAll methods which will be run before and after all tests. They are used to initially migrate the database and to shut down the actor system properly in the end.
Testing the repository
Now that we our parts in place we can write an integration test for our repository implementation.
First we need to do some things globally for the test scope.
1 private val dbConfig: DatabaseConfig[JdbcProfile] =
2 DatabaseConfig.forConfig("database", system.settings.config)
3 private val repo = new Repository(dbConfig)
4
5 override protected def beforeEach(): Unit = {
6 flyway.clean()
7 val _ = flyway.migrate()
8 super.beforeEach()
9 }
10
11 override protected def afterEach(): Unit = {
12 flyway.clean()
13 super.afterEach()
14 }
15
16 override protected def afterAll(): Unit = {
17 repo.close()
18 super.afterAll()
19 }
We create one Repository instance for all our tests here. The downside is that if one test crashes it then the other will be affected too. On the other hand we avoid running into database connection limits and severe code limbo to ensure closing a repository connection after each test no matter the result.
Also we clean and migrate before each test and clean also after each test. This ensures having a clean environment.
Onwards to the test for loading a single product.
1 "#loadProduct" when {
2 "the ID does not exist" must {
3 "return an empty list of rows" in {
4 val id = UUID.randomUUID
5 for {
6 rows <- repo.loadProduct(id)
7 } yield {
8 rows must be(empty)
9 }
10 }
11 }
12
13 "the ID exists" must {
14 "return a list with all product rows" in {
15 genProduct.sample match {
16 case None => fail("Could not generate data sample!")
17 case Some(p) =>
18 for {
19 _ <- repo.saveProduct(p)
20 rows <- repo.loadProduct(p.id)
21 } yield {
22 Product.fromDatabase(rows) match {
23 case None => fail("No product created from database rows!")
24 case Some(c) =>
25 c.id must be(p.id)
26 c mustEqual p
27 }
28 }
29 }
30 }
31 }
32 }
Loading a non existing product must not produce any result and is simple to test if our database is empty. Testing the loading of a real product is not that much more complicated. We use the ScalaCheck generators to create one, save it and load it again. The loaded product must of course be equal to the saved one.
1 "#loadProducts" when {
2 "no products exist" must {
3 "return an empty stream" in {
4 val src = Source.fromPublisher(repo.loadProducts())
5 for {
6 ps <- src.runWith(Sink.seq)
7 } yield {
8 ps must be(empty)
9 }
10 }
11 }
12
13 "some products exist" must {
14 "return a stream with all product rows" in {
15 genProducts.sample match {
16 case None => fail("Could not generate data sample!")
17 case Some(ps) =>
18 val expected = ps.flatMap(
19 p => p.names.toNonEmptyList.toList.map(
20 n => (p.id, n.lang, n.name)
21 )
22 )
23 for {
24 _ <- Future.sequence(ps.map(p => repo.saveProduct(p)))
25 src = Source
26 .fromPublisher(repo.loadProducts())
27 // more code omitted here
28 rows <- src.runWith(Sink.seq)
29 } yield {
30 rows must not be (empty)
31 rows.size mustEqual ps.size
32 rows.toList.sorted mustEqual ps.sorted
33 }
34 }
35 }
36 }
37 }
Testing the loading of all products if none exit is trivial like the one for a non existing single product. For the case of multiple products we generate a list of them which we save. Afterwards we load them and use the same transformation logic like in the routes to be able to construct proper Product instances. One thing you might notice is the explicit sorting which is due to the fact that we want to ensure that our product lists are both sorted before comparing them.
1 "#saveProduct" when {
2 "the product does not already exist" must {
3 "save the product to the database" in {
4 genProduct.sample match {
5 case None => fail("Could not generate data sample!")
6 case Some(p) =>
7 for {
8 cnts <- repo.saveProduct(p)
9 rows <- repo.loadProduct(p.id)
10 } yield {
11 withClue("Data missing from database!")(
12 cnts.fold(0)(_ + _) must be(p.names.toNonEmptyList.size + 1))
13 Product.fromDatabase(rows) match {
14 case None => fail("No product created from database rows!")
15 case Some(c) =>
16 c.id must be(p.id)
17 c mustEqual p
18 }
19 }
20 }
21 }
22 }
23
24 "the product does already exist" must {
25 "return an error and not change the database" in {
26 (genProduct.sample, genProduct.sample) match {
27 case (Some(a), Some(b)) =>
28 val p = b.copy(id = a.id)
29 for {
30 cnts <- repo.saveProduct(a)
31 nosv <- repo.saveProduct(p).recover {
32 case _ => 0
33 }
34 rows <- repo.loadProduct(a.id)
35 } yield {
36 withClue("Saving a duplicate product must fail!")(nosv must be(0))
37 Product.fromDatabase(rows) match {
38 case None => fail("No product created from database rows!")
39 case Some(c) =>
40 c.id must be(a.id)
41 c mustEqual a
42 }
43 }
44 case _ => fail("Could not create data sample!")
45 }
46 }
47 }
48 }
Here we test the saving which in the first case should simply write the appropriate data into the database. If the product already exists however this should not happen. Our database constraints will ensure that this does not happen (or so we hope ;-)). Slick will throw an exception which we catch in the test code using the recover method from Future to return a zero indicating no affected database rows. In the end we test for this zero and also check if the originally saved product has not been changed.
1 "#updateProduct" when {
2 "the product does exist" must {
3 "update the database" in {
4 (genProduct.sample, genProduct.sample) match {
5 case (Some(a), Some(b)) =>
6 val p = b.copy(id = a.id)
7 for {
8 cnts <- repo.saveProduct(a)
9 upds <- repo.updateProduct(p)
10 rows <- repo.loadProduct(a.id)
11 } yield {
12 withClue("Already existing product was not created!")(
13 cnts.fold(0)(_ + _) must be(a.names.toNonEmptyList.size + 1)
14 )
15 Product.fromDatabase(rows) match {
16 case None => fail("No product created from database rows!")
17 case Some(c) =>
18 c.id must be(a.id)
19 c mustEqual p
20 }
21 }
22 case _ => fail("Could not create data sample!")
23 }
24 }
25 }
26
27 "the product does not exist" must {
28 "return an error and not change the database" in {
29 genProduct.sample match {
30 case None => fail("Could not generate data sample!")
31 case Some(p) =>
32 for {
33 nosv <- repo.updateProduct(p).recover {
34 case _ => 0
35 }
36 rows <- repo.loadProduct(p.id)
37 } yield {
38 withClue("Updating a not existing product must fail!")
39 (nosv must be(0))
40 withClue("Product must not exist in database!")
41 (rows must be(empty))
42 }
43 }
44 }
45 }
46 }
For testing an update we generate two samples, save one to the database, change the id of the other to the one from the first and execute an update. This update should proceed without problems and the data in the database must have been changed correctly.
If the product does not exist then we use the same recover technique like in the saveProduct test.
Congratulations, we have made a check mark on our first integration test using a real database using randomly generated data!
Testing the routes
Regarding the route testing there several options as always. For this one we will define some use cases and then develop some more helper code which will allow us to fire up our routes and do real HTTP requests and then check the database and the responses. We build upon our BaseSpec class and call it BaseUseCaseSpec. In it we will do some more things like define a global base URL which can be used from the tests to make correct requests. Additionally we will write a small actor which simply starts an Akka-HTTP server.
1 final class BaseUseCaseActor(repo: Repository, mat: ActorMaterializer)
2 extends Actor with ActorLogging {
3 import context.dispatcher
4
5 implicit val system: ActorSystem = context.system
6 implicit val materializer: ActorMaterializer = mat
7
8 override def receive: Receive = {
9 case BaseUseCaseActorCmds.Start =>
10 val productRoutes = new ProductRoutes(repo)
11 val productsRoutes = new ProductsRoutes(repo)
12 val routes = productRoutes.routes ~ productsRoutes.routes
13 val host = context.system.settings.config.getString("api.host")
14 val port = context.system.settings.config.getInt("api.port")
15 val _ = Http().bindAndHandle(routes, host, port)
16 case BaseUseCaseActorCmds.Stop =>
17 context.stop(self)
18 }
19 }
20
21 object BaseUseCaseActor {
22 def props(repo: Repository, mat: ActorMaterializer): Props =
23 Props(new BaseUseCaseActor(repo, mat))
24
25 sealed trait BaseUseCaseActorCmds
26
27 object BaseUseCaseActorCmds {
28 case object Start extends BaseUseCaseActorCmds
29 case object Stop extends BaseUseCaseActorCmds
30 }
31 }
As you can see, the actor is quite simple. Upon receiving the Start command it will initialise the routes and start an Akka-HTTP server. To be able to do this it needs the database repository and an actor materializer which are passed via the constructor. Regarding the BaseUseCaseSpec we will concentrate on the code that differs from the base class.
1 abstract class BaseUseCaseSpec
2 extends TestKit(
3 ActorSystem(
4 "it-test",
5 ConfigFactory
6 .parseString(s"api.port=${BaseUseCaseSpec.findAvailablePort()}")
7 .withFallback(ConfigFactory.load())
8 )
9 )
10 with AsyncWordSpecLike
11 with MustMatchers
12 with ScalaCheckPropertyChecks
13 with BeforeAndAfterAll
14 with BeforeAndAfterEach {
15 // ...
16 final val baseUrl: String = s"""http://${system.settings.config
17 .getString("api.host")}:${system.settings.config
18 .getInt("api.port")}"""
19
20 protected val dbConfig: DatabaseConfig[JdbcProfile] =
21 DatabaseConfig.forConfig("database", system.settings.config)
22 protected val repo = new Repository(dbConfig)
23
24 override protected def beforeAll(): Unit = {
25 val _ = flyway.migrate()
26 val a = system.actorOf(BaseUseCaseActor.props(repo, materializer))
27 a ! BaseUseCaseActorCmds.Start
28 }
29 // ...
30 }
Here we create our base URL and our database repository while we use the beforeAll function to initialise our actor before any test is run. Please note that this has the same drawback like sharing the repository across tests: If a test crashes your service then the others will be affected too.
But let’s write a test for our first use case: loading a product!
Again we will use the beforeEach and afterEach helpers to clean up our database. Now let’s take a look at a test for loading a product that does not exist.
1 "Loading a Product by ID" when {
2 "the ID does not exist" must {
3 val expectedStatus = StatusCodes.NotFound
4
5 s"return $expectedStatus" in {
6 val id = UUID.randomUUID
7
8 for {
9 resp <- http.singleRequest(
10 HttpRequest(
11 method = HttpMethods.GET,
12 uri = s"$baseUrl/product/$id",
13 headers = Seq(),
14 entity = HttpEntity(
15 contentType = ContentTypes.`application/json`,
16 data = ByteString("")
17 )
18 )
19 )
20 } yield {
21 resp.status must be(expectedStatus)
22 }
23 }
24 }
25 }
Maybe a bit verbose but we do a real HTTP request here and check the response status code. So how does it look if we run it?
1 [info] LoadProduct:
2 [info] Loading a Product by ID
3 [info] when the ID does not exist
4 [info] - must return 404 Not Found *** FAILED ***
5 [info] 200 OK was not equal to 404 Not Found (LoadProduct.scala:88)
That is not cool. What happened? Let’s take a look at the code.
1 get {
2 complete {
3 for {
4 rows <- repo.loadProduct(id)
5 prod <- Future { Product.fromDatabase(rows) }
6 } yield prod
7 }
8 }
Well, no wonder - we are simply returning the output of our fromDatabase helper function which may be empty. This will result in an HTTP status code 200 with an empty body. If you don’t believe me just fire up the service and do a request by hand via curl or httpie.
Luckily for us Akka-HTTP has us covered with the rejectEmptyResponse directive which we can use.
1 get {
2 rejectEmptyResponse {
3 complete {
4 for {
5 rows <- repo.loadProduct(id)
6 prod <- Future { Product.fromDatabase(rows) }
7 } yield prod
8 }
9 }
10 }
Cool, it seems we’re set with this one. So onward to testing to load an existing product via the API.
1 "Loading a Product by ID" when {
2 "the ID does exist" must {
3 val expectedStatus = StatusCodes.OK
4
5 s"return $expectedStatus and the Product" in {
6 genProduct.sample match {
7 case None => fail("Could not generate data sample!")
8 case Some(p) =>
9 for {
10 _ <- repo.saveProduct(p)
11 rows <- repo.loadProduct(p.id)
12 resp <- http.singleRequest(
13 HttpRequest(
14 method = HttpMethods.GET,
15 uri = s"$baseUrl/product/${p.id}",
16 headers = Seq(),
17 entity = HttpEntity(
18 contentType = ContentTypes.`application/json`,
19 data = ByteString("")
20 )
21 )
22 )
23 body <- resp.entity.dataBytes.runFold(ByteString(""))(_ ++ _)
24 } yield {
25 withClue("Seeding product data failed!")(rows must not be(empty))
26 resp.status must be(expectedStatus)
27 decode[Product](body.utf8String) match {
28 case Left(e) => fail(s"Could not decode response: $e")
29 case Right(d) => d mustEqual p
30 }
31 }
32 }
33 }
34 }
35 }
Here we simple save our generated product into the database before executing our request. We also check if the product has actually been written to be on the safe side. Additionally we also check if the decoded response body matches the product we expect. In contrast to our first test this one works instantly so not all hope is lost for our coding skills. ;-)
We’ll continue with the use case of saving (or creating) a product via the API. This time we will make the code snippets shorter.
1 val expectedStatus = StatusCodes.BadRequest
2
3 s"return $expectedStatus" in {
4 for {
5 resp <- http.singleRequest(
6 HttpRequest(
7 method = HttpMethods.POST,
8 uri = s"$baseUrl/products",
9 headers = Seq(),
10 entity = HttpEntity(
11 contentType = ContentTypes.`application/json`,
12 data = ByteString(
13 scala.util.Random.alphanumeric.take(256).mkString
14 )
15 )
16 )
17 )
18 } yield {
19 resp.status must be(expectedStatus)
20 }
21 }
Here we test posting garbage instead of valid JSON to the endpoint which must result in a “400 Bad Request” returned to us.
1 val expectedStatus = StatusCodes.InternalServerError
2
3 s"return $expectedStatus and not save the Product" in {
4 (genProduct.sample, genProduct.sample) match {
5 case (Some(a), Some(b)) =>
6 val p = b.copy(id = a.id)
7 for {
8 _ <- repo.saveProduct(a)
9 rows <- repo.loadProduct(a.id)
10 resp <- http.singleRequest(
11 HttpRequest(
12 method = HttpMethods.POST,
13 uri = s"$baseUrl/products",
14 headers = Seq(),
15 entity = HttpEntity(
16 contentType = ContentTypes.`application/json`,
17 data = ByteString(p.asJson.noSpaces)
18 )
19 )
20 )
21 rows2 <- repo.loadProduct(a.id)
22 } yield {
23 withClue("Seeding product data failed!")(rows must not be(empty))
24 resp.status must be(expectedStatus)
25 Product.fromDatabase(rows2) match {
26 case None =>
27 fail("Seeding product was not saved to database!")
28 case Some(s) =>
29 withClue("Existing product must not be changed!")(s mustEqual a)
30 }
31 }
32 case _ => fail("Could not generate data sample!")
33 }
34 }
This one executes it bit more but basically we try to save (or better create) an already existing product. Therefore the constraints of our database should product an error which in turn must return a “500 Internal Server Error” to use. Additionally we verify that the existing product in the database was not changed.
1 val expectedStatus = StatusCodes.OK
2
3 s"return $expectedStatus and save the Product" in {
4 genProduct.sample match {
5 case None => fail("Could not generate data sample!")
6 case Some(p) =>
7 for {
8 resp <- http.singleRequest(
9 HttpRequest(
10 method = HttpMethods.POST,
11 uri = s"$baseUrl/products",
12 headers = Seq(),
13 entity = HttpEntity(
14 contentType = ContentTypes.`application/json`,
15 data = ByteString(p.asJson.noSpaces)
16 )
17 )
18 )
19 rows <- repo.loadProduct(p.id)
20 } yield {
21 resp.status must be(expectedStatus)
22 Product.fromDatabase(rows) match {
23 case None => fail("Product was not saved to database!")
24 case Some(s) => s mustEqual p
25 }
26 }
27 }
28 }
Last but not least we are testing to save a not already existing valid product into the database. Again we check for the expected status code of “200 OK” and verify that the product saved into the database is the one we sent to the API. Let’s move on to testing the loading of all products now.
1 val expectedStatus = StatusCodes.OK
2
3 s"return $expectedStatus and an empty list" in {
4 for {
5 resp <- http.singleRequest(
6 HttpRequest(
7 method = HttpMethods.GET,
8 uri = s"$baseUrl/products",
9 headers = Seq(),
10 entity = HttpEntity(
11 contentType = ContentTypes.`application/json`,
12 data = ByteString("")
13 )
14 )
15 )
16 body <- resp.entity.dataBytes.runFold(ByteString(""))(_ ++ _)
17 } yield {
18 resp.status must be(expectedStatus)
19 decode[List[Product]](body.utf8String) match {
20 case Left(e) => fail(s"Could not decode response: $e")
21 case Right(d) => d must be(empty)
22 }
23 }
24 }
Our first test case is loading all products if no product exists so we expect an empty list here and the appropriate status code.
1 val expectedStatus = StatusCodes.OK
2
3 s"return $expectedStatus and a list with all products" in {
4 genProducts.sample match {
5 case None => fail("Could not generate data sample!")
6 case Some(ps) =>
7 for {
8 _ <- Future.sequence(ps.map(p => repo.saveProduct(p)))
9 resp <- http.singleRequest(
10 HttpRequest(
11 method = HttpMethods.GET,
12 uri = s"$baseUrl/products",
13 headers = Seq(),
14 entity = HttpEntity(
15 contentType = ContentTypes.`application/json`,
16 data = ByteString("")
17 )
18 )
19 )
20 body <- resp.entity.dataBytes.runFold(ByteString(""))(_ ++ _)
21 } yield {
22 resp.status must be(expectedStatus)
23 decode[List[Product]](body.utf8String) match {
24 case Left(e) => fail(s"Could not decode response: $e")
25 case Right(d) => d.sorted mustEqual ps.sorted
26 }
27 }
28 }
29 }
This one is also straight forward: We save our generated list of products to the database and query the API which must return a “200 OK” status code and a correct list of products in JSON format. Looks like we have one more use case to tackle: Updating a product via the API.
1 val expectedStatus = StatusCodes.BadRequest
2
3 s"return $expectedStatus" in {
4 genProduct.sample match {
5 case None => fail("Could not generate data sample!")
6 case Some(p) =>
7 for {
8 _ <- repo.saveProduct(p)
9 rows <- repo.loadProduct(p.id)
10 resp <- http.singleRequest(
11 HttpRequest(
12 method = HttpMethods.PUT,
13 uri = s"$baseUrl/product/${p.id}",
14 headers = Seq(),
15 entity = HttpEntity(
16 contentType = ContentTypes.`application/json`,
17 data = ByteString(scala.util.Random.alphanumeric.take(256).mkString)
18 )
19 )
20 )
21 rows2 <- repo.loadProduct(p.id)
22 } yield {
23 withClue("Seeding product data failed!")(rows must not be(empty))
24 resp.status must be(expectedStatus)
25 Product.fromDatabase(rows2) match {
26 case None =>
27 fail("Seeding product was not saved to database!")
28 case Some(s) =>
29 withClue("Existing product must not be changed!")(s mustEqual p)
30 }
31 }
32 }
First we test with garbage JSON in the request. Before doing the request we actually create a product to avoid getting an error caused by a possibly missing product. Afterwards we check for the expected “400 Bad Request” status code and verify that our product has not been updated.
1 val expectedStatus = StatusCodes.OK
2
3 s"return $expectedStatus and update the Product" in {
4 (genProduct.sample, genProduct.sample) match {
5 case (Some(a), Some(b)) =>
6 val p = b.copy(id = a.id)
7 for {
8 _ <- repo.saveProduct(a)
9 rows <- repo.loadProduct(a.id)
10 resp <- http.singleRequest(
11 HttpRequest(
12 method = HttpMethods.PUT,
13 uri = s"$baseUrl/product/${p.id}",
14 headers = Seq(),
15 entity = HttpEntity(
16 contentType = ContentTypes.`application/json`,
17 data = ByteString(p.asJson.noSpaces)
18 )
19 )
20 )
21 rows2 <- repo.loadProduct(p.id)
22 } yield {
23 withClue("Seeding product data failed!")(rows must not be(empty))
24 resp.status must be(expectedStatus)
25 Product.fromDatabase(rows2) match {
26 case None => fail("Seeding product was not saved to database!")
27 case Some(s) => s mustEqual p
28 }
29 }
30 case _ => fail("Could not generate data sample!")
31 }
32 }
Next we test updating an existing product using valid JSON. We check the status code and if the product has been correctly updated within the database.
1 val expectedStatus = StatusCodes.InternalServerError
2
3 s"return $expectedStatus" in {
4 genProduct.sample match {
5 case None => fail("Could not generate data sample!")
6 case Some(p) =>
7 for {
8 resp <- http.singleRequest(
9 HttpRequest(
10 method = HttpMethods.PUT,
11 uri = s"$baseUrl/product/${p.id}",
12 headers = Seq(),
13 entity = HttpEntity(
14 contentType = ContentTypes.`application/json`,
15 data = ByteString(p.asJson.noSpaces)
16 )
17 )
18 )
19 rows <- repo.loadProduct(p.id)
20 } yield {
21 resp.status must be(expectedStatus)
22 rows must be(empty)
23 }
24 }
25 }
Finally we test updating a non existing product which should produce the expected status code and not save into the database. Wow, it seems we done for good with our impure implementation. Well except for some benchmarking but let’s save that for later.
If we look onto our test code coverage (which is a metric that you should use) then things look pretty good. We are missing some parts but in general we should have things covered.
Testing the pure service
We will skip the explanation of the ScalaCheck generators because they only differ slightly from the ones used in the impure part.
Unit Tests
The model tests are omitted here because they are basically the same as in the impure section. If you are interested in them just look at the source code. In contrast to the impure part we will now write unit tests for our routes. Meaning we will be able to test our routing logic without spinning up a database.
Testing our routes
To be able to test our routes we will have to implement a TestRepository first which we will use instead of the concrete implementation which is wired to a database.
1 class TestRepository[F[_]: Effect](data: Seq[Product]) extends Repository[F] {
2 override def loadProduct(id: ProductId) = {
3 data.find(_.id === id) match {
4 case None => Seq.empty.pure[F]
5 case Some(p) =>
6 val ns = p.names.toNonEmptyList.toList.to[Seq]
7 ns.map(n => (p.id, n.lang, n.name)).pure[F]
8 }
9 }
10
11 override def loadProducts() = {
12 Stream.empty
13 }
14
15 override def saveProduct(p: Product): F[Int] =
16 data.find(_.id === p.id).fold(0.pure[F])(_ => 1.pure[F])
17
18 override def updateProduct(p: Product): F[Int] =
19 data.find(_.id === p.id).fold(0.pure[F])(_ => 1.pure[F])
20
21 }
As you can see we try to stay abstract (using our HKT F[_] here) and we have basically left out the implementation of loadProducts because it will just return an empty stream. We will get back to it later on. Aside from that the class can be initialised with a (potentially empty) list of Product entities which will be used as a “database”. The save and update functions won’t change any data, they will just return a 0 or a 1 depending on the product being present in the seed data list.
1 val emptyRepository: Repository[IO] = new TestRepository[IO](Seq.empty)
2 val expectedStatusCode = Status.NotFound
3
4 s"return $expectedStatusCode" in {
5 forAll("id") { id: ProductId =>
6 Uri.fromString("/product/" + id.toString) match {
7 case Left(_) => fail("Could not generate valid URI!")
8 case Right(u) =>
9 def service: HttpRoutes[IO] =
10 Router("/" -> new ProductRoutes(emptyRepository).routes)
11 val response: IO[Response[IO]] = service.orNotFound.run(
12 Request(method = Method.GET, uri = u)
13 )
14 val result = response.unsafeRunSync
15 result.status must be(expectedStatusCode)
16 result.body.compile.toVector.unsafeRunSync must be(empty)
17 }
18 }
19 }
Above you can see the test for querying a non existing product which must return an empty response using a “404 Not Found” status code. First we try to create a valid URI from our generated ProductId. If that succeeds we create a small service wrapper for our routes in which we inject our empty TestRepository. Finally we create a response using this service and a request that we construct. Because we are in the land of IO we have to actually execute it (via unsafeRunSync) to get any results back. Finally we validate the status code and the response body.
1 [info] when GET /product/ID
2 [info] when product does not exist
3 [info] - must return 404 Not Found *** FAILED ***
4 [info] TestFailedException was thrown during property evaluation.
5 [info] Message: 200 OK was not equal to 404 Not Found
6 [info] Location: (ProductRoutesTest.scala:46)
7 [info] Occurred when passed generated values (
8 [info] id = 3f298ae4-4f7b-415b-9888-c2e9b34a4883
9 [info] )
It seems that we just pass an empty response if we do not find the product. This is not nice, so let’s fix this. The culprit is the following line in our ProductRoutes file:
1 for {
2 // ...
3 resp <- Ok(Product.fromDatabase(rows))
4 } yield resp
Okay, this looks easy. Let’s try this one:
1 for {
2 // ...
3 resp <- Product.fromDatabase(rows).fold(NotFound())(p => Ok(p))
4 } yield resp
Oh no, the compiler complains:
1 Cannot convert from Product to an Entity, because no
2 EntityEncoder[F, com.wegtam.books.pfhais.pure.models.Product]
3 instance could be found.
Right, before we had an Option[Product] here for which we created an implicit JSON encoder. So if we create one for the Product itself then we should be fine.
1 implicit def encodeProduct[A[_]: Applicative]: EntityEncoder[A, Product] =
2 jsonEncoderOf
Now back to our test:
1 [info] when GET /product/ID
2 [info] when product does not exist
3 [info] - must return 404 Not Found
Great! Seems like we are doing fine, so let’s continue. Next in line is testing a query for an existing product.
1 implicit def decodeProduct: EntityDecoder[IO, Product] = jsonOf
2 val expectedStatusCode = Status.Ok
3
4 s"return $expectedStatusCode and the product" in {
5 forAll("product") { p: Product =>
6 Uri.fromString("/product/" + p.id.toString) match {
7 case Left(_) => fail("Could not generate valid URI!")
8 case Right(u) =>
9 val repo: Repository[IO] = new TestRepository[IO](Seq(p))
10 def service: HttpRoutes[IO] =
11 Router("/" -> new ProductRoutes(repo).routes)
12 val response: IO[Response[IO]] = service.orNotFound.run(
13 Request(method = Method.GET, uri = u)
14 )
15 val result = response.unsafeRunSync
16 result.status must be(expectedStatusCode)
17 result.as[Product].unsafeRunSync must be(p)
18 }
19 }
20 }
This time we generate a whole product, again try to create a valid URI and continue as before. But this time we inject our TestRepository containing a list with our generated product. In the end we test our expected status code and the response body must contain our product. For the last part to work we must have an implicit EntityDecoder in scope.
1 val expectedStatusCode = Status.BadRequest
2
3 s"return $expectedStatusCode" in {
4 forAll("id") { id: ProductId =>
5 Uri.fromString("/product/" + id.toString) match {
6 case Left(_) => fail("Could not generate valid URI!")
7 case Right(u) =>
8 def service: HttpRoutes[IO] =
9 Router("/" -> new ProductRoutes(emptyRepository).routes)
10 val payload = scala.util.Random.alphanumeric.take(256).mkString
11 val response: IO[Response[IO]] = service.orNotFound.run(
12 Request(method = Method.PUT, uri = u)
13 .withEntity(payload.asJson.noSpaces)
14 )
15 val result = response.unsafeRunSync
16 result.status must be(expectedStatusCode)
17 result.body.compile.toVector.unsafeRunSync must be(empty)
18 }
19 }
20 }
Here we are trying to update a product which doesn’t have to exist because we send totally garbage JSON with the request which should result in a “400 Bad Request” status code. However if we run our test we get an exception instead:
1 [info] when PUT /product/ID
2 [info] when request body is invalid
3 [info] - must return 400 Bad Request *** FAILED ***
4 [info] InvalidMessageBodyFailure was thrown during property
5 [info] Occurred when passed generated values (
6 [info] id = 932682c9-1f9a-463a-84db-a0992d466aa3
7 [info] )
So let’s take a deep breath and look at our code:
1 case req @ PUT -> Root / "product" / UUIDVar(id) =>
2 for {
3 p <- req.as[Product]
4 _ <- repo.updateProduct(p)
5 r <- NoContent()
6 } yield r
As we can see we do no error handling at all. So maybe we can rewrite this a little bit.
1 case req @ PUT -> Root / "product" / UUIDVar(id) =>
2 req
3 .as[Product]
4 .flatMap { p =>
5 repo.updateProduct(p) *> NoContent()
6 }
7 .handleErrorWith {
8 case InvalidMessageBodyFailure(_, _) => BadRequest()
9 }
Now we explicitly handle any error which occurs when decoding the request entity. But in fact: I am lying to you. We only handle the invalid message body failure here. On the other hand it is enough to make our test happy. :-)
Onwards to our next test cases in which we use a valid JSON payload for our request.
1 val expectedStatusCode = Status.NotFound
2
3 s"return $expectedStatusCode" in {
4 forAll("product") { p: Product =>
5 Uri.fromString("/product/" + p.id.toString) match {
6 case Left(_) => fail("Could not generate valid URI!")
7 case Right(u) =>
8 def service: HttpRoutes[IO] =
9 Router("/" -> new ProductRoutes(emptyRepository).routes)
10 val response: IO[Response[IO]] = service.orNotFound.run(
11 Request(method = Method.PUT, uri = u)
12 .withEntity(p)
13 )
14 val result = response.unsafeRunSync
15 result.status must be(expectedStatusCode)
16 result.body.compile.toVector.unsafeRunSync must be(empty)
17 }
18 }
19 }
We expect a “404 Not Found” if we try to update a valid product which does not exist. But what do we get in the tests?
1 Message: 204 No Content was not equal to 404 Not Found
Well not exactly what we planned for but it is our own fault. We used the *> operator which ignores the value from the previous operation. So we need to fix that.
1 case req @ PUT -> Root / "product" / UUIDVar(id) =>
2 req
3 .as[Product]
4 .flatMap { p =>
5 for {
6 cnt <- repo.updateProduct(p)
7 res <- cnt match {
8 case 0 => NotFound()
9 case _ => NoContent()
10 }
11 } yield res
12 }
13 .handleErrorWith {
14 case InvalidMessageBodyFailure(_, _) => BadRequest()
15 }
We rely on the return value of our update function which contains the number of affected database rows. If it is zero then nothing has been done implying that the product was not found. Otherwise we return our “204 No Content” response as before. Still we miss one last test for our product routes.
1 val expectedStatusCode = Status.NoContent
2
3 s"return $expectedStatusCode" in {
4 forAll("product") { p: Product =>
5 Uri.fromString("/product/" + p.id.toString) match {
6 case Left(_) => fail("Could not generate valid URI!")
7 case Right(u) =>
8 val repo: Repository[IO] = new TestRepository[IO](Seq(p))
9 def service: HttpRoutes[IO] =
10 Router("/" -> new ProductRoutes(repo).routes)
11 val response: IO[Response[IO]] = service.orNotFound.run(
12 Request(method = Method.PUT, uri = u)
13 .withEntity(p)
14 )
15 val result = response.unsafeRunSync
16 result.status must be(expectedStatusCode)
17 result.body.compile.toVector.unsafeRunSync must be(empty)
18 }
19 }
20 }
Basically this is the same test as before with the exception that we now give our routes a properly seeded test repository. Great, we have tested our ProductRoutes and without having to spin up a database! But we still have work to do, so let’s move on to testing the ProductsRoutes implementation. Before we do that we adapt the code for creating a product using our gained knowledge from our update test.
1 case req @ POST -> Root / "products" =>
2 req
3 .as[Product]
4 .flatMap { p =>
5 for {
6 cnt <- repo.saveProduct(p)
7 res <- cnt match {
8 case 0 => NotFound()
9 case _ => InternalServerError()
10 }
11 } yield res
12 }
13 .handleErrorWith {
14 case InvalidMessageBodyFailure(_, _) => BadRequest()
15 }
Now to our tests, we will start with sending garbage JSON via the POST request.
1 val expectedStatusCode = Status.BadRequest
2
3 s"return $expectedStatusCode" in {
4 def service: HttpRoutes[IO] =
5 Router("/" -> new ProductsRoutes(emptyRepository).routes)
6 val payload = scala.util.Random.alphanumeric.take(256).mkString
7 val response: IO[Response[IO]] = service.orNotFound.run(
8 Request(method = Method.POST, uri = Uri.uri("/products"))
9 .withEntity(payload.asJson.noSpaces)
10 )
11 val result = response.unsafeRunSync
12 result.status must be(expectedStatusCode)
13 result.body.compile.toVector.unsafeRunSync must be(empty)
14 }
There is nothing special here, the test is same as for the ProductRoutes except for the changed URI and HTTP method. Also the code is a bit simpler because we do not need to generate a dynamic request URI like before.
1 val expectedStatusCode = Status.NoContent
2
3 s"return $expectedStatusCode" in {
4 forAll("product") { p: Product =>
5 val repo: Repository[IO] = new TestRepository[IO](Seq(p))
6 def service: HttpRoutes[IO] =
7 Router("/" -> new ProductsRoutes(repo).routes)
8 val response: IO[Response[IO]] = service.orNotFound.run(
9 Request(method = Method.POST, uri = Uri.uri("/products"))
10 .withEntity(p)
11 )
12 val result = response.unsafeRunSync
13 result.status must be(expectedStatusCode)
14 result.body.compile.toVector.unsafeRunSync must be(empty)
15 }
16 }
Saving a product using valid JSON payload should succeed and in fact it does because of the code we have in our TestRepository instance. If you remember we use the following code for saveProduct:
1 override def saveProduct(p: Product): F[Int] =
2 data.find(_.id === p.id).fold(0.pure[F])(_ => 1.pure[F])
This code will return a 0 if the product we try to save does not exist in the seed data set and only a 1 if it can be found within aforementioned set. This code clearly doesn’t make any sense except for our testing. This way we can ensure the behaviour of the save function without having to create a new Repository instance with hard coded behaviour. :-)
1 val expectedStatusCode = Status.InternalServerError
2
3 s"return $expectedStatusCode" in {
4 forAll("product") { p: Product =>
5 def service: HttpRoutes[IO] =
6 Router("/" -> new ProductsRoutes(emptyRepository).routes)
7 val response: IO[Response[IO]] = service.orNotFound.run(
8 Request(method = Method.POST, uri = Uri.uri("/products"))
9 .withEntity(p)
10 )
11 val result = response.unsafeRunSync
12 result.status must be(expectedStatusCode)
13 result.body.compile.toVector.unsafeRunSync must be(empty)
14 }
15 }
We use the empty repository this time to ensure that the saveProduct function will return a zero, triggering the desired logic in our endpoint. Almost done, so let’s check the endpoint for returning all products.
1 val expectedStatusCode = Status.Ok
2
3 s"return $expectedStatusCode and an empty list" in {
4 def service: HttpRoutes[IO] =
5 Router("/" -> new ProductsRoutes(emptyRepository).routes)
6 val response: IO[Response[IO]] = service.orNotFound.run(
7 Request(method = Method.GET, uri = Uri.uri("/products"))
8 )
9 val result = response.unsafeRunSync
10 result.status must be(expectedStatusCode)
11 result.as[List[Product]].unsafeRunSync mustEqual List.empty[Product]
12 }
We simply expect an empty list if no products exist. It is as simple as that and works right out of the box. Last but not least we need to test the return of existing products. But before we do this let’s take a look at our TestRepository implementation.
1 override def loadProducts() = Stream.empty
Uh, oh, that does not bode well! So we will need to fix that first. Because we were wise to chose fs2 as our streaming library of choice the solution is as simple as this.
1 override def loadProducts() = {
2 val rows = data.flatMap { p =>
3 val ns = p.names.toNonEmptyList.toList.to[Seq]
4 ns.map(n => (p.id, n.lang, n.name))
5 }
6 Stream.emits(rows)
7 }
Now we can write our last test.
1 implicit def decodeProducts: EntityDecoder[IO, List[Product]] = jsonOf
2 val expectedStatusCode = Status.Ok
3
4 s"return $expectedStatusCode and a list of products" in {
5 forAll("products") { ps: List[Product] =>
6 val repo: Repository[IO] = new TestRepository[IO](ps)
7 def service: HttpRoutes[IO] =
8 Router("/" -> new ProductsRoutes(repo).routes)
9 val response: IO[Response[IO]] = service.orNotFound.run(
10 Request(method = Method.GET, uri = Uri.uri("/products"))
11 )
12 val result = response.unsafeRunSync
13 result.status must be(expectedStatusCode)
14 result.as[List[Product]].unsafeRunSync mustEqual ps
15 }
16 }
To decode the response correctly an implicit EntityDecoder of the appropriate type is needed in scope. But the rest of the test should look pretty familiar to you by now.
It seems the only parts left to test are the FlywayDatabaseMigrator and our DoobieRepository classes. Testing them will require a running database so we are leaving the cosy world of unit tests behind and venture forth into integration test land. But fear not, we already have some - albeit impure - experience here.
Integration Tests
As usual we start up by implementing a base class that we can use to provide common settings and functions across our tests.
1 abstract class BaseSpec extends WordSpec
2 with MustMatchers
3 with ScalaCheckPropertyChecks
4 with BeforeAndAfterAll
5 with BeforeAndAfterEach {
6
7 protected val config = ConfigFactory.load()
8 protected val dbConfig = loadConfig[DatabaseConfig](config, "database")
9
10 override def beforeAll(): Unit = {
11 val _ = withClue("Database configuration could not be loaded!") {
12 dbConfig.isRight must be(true)
13 }
14 }
15 }
You can see that we keep it simple here and only load the database configuration and ensure that is has been indeed loaded correctly in the beforeAll function.
Testing the FlywayDatabaseMigrator
To ensure the basic behaviour of our FlywayDatabaseMigrator we write a simple test.
1 "the database is not available" must {
2 "throw an exception" in {
3 val cfg = DatabaseConfig(
4 driver = "This is no driver name!",
5 url = "jdbc://some.host/whatever",
6 user = "no-user",
7 pass = "no-password"
8 )
9 val migrator: DatabaseMigrator[IO] = new FlywayDatabaseMigrator
10 val program = migrator.migrate(cfg.url, cfg.user, cfg.pass)
11 an[FlywayException] must be thrownBy program.unsafeRunSync
12 }
13 }
Within this test we construct an invalid database configuration and expect that the call to migrate throws an exception. If you remember, we had this issue already and chose not to handle any exceptions but let the calling site do this - for example via a MonadError instance.
1 dbConfig.map { cfg =>
2 val migrator: DatabaseMigrator[IO] = new FlywayDatabaseMigrator
3 val program = migrator.migrate(cfg.url, cfg.user, cfg.pass)
4 program.unsafeRunSync must be > 0
5 }
6 // ---
7 dbConfig.map { cfg =>
8 val migrator: DatabaseMigrator[IO] = new FlywayDatabaseMigrator
9 val program = migrator.migrate(cfg.url, cfg.user, cfg.pass)
10 val _ = program.unsafeRunSync
11 program.unsafeRunSync must be(0)
12 }
The other two tests are also quite simple, we just expect it to return either zero or the number of applied migrations depending on the state of the database. It goes without saying that we of course use the beforeEach and afterEach helpers within the test to prepare and clean our database properly.
Last but not least we take a look at testing our actual repository implementation which uses Doobie. To avoid trouble we need to define a globally available ContextShift in our test which is as simple as this:
1 implicit val cs = IO.contextShift(ExecutionContexts.synchronous)
Now we can start writing our tests.
1 val tx = Transactor
2 .fromDriverManager[IO](c.driver, c.url, c.user, c.pass)
3 val repo = new DoobieRepository(tx)
4 forAll("ID") { id: ProductId =>
5 for {
6 rows <- repo.loadProduct(id)
7 } yield {
8 rows must be(empty)
9 }
10 }
Here we simply test that the loadProduct function returns an empty list if the requested product does not exist in the database.
1 forAll("product") { p: Product =>
2 for {
3 _ <- repo.saveProduct(p)
4 rows <- repo.loadProduct(p.id)
5 } yield {
6 rows must not be(empty)
7 Product.fromDatabase(rows) must contain(p)
8 }
9 }
From now on we’ll omit the transactor and repository creation from the code examples. As you can see a generated product is saved to the database and loaded again and verified in the end.
1 val rows = repo.loadProducts().compile.toList
2 rows.unsafeRunSync must be(empty)
Testing that loadProducts returns an empty stream if no products exist is as simple as the code above. :-)
1 forAll("products") { ps: List[Product] =>
2 for {
3 _ <- ps.traverse(repo.saveProduct)
4 rows = repo.loadProducts()
5 .groupAdjacentBy(_._1)
6 .map {
7 case (id, rows) => Product.fromDatabase(rows.toList)
8 }
9 .collect {
10 case Some(p) => p
11 }
12 .compile
13 .toList
14 } yield {
15 val products = rows.unsafeRunSync
16 products must not be(empty)
17 products mustEqual ps
18 }
19 }
In contrast the test code for checking the return of existing products is a bit more involving. But let’s step through it together. First we save the list of generated products to the database which we do using the traverse function provided by Cats. In impure land we used Future.sequence here if you remember - but now we want to stay pure. ;-)
Next we call our loadProducts function and apply a part of the logic from our ProductsRoutes to it, namely we construct a proper stream of products which we turn into a list via compile and list in the end. Finally we check that the list is not empty and equal to our generated list.
1 forAll("product") { p: Product =>
2 for {
3 cnt <- repo.saveProduct(p)
4 rows <- repo.loadProduct(p.id)
5 } yield {
6 cnt must be > 0
7 rows must not be(empty)
8 Product.fromDatabase(rows) must contain(p)
9 }
10 }
The code for testing saveProduct is nearly identical to the loadProduct test as you can see. We simply check additionally that the function returns the number of affected database rows.
1 forAll("product") { p: Product =>
2 for {
3 cnt <- repo.updateProduct(p)
4 rows <- repo.loadProduct(p.id)
5 } yield {
6 cnt must be(0)
7 rows must be(empty)
8 }
9 }
Updating a non existing product must return a zero and save nothing to the database, which is what we test above.
1 forAll("productA", "productB") { (a: Product, b: Product) =>
2 val p = b.copy(id = a.id)
3 for {
4 _ <- repo.saveProduct(a)
5 cnt <- repo.updateProduct(p)
6 rows <- repo.loadProduct(p.id)
7 } yield {
8 cnt must be > 0
9 rows must not be(empty)
10 Product.fromDatabase(rows) must contain(p)
11 }
12 }
Finally we test updating a concrete product by generating two of them, saving the first into the database and running an update using the second with the id from the first.
Wow, it seems we are finished! Congratulations, we can now check mark the point “write a pure http service in Scala” on our list. :-)