Leanpub: Publish Early, Publish Often

Assertions

Assert Magic

One of the many great features of Spock is the way assertion failures are shown. The power assert from Groovy is based on Spock's assertion feature, but Spock takes it to a next level. Let's create a simple specification for a course service, which is able to create new courses:

// File: CourseServiceSpec.groovy
package com.mrhaki.blog

@Grab('org.spockframework:spock-core:0.4-groovy-1.7')
import spock.lang.Specification

class CourseServiceSpec extends Specification {

    def "Create new course with teacher and description"() {
        setup:
        def courseService = new CourseService()

        when:
        def course = courseService.create('mrhaki', 'Groovy Goodness')

        then:
        'Mrhaki' == course.teacher.name
        'Groovy Goodness' == course.description
        !course.students
    }

}

class CourseService {
    Course create(String teacherName, String description) {
        new Course(teacher: new Person(name: teacherName), description: description)
    }
}

class Course {
    Person teacher
    String description
    List<Person> students
}

class Person {
    String name
}

At lines 16, 17, 18 we define the assertions for our specification. First of all we notice we don't add the keyword assert for each assertion. Because we are in the then block we can omit the assert keyword. Notice at line 18 we can test for null values by using the Groovy truth. We also notice we only have to write a simple assertion. Spock doesn't need a bunch of assertEquals() methods like JUnit to test the result.

Now it is time to run our specification as JUnit test and see the result:

$ groovy CourseServiceSpec.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 232
Test Failure: Create new course with teacher and description(com.mrhaki.blog.CourseServiceSpec)
Condition not satisfied:

'Mrhaki' == course.teacher.name
         |  |      |       |
         |  |      |       mrhaki
         |  |      com.mrhaki.blog.Person@34b6a6d6
         |  com.mrhaki.blog.Course@438346a3
         false
         1 difference (83% similarity)
         (M)rhaki
         (m)rhaki

    at com.mrhaki.blog.CourseServiceSpec.Create new course with teacher and description(CourseServiceS\
pec.groovy:16)

Wow that is a very useful message for what is going wrong! We can see our condition 'Mrhaki' == course.teacher.name is not satisfied, but we even get to see which part of the String value is not correct. In this case the first character should be lowercase instead of uppercase, but the message clearly shows the rest of the String value is correct. As a matter of fact we even know 83% of the String values is similar.

Another nice feature of Spock is that only the line which is important is shown in the abbreviated stacktrace. So we don't have to scroll through a big stacktrace with framework classes to find out where in our class the exception occurs. We immediately see that at line 16 in our specification the condition is not satisfied.

In our sample we have three assertions to be checked in the then. If we get a lot of assertions in the then block we can refactor our specification and put the assertions in a new method. This method must have void return type and we must add the assert keyword again. After these changes the assertions work just like when we put them in the then block:

package com.mrhaki.blog

@Grab('org.spockframework:spock-core:0.4-groovy-1.7')
import spock.lang.Specification

class CourseServiceSpec extends Specification {

    def "Create new course with teacher and description"() {
        setup:
        def courseService = new CourseService()

        when:
        def course = courseService.create('mrhaki', 'Groovy Goodness')

        then:
        assertCourse course
    }

    private void assertCourse(course) {
        assert 'mrhaki' == course.teacher.name
        assert 'Grails Goodness' == course.description
        assert !course.students
    }

}

class CourseService {
    Course create(String teacherName, String description) {
        new Course(teacher: new Person(name: teacherName), description: description)
    }
}

class Course {
    Person teacher
    String description
    List<Person> students
}

class Person {
    String name
}

When can run our specification and get the following output:

$ groovy CourseServiceSpec.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 228
Test Failure: Create new course with teacher and description(com.mrhaki.blog.CourseServiceSpec)
Condition not satisfied:

'Grails Goodness' == course.description
                  |  |      |
                  |  |      Groovy Goodness
                  |  com.mrhaki.blog.Course@3435ec9
                  false
                  4 differences (73% similarity)
                  Gr(ails) Goodness
                  Gr(oovy) Goodness

    at com.mrhaki.blog.CourseServiceSpec.assertCourse(CourseServiceSpec.groovy:21)
    at com.mrhaki.blog.CourseServiceSpec.Create new course with teacher and description(CourseServiceS\
pec.groovy:16)

Spock provides very useful assertion messages when the condition is not satisfied. We see immediately what wasn't correct, because of the message and the fact the stacktrace only shows the line where the code is wrong.

Original post written on July 5, 2010

Grouping Assertions

In a Spock specification we write our assertion in the then: or expect: blocks. If we need to write multiple assertions for an object we can group those with the with method. We specify the object we want write assertions for as argument followed by a closure with the real assertions. We don't need to use the assert keyword inside the closure, just as we don't have to use the assert keyword in an expect: or then: block.

In the following example specification we have a very simple implementation for finding an User object. We want to check that the properties username and name have the correct value.

@Grab("org.spockframework:spock-core:1.0-groovy-2.4")
import spock.lang.Specification
import spock.lang.Subject

class UserServiceSpec extends Specification {

    @Subject
    private UserService userService = new DefaultUserService()
    
    def "check user properties"() {
        when:
        final User user = userService.findUser("mrhaki")
        
        then:
        // Assert each property.
        user.username == "mrhaki"
        user.name == "Hubert A. Klein Ikkink"
    }

    def "check user properties using with()"() {
        when:
        final User user = userService.findUser("mrhaki")
        
        then:
        // Assert using with().
        with(user) {
            username == "mrhaki"
            name == "Hubert A. Klein Ikkink"
        }
    }

    def "check expected user properties using with()"() {
        expect:
        with(userService.findUser("mrhaki")) {
            username == "mrhaki"
            name == "Hubert A. Klein Ikkink"
        }
    }
    

}

interface UserService {
    User findUser(final String username)    
}

class DefaultUserService implements UserService {
    User findUser(final String username) {
        new User(username: "mrhaki", name: "Hubert A. Klein Ikkink")
    }
}

class User {
    String username
    String name
}

Written with Spock 1.0.

Original post written on January 29, 2016

Group Assertions With verifyAll

We all know writing tests or specifications with Spock is fun. We can run our specifications and when one of our assertions in a feature method invalid, the feature method is marked as failed. If we have more than one assertion in our feature method, only the first assertion that fails is returned as an error. Any other assertion after a failing assertion are not checked. To let Spock execute all assertions and return all failing assertions at once we must use the verifyAll method. We pass a closure with all our assertions to the method. All assertions will be checked when use the verifyAll and if one or more of the assertions is invalid the feature method will fail.

In the following example specification we have 3 assertions in the feature method check all properties are valid. We don't use the verifyAll method in our first example.

// File: verifyall.groovy
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.Specification
import spock.lang.Subject

class CourseSpec extends Specification {

    @Subject
    private course = new Course()
    
    void 'check all properties are valid'() {
        given:
        course.with {
            name = 'G'
            location = 'T'
            numberOfDays = 0
        }
    
        expect:
        with(course) {
            name.size() > 2
            location.size() > 2
            numberOfDays > 0
        }
    }

}

class Course {
    String name
    String location
    int numberOfDays
}

Let's see what happens when we don't use the verifyAll method and run the specification:

$ groovy verifyall.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 28
Test Failure: check all properties are valid(CourseSpec)
Condition not satisfied:

name.size() > 2
|    |      |
G    1      false

 at CourseSpec.check all properties are valid_closure2(verifyall.groovy:21)
 at groovy.lang.Closure.call(Closure.java:414)
 at spock.lang.Specification.with(Specification.java:186)
 at CourseSpec.check all properties are valid(verifyall.groovy:20)

In the next example we use verifyAll to group the assertions for our course object:

// File: verifyall.groovy
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.Specification
import spock.lang.Subject


class CourseSpec extends Specification {

    @Subject
    private course = new Course()
    
    void 'check all properties are valid'() {
        given:
        course.with {
            name = 'G'
            location = 'T'
            numberOfDays = 0
        }
    
        expect:
        with(course) {
            verifyAll {
                name.size() > 2
                location.size() > 2
                numberOfDays > 0
            }
        }
    }

}

class Course {
    String name
    String location
    int numberOfDays
}

We re-run the specification and now we get different output with all three assertions mentioned:

$ groovy verifyall.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 26
Test Failure: check all properties are valid(CourseSpec)
Condition failed with Exception:

with(course) { verifyAll { name.size() > 2 location.size() > 2 numberOfDays > 0 } }

 at CourseSpec.check all properties are valid(verifyall.groovy:20)
Caused by: org.junit.runners.model.MultipleFailureException: There were 3 errors:
  org.spockframework.runtime.ConditionNotSatisfiedError(Condition not satisfied:

name.size() > 2
|    |      |
G    1      false
)
  org.spockframework.runtime.ConditionNotSatisfiedError(Condition not satisfied:

location.size() > 2
|        |      |
T        1      false
)
  org.spockframework.runtime.ConditionNotSatisfiedError(Condition not satisfied:

numberOfDays > 0
|            |
0            false
)
 at CourseSpec.$spock_feature_0_0_closure2$_closure3(verifyall.groovy:24)
 at CourseSpec.$spock_feature_0_0_closure2$_closure3(verifyall.groovy)
 at groovy.lang.Closure.call(Closure.java:414)
 at spock.lang.Specification.verifyAll(Specification.java:223)
 at CourseSpec.check all properties are valid_closure2(verifyall.groovy:21)
 at groovy.lang.Closure.call(Closure.java:414)
 at spock.lang.Specification.with(Specification.java:186)
 ... 1 more

Written with Spock 1.1-groovy-2.4.

Original post written on September 18, 2017

Check for Exceptions with Spock

With Spock we can easily write feature methods in our specification to test if an exception is thrown by the methods invoked in a when block. Spock support exception conditions with the thrown() and notThrown() methods. We can even get a reference to the expected exception and check for example the message.

The following piece of code contains the specification to check for exceptions that can be thrown by a cook() method of the RecipeService class. And we check that exception are not thrown. The syntax is clear and concise, what we expect from Spock.

package com.mrhaki.blog

@Grab('org.spockframework:spock-core:0.5-groovy-1.7')
import spock.lang.Specification

class RecipeServiceSpec extends Specification {
    def "If cooking for more minutes than maximum allowed by recipe throw BurnedException"() {
        setup:
            def recipeService = new RecipeService()
            def recipe = new Recipe(time: 5, device: 'oven')

        when:
            recipeService.cook(recipe, 10, 'oven')

        then:
            thrown BurnedException
    }

    def "If cooking on wrong device throw InvalidDeviceException"() {
        setup:
            def recipeService = new RecipeService()
            def recipe = new Recipe(device: 'oven', time: 10)

        when:
            recipeService.cook(recipe, 2, 'microwave')

        then:
            InvalidDeviceException ex = thrown()
            // Alternative syntax: def ex = thrown(InvalidDeviceException)
            ex.message == 'Please use oven for this recipe.'
    }

    def """If the recipe is cooked on the right device and
           for the correct amount of minutes, 
           then no exception is thrown"""() {
        setup:
            def recipeService = new RecipeService()
            def recipe = new Recipe(device: 'oven', time: 30)

        when:
            recipeService.cook(recipe, 30, 'oven')

        then:
            notThrown BurnedException
            notThrown InvalidDeviceException
    }
}

class RecipeService {
    def cook(Recipe recipe, int minutes, String device) {
        if (minutes > recipe.time) {
            throw new BurnedException()
        }
        if (device != recipe.device) {
            throw new InvalidDeviceException("Please use $recipe.device for this recipe.")
        }
    }
}

class Recipe {
    int time
    String device
}

class BurnedException extends RuntimeException {
    BurnedException(String message) {
        super(message)
    }
}

class InvalidDeviceException extends RuntimeException {
    InvalidDeviceException(String message) {
        super(message)
    }
}

Original post written on January 21, 2011

Check No Exceptions Are Thrown At All

In a previous post we learned that we can check a specific exception is not thrown in our specification with the notThrown method. If we are not interested in a specific exception, but just want to check no exception at all is thrown, we must use the noExceptionThrown method. This method return true if the called code doesn't throw an exception.

In the following example we invoke a method (cook) that can throw an exception. We want to test the case when no exception is thrown:

package com.mrhaki.blog

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')
import spock.lang.Specification

class RecipeServiceSpec extends Specification {
    def """If the recipe is cooked on the right device and
           for the correct amount of minutes,
           then no exception is thrown"""() {
        setup:
        def recipeService = new RecipeService()
        def recipe = new Recipe(device: 'oven', time: 30)

        when:
        recipeService.cook(recipe, 30, 'oven')

        then: 'no exceptions must be thrown'
        noExceptionThrown()
    }
}

class RecipeService {
    def cook(Recipe recipe, int minutes, String device) {
        if (minutes > recipe.time) {
            throw new BurnedException()
        }
        if (device != recipe.device) {
            throw new InvalidDeviceException("Please use $recipe.device for this recipe.")
        }
    }
}

class Recipe {
    int time
    String device
}

import groovy.transform.InheritConstructors

@InheritConstructors
class BurnedException extends RuntimeException {
}

@InheritConstructors
class InvalidDeviceException extends RuntimeException {
}

Written with Spock 1.0-groovy-2.4.

Original post written on September 30, 2016

Assert Elements In Collections In Any Order

Since Spock 2.1 we have 2 new operators we can use for assertions to check collections: =~ and ==~. We can use these operators with implementations of the Iterable interface when we want to check that a given collection has the same elements as an expected collection and we don’t care about the order of the elements. Without the new operators we would have to cast our collections to a Set first and than use the == operator.

The difference between the operators =~ and ==~ is that =~ is lenient and ==~ is strict. With the lenient match operator we expect that each element in our expected collection appears at least once in the collection we want to assert. The strict match operator expects that each element in our expected collection appears exactly once in the collection we want to assert.

In the following example we see different uses of the new operators and some other idiomatic Groovy ways to check elements in a collection in any order:

package mrhaki

import spock.lang.Specification;

class CollectionConditions extends Specification {

    void "check items in list are present in the same order"() {
        when:
        List<Integer> result = [1, 10, 2, 20]

        then:
        // List result is ordered so the items
        // are not the same as the expected List.
        result != [20, 10, 2, 1]
        result == [1, 10, 2, 20]

        // We can cast the result and expected list to Set
        // and now the contents and order is the same.
        result as Set == [20, 10, 2, 1] as Set
    }

    void "check all items in list are present in any order"() {
        when:
        List<Integer> result = [1, 10, 2, 20]

        then:
        result ==~ [20, 10, 2, 1]

        /* The following assert would fail:
           result ==~ [20, 10]

           Condition not satisfied:

           result ==~ [20, 10]
           |      |
           |      false
           [1, 10, 2, 20]

           Expected: iterable with items [<20>, <10>] in any order
                but: not matched: <1>*/

        // Negating also works
        result !==~ [20, 10]
    }

    void "lenient check all items in list are present in any order"() {
        when:
        List<Integer> result = [1, 1, 10, 2, 2, 20]

        then:
        // result has extra values 1, 2 but with lenient
        // check the assert is still true.
        result =~ [20, 10, 2, 1]

        /* The following assert would fail:

        result =~ [20, 10]

        Condition not satisfied:

        result =~ [20, 10]
        |      |
        |      false
        |      2 differences (50% similarity, 0 missing, 2 extra)
        |      missing: []
        |      extra: [1, 2]
        [1, 10, 2, 20] */

        // Negating works
        result !=~ [20, 10]
    }

    void "check at least one item in list is part of expected list in any order"() {
        when:
        List<Integer> result = [1, 10, 2, 20]

        then:
        result.any { i -> i in [20, 10]}
    }

    void "check every item in list is part of expected list in any order"() {
        when:
        List<Integer> result = [1, 1, 10, 2, 2, 20]

        then:
        result.every { i -> i in [20, 10, 2, 1]}
    }
}

Written with Spock 2.3-groovy-4.0.

Original post written on March 24, 2023

Support for Hamcrest Matchers

Spock has support for Hamcrest matchers and adds some extra syntactic sugar. In an expect: block in a Spock specification method we can use the following syntax value Matcher. Let's create a sample Spock specification and use this syntax with the Hamcrest matcher hasKey:

// File: SampleSpecification.groovy
package com.mrhaki.spock

@Grab('org.hamcrest:hamcrest-all:1.3')
import static org.hamcrest.Matchers.*

@Grab('org.spockframework:spock-core:0.7-groovy-2.0')
import spock.lang.Specification

class SampleSpecification extends Specification {

    def "sample usage of hamcrest matcher hasKey"() {
        given:
        final sampleMap = [name: 'mrhaki']

        expect:
        sampleMap hasKey('name')
        sampleMap not(hasKey('name')) // To show assertion message.
    }

}

We can run the code ($groovy SampleSpecification.groovy) and see in the output a very useful assertion message for the second matcher in the expect: block. We directly see what went wrong and what was expected.

$ groovy SampleSpecification.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 210
Test Failure: sample usage of hamcrest matcher hasKey(com.mrhaki.spock.SampleSpecification)
Condition not satisfied:

sampleMap not(hasKey('name'))
|         |
|         false
[name:mrhaki]

Expected: not map containing ["name"->ANYTHING]
     but: was <{name=mrhaki}>

 at com.mrhaki.spock.SampleSpecification.sample usage of hamcrest matcher hasKey(SampleSpecification.g\
roovy:18)

With Spock we can rewrite the specification and use the static method that() in spock.util.matcher.HamcrestSupport as a shortcut for the Hamcrest assertThat() method. The following sample shows how we can use that(). With this method we can use the assertion outside an expect: or then: block.

// File: SampleSpecification.groovy
package com.mrhaki.spock

@Grab('org.hamcrest:hamcrest-all:1.3')
import static org.hamcrest.Matchers.*

@Grab('org.spockframework:spock-core:0.7-groovy-2.0')
import static spock.util.matcher.HamcrestSupport.*
import spock.lang.Specification

class SampleSpecification extends Specification {

    def "sample usage of hamcrest matcher hasKey"() {
        given:
        final sampleMap = [name: 'mrhaki']

        expect:
        that sampleMap, hasKey('name')
    }

}

Finally we can use the expect() method in spock.util.matcher.HamcrestSupport to add the assertion in a then: block. This improves readability of our specification.

// File: SampleSpecification.groovy
package com.mrhaki.spock

@Grab('org.hamcrest:hamcrest-all:1.3')
import static org.hamcrest.Matchers.*

@Grab('org.spockframework:spock-core:0.7-groovy-2.0')
import static spock.util.matcher.HamcrestSupport.*
import spock.lang.Specification

class SampleSpecification extends Specification {

    def "sample usage of hamcrest matcher hasKey"() {
        when:
        final sampleMap = [name: 'mrhaki']

        then:
        expect sampleMap, hasKey('name')
    }

}

Code written with Spock 0.7-groovy-2.0

Original post written on May 14, 2013

Using a Custom Hamcrest Matcher

In a previous blog post we learned how we can use Hamcrest matchers. We can also create a custom matcher and use it in our Spock specification. Let's create a custom matcher that will check if elements in a list are part of a range.

In the following specification we create the method inRange() which will return an instance of a Matcher object. This object must implement a matches() method and extra methods to format the description when the matcher fails. We use Groovy's support to create a Map and turn it into an instance of BaseMatcher.

// File: SampleSpecification.groovy
package com.mrhaki.spock

@Grab('org.hamcrest:hamcrest-all:1.3')
import static org.hamcrest.Matchers.*
import org.hamcrest.*

@Grab('org.spockframework:spock-core:0.7-groovy-2.0')
import static spock.util.matcher.HamcrestSupport.*
import spock.lang.Specification

class SampleSpecification extends Specification {

    def "sample usage of custom hamcrest matcher"() {
        given:
        final list = [2,3,4]

        expect:
        that list, inRange(0..10)
        that list, not(inRange(0..3))
    }

    /**
     * Create custom Hamcrest matcher to check if a list has elements
     * that are contained in the define range.
     *
     * @param range Range to check if elements in the list are in this range.
     * @return Hamcrest matcher to check if elements in the list are part of the range.
     */
    private inRange(final range) {
        [
            matches: { list -> range.intersect(list) == list },
            describeTo: { description ->
                description.appendText("list be in range ${range}")
            },
            describeMismatch: { list, description ->
                description.appendValue(list.toListString()).appendText(" was not in range ${range}")
            }
        ] as BaseMatcher
    }

}

We can run the specification ($ groovy SampleSpecification.groovy) and everything should work and all tests must pass.

We change the code to see the description we have added. So we change that list, inRange(0..10) to that list, inRange(0..3). We run the specification again ($ groovy SampleSpecification.groovy) and look at the output:

JUnit 4 Runner, Tests: 1, Failures: 1, Time: 200
Test Failure: sample usage of custom hamcrest matcher(com.mrhaki.spock.SampleSpecification)
Condition not satisfied:

that list, inRange(0..3)
|    |
|    [2, 3, 4]
false

Expected: list be in range [0, 1, 2, 3]
     but: "[2, 3, 4]" was not in range [0, 1, 2, 3]

    at com.mrhaki.spock.SampleSpecification.sample usage of custom hamcrest matcher(SampleSpecificatio\
n.groovy:18)

Notice the output shows the text we have defined in the describeTo() and describeMismatch() methods.

Code written with Spock 0.7-groovy-0.2.

Original post written on May 14, 2013