Leanpub: Publish Early, Publish Often

Tests

The Ruby community is well known for its adoption of good testing practices. For example, the PHP community is not even close the same level of understanding of the importance of testing in every stage of app development. Moreover, the average PHP developer does not write nearly as many tests as the average Ruby developer does. One of the secrets of Ruby’s testing success is its great tools. Ruby testing tools provide powerful syntax and features, made possible by metaprogramming and DSLs.

Let’s not pat ourselves on the back too quickly, though. As always, great power comes with great responsibility. These tools are a double-edged sword. You need to recognize the good parts and the not-so-good parts, so as not to hurt yourself.

Testing rules:

Tests should be simple and obvious. They should not require testing, themselves.
Each test should do only one thing.
Tests should have clear assertions.
Tests should be predictable and repeatable, producing the same result on each run.

Unit tests

Unit tests cover code-level logic. It instantiates an object, and makes assertions on the result of a method or function call. There are two popular unit testing solutions in Ruby: Minitest and RSpec.

RSpec is a very popular gem that provides an expressive DSL for all aspects of testing. It allows writing test in a BDD “expects” style. Its assertions tend to be human-readable, and it has lots of special extensions. However, its main advantage is also its main drawback – you need to learn a large DSL and assertion syntax. RSpec extensions make this worse, requiring you to put in quite a lot of effort to learn everything. RSpec has community-backed best practices that can be accessed on betterspecs.

 1 RSpec.describe "Using an array as a stack" do
 2   def build_stack
 3     []
 4   end
 5 
 6   before(:example) do
 7     @stack = build_stack
 8   end
 9 
10   it 'is initially empty' do
11     expect(@stack).to be_empty
12   end
13 
14   context "after an item has been pushed" do
15     before(:example) do
16       @stack.push :item
17     end
18 
19     it 'allows the pushed item to be popped' do
20       expect(@stack.pop).to eq(:item)
21     end
22   end
23 end

Minitest became a part of the Ruby standard library, and that is why it is preferable for testing gems and libs, as it does not require additional dependencies. It provides a very small assertions interface, that is easy to learn and adopt. The main advantage of Minitest is that tests are just POROs. That is why you do not need to learn anything new to structure your tests – it is just pure Ruby. Use the same techniques as in your other code. Code initialization and calls are almost identical to real usage. IMHO Minitest is the better choice for writing simple, clear and predictable tests.

 1 class TestMeme < Minitest::Test
 2   def setup
 3     @meme = Meme.new
 4   end
 5 
 6   def test_that_kitty_can_eat
 7     assert_equal "OHAI!", @meme.i_can_has_cheezburger?
 8   end
 9 
10   def test_that_it_will_not_blend
11     refute_match /^no/i, @meme.will_it_blend?
12   end
13 
14   def test_that_will_be_skipped
15     skip "test this later"
16   end
17 end

Minitest also provides a Spec style, which has a lot of RSpec syntactic sugar, but still has a dramatically simpler code base (when you look inside Minitest gem code).

 1 describe Meme do
 2   before do
 3     @meme = Meme.new
 4   end
 5 
 6   describe "when asked about cheeseburgers" do
 7     it "must respond positively" do
 8       @meme.i_can_has_cheezburger?.must_equal "OHAI!"
 9     end
10   end
11 
12   describe "when asked about blending possibilities" do
13     it "won't say no" do
14       @meme.will_it_blend?.wont_match /^no/i
15     end
16   end
17 end

Stay consistent, don’t write tests in different styles with Minitest. Choose the one you like the most, and write all tests in it. But remember that using Spec style with Minitest reduces its advantages, like normal Ruby syntax and usage of normal OOP techniques.

Test Behavior, not Configuration

The shoulda-matchers gem is very popular for testing aspects of ActiveRecord model configuration, like has_one.

1 class Person < ActiveRecord::Base
2   has_one :partner
3 end
4 
5 class PersonTest < ActiveSupport::TestCase
6   should have_one(:partner)
7 end

But why do we need to test indirect signs of expected behavior instead of testing the behavior directly?

We could better write behavior tests like this:

1 class PersonTest < Minitest::Test
2   def test_has_parter
3     person = Person.new
4     partner = Partner.new
5     assert_equal partner, person.partner
6   end
7 end

The test above will continue passing, regardless of any changes to the implementation of person.partner. This is exactly how your app expects the Person class to behave, so why should your tests be different, and rely upon the internal implementation?

Test Behavior, not Configuration

Integration tests

Ruby allows you to write integration tests. These emulate the real interaction between the web app and the browser. They usually contain a set of commands to click on links, visit certain pages, fill in form fields, and validate the results of these actions.

Integration tests can be written using the Capybara framework. Basically, it acts as a “fake browser,” making requests to your Rack app and parsing the responses, but is not capable of running JavaScript. To fully emulate an end-user browser, you need to use the Poltergeist or Capybara Webkit add-ons. They run the same commands inside a headless Webkit browser. Of course, this incurs a speed penalty. Tests with JS run slower than tests without JS. You need to be aware of that, and activate JS per test, only in the tests that require it.

Understand that integration tests should not make assertions against the internal state of the web app. They should not make assertions on code variables, etc. They should only assert external output, like the response body and HTTP status codes.

Also, do not overuse integration tests. Keep in mind that in any (ANY!) case, integration tests are an order of magnitude slower than unit tests. When you write bad quality code – e.g. putting a lot of logic into Controllers – the only way to test it is to visit the corresponding page. You will need to execute the whole app request cycle to test only tiny things. That should encourage you to remove logic from the controller, put it into separate classes, and just call the new classes from the controller. That way, you can test 90% of the logic via unit tests (which are fast) and make only a few “smoke tests” on the controller action (to test success/failure scenarios, but not every part of the business logic).

Test data: Fixtures vs Factories

Stub external services call

Your tests should not depend on the availability of external services. Ideally, you should be able to run all tests without Internet access, but that doesn’t mean that external integrations should not be tested. If your app makes a request to an external service, this request should be stubbed. Instead of making a real call, it should “pretend” and return a ready-made response (there could be a couple of different responses, to test success/failure). Gems like Webmock and VCR can help you to catch real responses and then reuse them for subsequent test runs, or to write your own response content.

Stub request with Webmock:

1 stub_request(:post, "www.example.com")
2     .with(:query => {'user' => 'Ievgen'})
3     .to_return(:body => "Nice work!")

Catch requests and return a “fake” response with VCR:

1   VCR.use_cassette("synopsis") do
2       response = Net::HTTP.get_response(URI('http://www.iana.org/domains/reserved'))
3       assert_match /Example domains/, response.body
4     end

Speed-up test

To achieve fast tests, stub behavior, heavy computations, and external web calls, in addition to mocking dependencies. You can also run tests in parallel – another reason to keep them completely independent from one another.

Gem: parallel_tests

Learn Tests Design

To write cost-efficient and effort-efficient tests – tests that should not be rewritten from scratch after any small refactoring – you should design your tests almost as well as you design your other code.

Only test public interface of classes
Don’t test tiny sensitive private methods (they are very likely to change often)
Reuse repetitive test via mixins
Avoid redundantly testing the same functionality multiple times. Code should be organized in layers that are testable separately from each other.