Using minitest with Rails

minitest is the hot new Ruby testing framework that's included with Rails 1.9 (so actually, it's not that new, but it is hot). I've been using it with Rails for the first time, and have found it a joy to work with. In this post I'll briefly cover how to set it up, and explain why I've been enjoying it so much.

Installation

The minitest-rails gem is designed to make working with minitest in Rails especially easy, and I have to say, it does a great job. I normally prefer to avoid libraries that are designed to integrate separate frameworks, preferring to find out how things work for myself, but minitest-rails is actually quite small. I recommend having a browse through the code to see what it's up to (visit the GitHub repository and then press "t" to start browsing through the files).

To install minitest-rails the first thing we need to do is to add it to our Gemfile. It will pull in the minitest gem for us (though minitest is included in Ruby 1.9 you can also get the very latest version as a gem).

group :development, :test do
  gem 'minitest-rails'
end

The current version of minitest-rails is 0.0.5, but if you add that code to your Gemfile today you'll find that Bundler installs version 0.0.1 instead. Why? Well, minitest-rails 0.0.5 has a hard coded dependency on actionpack version 3.1.4.rc1, which rather scuppers any attempts to load it at the same time as actionpack 3.2.1. Bundler chooses to downgrade minitest-rails until it finds one that doesn't care too much which version of actionpack you install it with.

One of the minitest-rails contributors has worked around this problem by updating the dependencies on his GitHub repository, and you can install the gem from there instead. My Gemfile currently looks like this:

group :development, :test do
  gem 'minitest-rails',
    :git => 'git://github.com/rawongithub/minitest-rails.git',
    :branch => 'gemspec'
end

Note that I had to use his special gemspec branch (a "gemspec" is a file that describes how to build a rubygem package), as the files on master are packaged with Hoe (and Hoe is so broken that you don't commit a gemspec file). This prevents Bundler from being able to build and install the gem from the master branch. Hoe is a virus, and a pretty crappy one at that. But I digress...

I'm using Rails 3.2.1 and had a couple of problems getting compatible versions of the latest gems installed. Compatibility and version numbers may still be shifting around a bit, as Rails 3.2.1 is still quite fresh. I suggest you run bundle update and then check the versions in your Gemfile.lock file to make sure that you've got the latest compatible versions installed before continuing.

Now that minitest is installed we need to take a couple of steps to configure our app to use it. The minitest-rails README tells you what to do when you're creating a new project without the default test files:

$ rails new myapp --skip-test-unit

Sadly the README doesn't cover what you should do if you want to use minitest on an existing project, but it's not hard.

We need to stop Rails from loading test-unit. Find this line in config/application.rb:

require 'rails/all'

Replace it with this:

require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "active_resource/railtie"
require "sprockets/railtie"
# require "rails/test_unit/railtie"

If you're already using test-unit in your project, let's move those tests out of the way for a moment:

$ mv test test.test-unit

Now we can configure minitest. Run the generator to create test/minitest_helper.rb:

$ rails generate mini_test:install

That's it, you're setup.

Using minitest for BDD

minitest comes with its own lightweight spec-style DSL for BDD. If you want to use it all you need to do is to require minitest/spec, but the good news is that by default minitest-rails requires minitest/autorun for us, which is really just an easy way to require everything you need in one line.

Mocking and stubbing

Minitest does include mock object support, but not stubbing. You could argue that stubs are very easy to create yourself in Ruby, and that you don't really need explicit support for them. I didn't argue that; I like to be explicit about the difference between the two when writing my tests.

So I installed mocha, the king of Ruby mock object libraries. You'll need the latest version of mocha in order for it to play nicely with Rails 3.2.1. I'm using 0.10.4, and refer you to the usefulness of bundle update again (bad things happen if you use 0.10.0).

Coloured test output

There are two options for colouring the output that I've discovered so far:

1. minitest/pride
2. The turn gem

They're a little different, and both worth a look.

To enable pride just require minitest/pride after minitest/autorun in your minitest_helper.rb file. Here's my new project's test suite output with pride:

To enable turn just add it to the :development and :test groups in your Gemfile and re-run bundle. Here's the output from turn:

I've plumped for turn because it tells you how long your tests take to execute, and highlights the error message when Ruby raises an exception.

Running tests automatically with guard

Guard is great. I was banging on about using it to reload web pages for a live preview of your articles on the Nesta blog just last week, and here I am doing it again.

Add guard-minitest to the development and test groups in your Gemfile, then install the gem and create your Guardfile:

$ bundle
$ guard init minitest

Guard allows you to specify exactly how its various plugins should behave, and you'll need to edit the Guardfile so that it can find your models, controllers, tests, etc.

The current version of guard-minitest suggests the wrong paths (your model tests now live in test/models, rather than test/unit -- hurray!), so you'll want to tweak that. Here's my config:

guard 'minitest' do
  watch(%r|^test/.+_test.rb|)
  watch(%r|^lib/(.*)([^/]+)\.rb|) { |m| "test/#{m[1]}test_#{m[2]}.rb" }
  watch(%r|^test/.*test_helper\.rb|) { "test" }

  watch(%r|^app/controllers/(.*)\.rb|) { |m| "test/controllers/#{m[1]}_test.rb" }
  watch(%r|^app/helpers/(.*)\.rb|) { |m| "test/helpers/#{m[1]}_test.rb" }
  watch(%r|^app/models/(.*)\.rb|) { |m| "test/models/#{m[1]}_test.rb" }
end

Now when I run guard it runs my tests as I save them (or the corresponding files that are under test), and I immediately get to see how quickly they're running with turn. To run all the tests just press enter at the Guard prompt.

Why I like it so much

Most experienced ruby programmers aren't averse to opening ruby gems in their editor and having a good poke around. Since I setup MacVim to run in a terminal the other week, I've been able to pop open vim extremely quickly and use the command-t vim plugin (or Ack) to browse the code or investigate the cause of an error message. It's quicker than using Google and stack overflow, and frequently much more informative.

As a result I've been noticing and appreciating well structured projects and clean code more than I normally would, and investigating minitest has been a joy.

When I installed the database_cleaner gem (which cleans up your test database for you in between consecutive test runs), it was with the expectation that it might take me half an hour to get it running in my project.

I've spent longer than that in the past when working with RSpec. I've always had to Google for the right incantation to persuade RSpec to run code before the start of all my tests, and though it works fine, it's not exactly intuitive or easy to remember. The last time I tried it database_cleaner and RSpec didn't get on, and it took me longer still. It's friction that I could well do without.

Let's have a look at what minitest has put in our minitest_helper.rb file:

class MiniTest::Rails::Spec
  # Uncomment if you want to support fixtures for all specs
  # or place within spec class you want to support fixtures for
  # include MiniTest::Rails::Fixtures
end

class MiniTest::Rails::Model
  # Add methods to be used by model specs here
end

class MiniTest::Rails::Controller
  # Add methods to be used by controller specs here
end

There's more boilerplate for helpers, mailers, etc., but you get the idea...

Nice and clean, and fairly obvious to anybody who has seen test-unit how you're expected to call some code before a test; you'd add setup and teardown methods.

I'm using the minitest/spec DSL, so my first thought was "if I'm using describe blocks, my test cases probably won't inherit directly from the MiniTest::Spec class, so the boilerplate code might not work for me..."

And then I remembered what I'd seen in the source of minitest/spec; the spec DSL is a lightweight wrapper around the test-unit–style classes. When you use describe you are in fact creating instances of MiniTest::Spec that inherit from MiniTest::Unit::TestCase (the super class of all test classes).

Brilliant! So to setup database_cleaner I added these methods to the boilerplate in minitest_helper.rb:

class MiniTest::Rails::Spec
  def setup
    DatabaseCleaner.start
  end

  def teardown
    DatabaseCleaner.clean
  end
end

It worked first time. You might be wondering whether or not you need to call setup and teardown from within your specs' before and after blocks. You don't; minitest takes care of all that for you, automatically. For somebody who has struggled with various implementations of nested BDD–style context blocks, this is wonderful.

Never underestimate the benefit of a good design and a small code base.

I also feel confident that now that I've adopted minitest, that the developers won't keep changing the API on me.

RSpec has been a moving target over the last few years, when it was frankly perfectly adequate at version 1. Those API changes may have been deemed necessary by the maintainers (which is fine; it's their project), but I've lost at least three full working days trying to deal with changes made to RSpec.

Even the simple things in RSpec are unnecessarily difficult.

By comparison to minitest's simple boilerplate above, here's some code for configuring RSpec to make code available to all your controller specs:

RSpec.configure do |config|
  config.include ControllerSpecHelpers, :type => :controller
end

You put code that you want to make available to your controller specs (or model specs, etc.) in a module and tell RSpec that you'd like it to include it. I'm really not sure why you wouldn't just have a class, and call Ruby's include method. Maybe that doesn't seem as "cool".

My problem with it is that you can't infer anything about what's going on behind the scenes. For example, what other values could you pass in for the :type parameter? Off to the docs you go (or more likely, the code). And when you try and find the code, which file do you look in? You can't grep for the name of a class, and you'll not find the code that implements it by searching for RSpec.configure. I don't use bundle open with RSpec; it gives me the willies. And which gem would you open anyway? There's rspec, rspec-core, rspec-expectations, rspec-mocks and rspec-rails to consider.

Talking of which, let's see how big these projects are. I wrote a bash function to count the number of lines in all the .rb files inside each gem's lib folder.

$ count_lines()
{
    echo "$(find $1 -name \*.rb | xargs wc -l | tail -n 1) [$1]"
}

Running it on RSpec we get a grand total of 12,980 lines:

   9 total [rspec-2.8.0/lib]
5732 total [rspec-core-2.8.0/lib]
2625 total [rspec-expectations-2.8.0/lib]
2257 total [rspec-mocks-2.8.0/lib]
2357 total [rspec-rails-2.8.1/lib]

And on minitest:

5180 total [minitest-2.11.2]
 428 total [minitest-rails-0.0.1]

It's less than half the size! What I'd really like to see is a speed comparision of a reasonably large test suite migrated from one to the other.

Useful links

• Matt Sears' minitest quick-reference
• The "MiniTest with Rails" Railscast video (I'm not a subscriber so haven't watched it, but Ryan probably does a much better job of setting this stuff up than I have)
• minitest and minitest-rails on GitHub
• guard and guard-minitest
• database_cleaner