gears

Faster Unit Testing in a Legacy JRuby on Rails App

Recently I was asked to add some small features to a legacy Rails app. My development cycle was excruciatingly slow: the app runs on JRuby 1.5.6 and Rails 2.3, both of which are now eight years old. It took ages to run unit tests and other console commands. On average, JRuby and Rails together would take two minutes to load and begin running the test suite, which ran in about 45 seconds. This is eons for unit tests; you can only get up for coffee so many times a day.

Faster Unit Testing in a Legacy JRuby on Rails App
Putting it Together – Faster Unit Testing in a Legacy App. Photo by Hans-Peter Gauster on Unsplash

The argument for JRuby has always been that it’s fast once the Java Virtual Machine warms up. This works out fine for long-running programs like a Rails app. They can dedicate a certain amount of initial time to “warming up.” It also means that shorter-running commands, like console commands and unit tests, have to wait for the JVM to boot and load the app.

Upgrading or doing development with a different runtime wasn’t an option for a variety of reasons. Because of this, the biggest development obstacle wasn’t the codebase itself, but just the slow cycle time in development and test mode. I tried all the usual strategies to improve startup time:

  • I couldn’t get a working installation of the JVM with a client mode.
  • Passing other Java flags yielded limited performance improvements to JRuby.
  • Spring and its equivalents are not compatible. (Not much is compatible with this app.)

A Way Around the Problem

What I needed was a way to keep the JVM and the app loaded so that I didn’t incur the same startup penalty every time I wanted to run a unit test. I was puttering around in the Rails console one day when it occurred to me: the console was the answer. All you need to do is load the Rails console with the test environment. Unit tests are just code and can be executed in the console like any other code. To do this, I load a test runner available in the standard library in JRuby 1.5.6; your mileage may vary depending on what version of Ruby you have and what test library you use.

1
2
3
> require 'test/unit/ui/console/testrunner'
> load 'tests/some_controller_test.rb'
> Test::Unit::UI::Console::TestRunner.run SomeControllerTest

This works: TestRunner#run will run every test case in the SomeControllerTest class and give output.

It’s not very ergonomic, though, for a few reasons. First, all that typing (or cut-and-paste) is a pain. This problem is easy to solve. You can write a few helper methods to include globally when you load the console with the test environment. In my case, this meant an instance of a class that made a list of test files, loaded them in the console, and knew how to run tests from simple commands like:

1
> runner.run_test 'users_controller'

You can find the complete example code here.

The second problem is that every time we make a change to the test file, we have to reload it. I use load instead of require for this because require will not reload a file twice. Calling the global Rails method reload! will reload all the application code but not the tests. (It also causes a ton of Java errors in my case, which is a whole other issue.)

I found I kept forgetting to reload the right file at the right time. To solve this, I wrote a tiny hot reloader using TCP ports and a Unix file watcher called entr. I load the Rails console with a TCP server running in its own thread. When a file changes, the file watcher pings the right TCP port with the name of the changed file, and the server executes code to reload it in my console session. My implementation is very specific to the setup for this app, but I’ve outlined a vanilla demo here.

Putting it Together

With this setup, I can run individual test files from the console, make changes, and rerun the tests.

Assuming I have a UsersController and corresponding tests, I can edit the controller and confirm I didn’t break anything in my test console:

1
2
3
4
5
6
7
8
9
10
# the controller has reloaded in the background after I saved my change
> runner.run_test 'users_controller'
Loaded suite UsersControllerTest
Starter
test_i_didnt_break_anything(UsersController)

Finished in 0.762 second.

1 tests, 1 assertions, 0 failures, 0 errors
>>

It spits the test output into the console and then returns me to a prompt when it’s done.

The speedup is impressive: I can run an individual test file in a couple seconds, instead of the full two minutes it took before. This method has drawbacks, though.

Weird Things that Happen

There are a whole bunch of gotchas, small and large, to running tests in the console this way:

  • Deleted methods stick around. So do Rails model validations and callbacks you remove.
  • Like in the development environment, you don’t want Rails caching your application code, so make sure to set cache_classes to false for your test environment configuration. Otherwise, your views won’t reload, even if you manually reload controller classes.
  • You have to be very careful mocking, stubbing, and using setup and teardown for tests. Your Ruby process doesn’t die at the end of a test run, so it remembers any global changes you’ve made.

I found that despite my best efforts, something would eventually get corrupted and I’d have to reboot the console (I usually blame ghosts in the ObjectSpace).

Coda

Despite the drawbacks, this gives me an hour or two of low-friction development against quick unit tests, which is a happier place than I was before. It also forced me to work more deliberately and really think about the way code is loaded in a Rails app. Sometimes when you can’t work through a problem, there’s a way to work around it.