Testing and JavaScript: Test Frameworks


8 November 2012, by

JavaScript is something that turns up pretty much everywhere, and it can be a tricky beastie; there’s no static typing, a long enough list of gotchas to let you hang yourself, and by and large it doesn’t leave you with much in the way of useful logs and stacktraces to help save the day. This makes testing important!

Unfortunately, that’s easier said than done. JavaScript code is very easy to write in a way that’s not very testable, and it just doesn’t have as much of a testing culture around it as say, Java. It means there’s not so much of a consensus on tools, and it’s somewhat difficult finding sensible ones for standard testing practices. Fortunately, I’ve made some lists! This is the first in a three part series: test frameworks.

QUnit

QUnit was written by the JQuery gang, to test JQuery while they built it (although note that it doesn’t actually have any dependency on JQuery nowadays), and is broadly based on the standard XUnit (JUnit, NUnit, PyUnit, etc.) conventions. It’s fairly popular, and provides a simple but effective framework within which to structure your code and manage your testing. This includes a basic set of assertions (ok, equal, deepEqual, strictEqual, expect and throws) and an almost uncomfortably simple module system: you make a call to module(name), and then all your tests are in that module until the next call to module(name). Setup and teardown methods can be attached to the module, and will get run for all tests within that module. This gives test modules that look like this:


module('Addition tests', {
  setup : function() {
    ok(true);
  }
});
test('Test adding numbers', function() {
  equals(1 + 1, 2);
});

In addition to the standard testing tools, it also lets you run tests with various extra controls to check for leaking globals (by adding ?noglobals to the url) to help you write well isolated code and tests, controls to write async tests so you can test callback-based control flow (by adding a ‘stop()’ call to the end of the test itself, and calling ‘start()’ when the callbacks should all be done), and provides a #qunit-fixture div within which you can put HTML elements to test some of your DOM manipulation code against, which gets cleanly reset between tests.

Since this is vanilla JavaScript, it does need to be run in a web page by a JavaScript engine, which in practice means a browser (running these without doing it manually in a browser is a topic for another post). This means for each set of tests, you need an html page built from a standard QUnit template that includes the QUnit JS library, your code, and then all of your test js scripts. On the other hand, this also means the output from the tests is a delightfully pretty little page, filled with info on each test execution, including the messages from assertions run, details of failures, and buttons to rerun individual tests, from which you can also use firebug or the chrome developer tools to debug and fix problems with them.

Jasmine

Jasmine is based on RSpec, a BDD testing framework for Ruby. In practice, this means a far stronger focus on having tests that clearly define their purpose and meaning from a higher-level perspective, giving tests that look more like this:

describe("addition", {
beforeEach(function() {
expect(true).toEqual(true);
});

it("should add numbers together", function() {
expect(1 + 1).toEqual(2);
});
});

As you might’ve noticed this moves away from standard XUnit conventions not only in structure, but also in assertion format: everything is built on chains of matchers following an expect() call, allowing things like expect(x).not.toContain(“hello world”). Setup and teardown are manged by including beforeEach(function) and afterEach(function) methods within the describe function.

Jasmine also includes some test double functionality, allowing you to build spies around JavaScript objects for testing to stub some of their methods or track their usage. How to do mocking in JavaScript is another post entirely, but having it tied properly into the framework like this definitely has value, letting you make asserts like expect(foo.bar).not.toHaveBeenCalledWith(123). Notably, it also lets you mock the JavaScript clock, thereby providing a asynchronous testing solution: you call jasmine.clock.useMock(), and can then call jasmine.Clock.tick(100) to synchronously call the callbacks that would be called after that much time. Jasmine also provides asynchronous testing support in a similar style as QUnit, here by redefining your tests as runs() blocks separated by waitFor() blocks, which test to ensure the asynchronous code has completed before continuing.

As with QUnit, Jasmine requires an HTML page in which to live, and a browser with which to run the tests, meaning you again get HTML output. This result follows in the BDD philosophy of the framework, so if you name your tests sensibly you’re left with clear explanations of the working and failing behaviour of the system. It’s far sparser and less developer-focused than the QUnit output, instead aiming to provide accurate documentation of the functionality of the system, in a format you could helpfully show to a product owner or similar.

JS-Test-Driver

JS-Test-Driver is again inspired by XUnit, but has taken a far more direct approach to the translation. To my mind this doesn’t produce nearly as clean or idiomatic a result as either of the other options, but it might well be easier to learn for people more familiar with other languages conventions than JavaScript’s, as it stays as closely tied as possible to the object model used everywhere else. Or people might disagree! An example:

<pre>exampleTestCase = TestCase("Addition Tests");

exampleTestCase.prototype.setUp = function() {
  assertTrue(true);
}

exampleTestCase.prototype.testAddingNumbers = function() {
  assertEquals(2, 1 + 1);
}

As shown, they’ve made effort to exactly map the relevant assertions provided by JUnit and NUnit to their JavaScript counterparts, and there’s a long list of other assertion options available in this vein.

Again, asynchronous testing can be done too: test cases can be instead created as AsyncTestCase(), and test functions can then take a queue argument and append functions to this queue to have them run sequentially after the test is completed. Each function in the queue can takes a callback list as an argument. If it adds any callback methods to this list then instrumented versions will be returned, to be run instead, and the next element in the queue (or the end of the test) will not occur until every instrumented callback has finished, or 30 seconds passes. Hmm. This is fairly definitively more confusing and messier than the other two models for this, but is arguably more powerful.

On top of all this, JS-Test-Driver is also a more ambitious project than the two frameworks described above: it provides a test-driver (as the name might suggest) with which to run your tests, acting as much more of an all-in-one solution, and in addition to this it can also produce code coverage statistics, all by itself. A full discussion of the ways you can drive these tests automatically and get this kind of data out isn’t really in the scope of this post, but it’s non-trivial, and having a driver integrated with test framework and ready to go with minimal work is pretty neat.

Notably JS-Test-Driver also has adapters to allow you to run various other frameworks on top of it, including QUnit and Jasmine. These appear to be implemented by just defining the API of each other module in terms of JS-Test-Driver’s, which means they don’t fully support some trickier features that can’t easily be translated (the QUnit adapter won’t do asynchronous tests, or anything DOM related), and they have various annoyances and compatibility issues (the Jasmine adapter can’t run single tests, and resets custom matchers you define between each test, unlike Jasmine), so this probably isn’t something to go for without good reason.

The also-rans

JsUnit – Used to be extremely popular, now no longer actively maintained, and the previous maintainers themselves have moved to Jasmine.

Buster.Js – A promising fully-featured framework and test runner, like JS-Test-Driver. Only recently arrived, currently in beta, and their homepage says ‘Many things are unstable at this point in time’. Worth watching though.

Screw.Unit – Another BDD framework very much in the same vein as Jasmine, but no longer actively maintained, and dependent on JQuery.

next article in series

Tags: , , , ,

Categories: Technical

«
»

One Response to “Testing and JavaScript: Test Frameworks”

  1. Some of the same contributors from JS-Test-Driver are now working on the ridiculously-named Testacular (http://vojtajina.github.com/testacular/), which addresses some of the JS-Test-Driver niggles mentioned above. It’s designed to be framework agnostic from the start, and should have better integration with Jasmine in particular.


Leave a Reply

* Mandatory fields


2 × six =

Submit Comment