Browser Level Testing - Martin Kleppmann
IE isn't so clever when working out the length of arrays with trailing commas:
var x = [1, ]; x.length 1 // Firefox 3.5 x.lenth = 2 // IE7
IE also gets a bit confused about what an HTML id really is. Given this HTML:
you see non standard behaviour in IE:
typeof(hello) = 'undefined' // FF 3.5 typeof(hello) == 'object' // IE7
In other words, if you want to guarantee that your application works in a specific browser you really need to test the behaviour in that browser. There are so many browsers in common use today that it needs automating, but how do you go about it?
Driving the browser with simulated events
Selenium is a good example of this technique. Selenium itself is loaded from localhost, while the site you're testing is often running on a remote host. This violates the browser security model, so you need to load the site that you're testing through a proxy.
- You can't test sites that redirect between different domains.
- You can't test things that occur outside of the DOM (pop-ups flash, etc.).
- It's slow (particularly in IE).
Browser specific automation APIs
As an alternative to simulating events, you can control browsers through their programmable APIs. It's a lot more work to implement, but it works a much better than event simulation. Projects taking this approach include WebDriver and Watir.
WebDriver is an open source project that is supported by Google. It has implemntations of browser specific APIs and supports IE, Firefox, Chrome and Opera (but not Safari). It also drives HTMLUnit and supports Celerity, which emulates the behaviour of a browser in a headless manner. As a result it's faster than testing in a real browser.
Watir supports IE, Firefox, Safari, Chrome and HTMLUnit.
Martin started a startup company that runs Selenium in the cloud, running on a bunch of virtual machines. He's come up with a DSL for describing tests called Go-Test.
Interestingly, the Selenium and WebDriver projects are joining forces. WebDriver will be used to control the browsers, while Selenium will provide the programming API. Martin thinks that Watir and Selenium should move towards providing front ends to WebDriver, to create a superb open source browser level testing framework.
Here's some example RSpec code for controlling WebDriver via Selenium:
Writing robust tests
The tests you can run with these frameworks do tend to be quite brittle -- you don't need to make many changes to your app before you can expect your tests to fail. You can mitigate the problem somewhat by writing flexible locators (or selectors). If you're not sure what a locator is, they're used within tests to identify tags in your HTML. For example:
id = searchInput name = search xpath = //form[@id='searchform']/input
Here are a couple of examples of how you can select a headline on a BBC article:
xpath = //td[@class=text']/div/a css = a.tsh
The xpath example is a little brittle; if the BBC stopped using tables to lay out this page then the <td> tag would disappear and a test that relied on the xpath locator would break. The CSS selector is (in this case) a much neater way of doing it, and would probably survive updates to the HTML (the
tsh class name has some semantic meaning, though I forget what Martin said it meant).
So there are 2 approaches that you can take when writing these front end tests:
- Disposable tests; created automatically with record/playback software. They might not survive for very long when your application gets refactored.
- Domain-specific abstractions; carefully crafted tests which are more effort to write, but which should be less effort to maintain.
Which approach is best? Which do you prefer? Martin would be interested in hearing your thoughts.
It was a great overview of the current state of browser testing, and (as a long term Selenium user) I'll definitely be having a look at WebDriver on Monday.
You can no doubt find out more at Martin's blog, or find him on Twitter as @martinkl.
More talks from Ruby Manor
- Data Visualisation with Ruby - Chris Lowis
- Denormalising Your Rails Application - Dan Lucraft
- gem that - James Adam
- Introduction to Rango - Jakub Šťastný
- Ruby and Cocoa - James Mead
- RubyGoLightly - Eleanor McHugh
- Secrets of the Standard Library - Paul Battley
- Short Order Ruby - Ben Griffiths
- Testing Third Party HTTP APIs - Richard Livsey and Bartosz Blimke
- The Joy of Painting with Ruby - Jason Cale
- Ruby Manor 2008 morning session and afternoon session