Your browser doesn’t support some of the latest CSS features used to build this site.

Read more about my usage of new CSS features.

Automatic visual regression testing has incalculable value - whether baked into a new build from the ground up or being retrofitted to legacy projects.

I find automated testing most useful when refactoring code I’ve inherited – having pages or components tested gives me the confidence to incessantly clean up CSS without having to worry too much about the repercussions.

I can produce a set of screenshots of how a component should look, then refactor like mad – delete things I think aren’t needed, tidy up the code and programmatically compare test screenshots as I’m working away.

How to?

So, here’s a quick guide to implementing visual regression testing using BackstopJS. Note: this example will be using Node and Gulp.

Prerequisites

Along with Node and Gulp installed, you will need Phantom and CasperJS installed globally on your machine:

npm install phantomjs casperjs -g

Installation

Next install BackstopJS and save it to your package.json in your development dependencies with npm install backstopjs --save-dev.

At this point Backstop requires you do something a little dodgy…at first it felt odd going into 3rd-party managed code and messing around, but you have to go into the node_modules directory and into backstopjs to install its dependencies locally:

cd node_modules/backstopjs
npm install
cd ../../

A nice tip here is to add these commands as a bash script, which can be called with NPM's built-in postinstall hook - see an example package file and example postinstall script.

Introducing gulp-chug

gulp-chug allows you to run tasks from other gulpfiles from your own.

In order to access the tasks set up in Backstop’s Gulp file without changing into the directory every time you’ll have to use this plugin.

There are some questions and issues around doing this, so at the moment I’m looking into a way to require the Gulp file directly and do this in a neater way but for now, chugging is fine.

npm install gulp-chug --save-dev

Aliasing tasks

Once you have Gulp chug in your devDependencies you can alias Backstop’s tasks to your own Gulp file:

var chug = require('gulp-chug');

gulp.task('genConfig', function () {
  return gulp.src('node_modules/backstopjs/gulpfile.js')
    .pipe(chug({
    tasks: 'genConfig'
  }));
});

gulp.task('reference', function () {
  return gulp.src('node_modules/backstopjs/gulpfile.js')
  .pipe(chug({
    tasks: 'reference'
  }));
});

gulp.task('test', function () {
  return gulp.src('node_modules/backstopjs/gulpfile.js')
  .pipe(chug({
  tasks: 'test'
  }));
});

Generating a configuration file

Run gulp genConfig and you’ll produce a JSON file similar to this one. Modify with your project’s details, set some configuration for your first scenario and you’re ready to go.

"scenarios": [
  {
    "label": "Homepage",
    "url": "http://codecomputerlove.com",
    "selectors": [
    "body",
    ".site-header",
    ".article",
    ".site-footer"
    ],
    "misMatchThreshold" : 1
  }
]

Generating base screenshots

gulp reference creates reference screenshots as your baseline — an approved set of passing tests. You won’t need to do this very often, usually when you initially set up the tests and create new ones.

Reference screenshots

Do some coding!

.awesome {
  background: #FF69B4;
}

…I’ll wait…

Test and compare

Run the tests again:

gulp test

This will open up a server that shows you screenshots of which tests have passed and if any have failed. Adjusting the misMatchThreshold can be tricky, but play around and get it right for your project.

A failing test screenshot using BackstopJS
A failing test after making some changes to the Code website.
A passing test screenshot using BackstopJS
My tests pass!

Discussion points

Where in an automated build pipeline it should go

I’m currently figuring out the best point during development to run these tests — whether manually as we’re making changes, as a git pre-commit hook or on a build server as part of the build pipeline.

Who should be running these tests

  • Front end developers?
  • All developers?
  • Internal test team?
  • Client test team?!

Any suggestions? Please let me know, as it’s still up for debate in our development team…


TL;DR

Automated visual regression testing is a powerful tool.

It gives me confidence in refactoring legacy code, comfort that a component I’ve built is more robust when making changes to the rest of the website and also lets developers who aren’t as acquainted with a project (or even front end development) confidence to make changes without negatively affecting the UI of a site. Implementing this in just one of my team’s projects has already captured errors and sped up our development process.

This post came out of a regression testing workshop I’ve been running for various development teams and disciplines. The example project can be found on GitHub. Have a play and let me know what you think.