You are currently offline, so some features of this site may not work fully.

Your browser doesn’t support some of the latest CSS features used to build this site.

Read more about my usage of new CSS features.

My slides and transcript(ish) from Front End North 2016

Welcome!

Hi everyone, thanks for coming to Front End North and for listening to me today.

I’m Dan, I’m a senior front-end developer at Code in Manchester and today I’ll be giving you a whistle stop tour of visual regression testing.

As it’s a going to be a brief talk, I’ve gone full-on opinionated on this subject and this is all based this on my own experience. There may be a lot of talking points out of the back of it so catch up with me later on if you want to chat some more about it.

In saying that, I hope you learn something from this that you can go back to work with and apply it right away.


So what is this magic?

Regression testing is a method of software testing that checks functionality that already exists against new changes that have been made to a codebase (thanks wikipedia), to catch (yes you guessed it) regressions!

More specifically to front-end development my talk will focus on visual regression testing — which looks at your user interface and shows any breakages when your HTML CSS or JavaScript has been changed.

…and why use it?

Because CSS is hard

Well, that’s not strictly true. it’s easy — fairly easy to learn, pretty easy to write, but very hard to maintain.

Working on a large CSS codebase, especially with other developers (who may not have expertise in front end development), is hard to maintain quality over time.

The cascading nature of CSS means that it is extremely brittle, and changing one bit of global CSS over here will very easily affect multiple components over there — some that might not even be on the page you’re building.

…and testing is boring!

So we’ve established that CSS can be tricky, and testing it is boring! Sorry if there are any testers in the room…but as developers we don’t want to be manually testing every single page and component of a website build each time we make a change — even QA engineers must find that boring!

Those who know me know that I love automation — taking the monotony out of these repetitive tasks like manually regression testing every page on a website, every component.

So automating things like visual testing takes the pain out of this boring, laborious process — so you can focus on your day to day work, the bit you’re good at, the bit you enjoy!

When to do it?

If you’re anything like me, you love refactoring CSS. But before I do this I’ll make sure I implement regression tests for all major pages and components, to give me peace of mind.

For new, greenfield projects:

  • Implement a test runner and test suite before starting components
  • Test all components as they get added to a style guide

Retrofitting to legacy projects:

  • When you inherit a codebase
  • Before you incessantly refactor any HTML/CSS
  • Before CSS re-architecture
  • Before a rebrand — (almost as if implementing on a greenfield project)
  • To give peace of mind when other developers work on to the project (as it’s legacy this has more chance of happening)

Where to use it within your project?

The best place in a project from my experience to implement visual regression testing is on a style guide or static component library for a website.

As a lot of us probably work with content managed websites, and dynamic content from a CMS can cause “breakages” when running regression tests. The aren’t smart enough to tell that it is content that has changed. Because it’s an image diff it can’t tell whether adding more text from the CMS is an actual breakage.

So test with static content.


The big one — how to use it.

I’ve used a few different software solutions for VRT all of which have their own benefits, but I’m going to focus BackstopJS for this talk — it’s one I’ve used more regularly, it’s easy to set up and has a great in-browser report for showing errors.

Prerequisites

You first need Phantom and CasperJS on your machine, then install Backstop from our good friend, npm:

npm install phantomjs casperjs -g
npm install backstopjs --D

Generate a config file

For this you have to run commands from Backstop’s npm scripts and gulpfile in the node_modules directory, so I use npm explore to delve into the dependencies. Explore lets you go into your node modules and utilise scripts and commands from within the dependency itself — a pretty useful thing that I discovered recently.

So first off you need to generate a config file using the following command:

npm explore -- backstopjs npm run genConfig

— in which you can add the DOM selectors of the components you want to test, something like this:

"scenarios": [
 {
   "label": "Styleguide",
   "url": "http://localhost:3000/styleguide",
   "selectors": [
     "body",
     ".site-header",
     ".article",
     ".site-footer"
   ]
 }
]
A stripped down copy of backstop.json

Generate reference screenshots

Run the reference command from Backstop to generate your baseline screenshots. These will be the valid screenshots to test against, so make sure they look good and correct.

npm explore -- backstopjs npm run reference

Here’s some I made earlier from a well known conference website…

So you generate these screenshots and test against this baseline of acceptance as you make changes to your project.

Break your CSS

Now time to make some changes to your code, and inevitably break it.

A looping animation of changing a CSS value and showing that all colours on a website have changed from yellow to blue.

This is me messing around with the Front End North site — you can see I’m changing all of the brand yellow to blue — this is an extreme example I know but you could be making legitimate changes, breaking things without realising.

Test

Now run the test command, again exploring the Backstop node module:

npm explore -- backstopjs npm run test
This is the command line interface running the tests after you’ve made some breaking changes.
Here’s a screenshot of the report you get when tests have failed.
Here’s the image diff showing that things have broken.

So you’ve seen that something’s broken, you go back and identify what change has caused the regression, fix and re-test.

Screenshots of a command line and in-browser test report
Now you can see that your tests pass, and your report is all nice and green!

Bonus automation

Now here’s some bonus tips for you.

I’m a lazy developer, and we all like to type as little as possible, so here I’ve added the previous commands to my npm scripts in my package file, to make it even easier for us. This can also be useful when running tests on a build server or with a git hook.

"scripts": {
  "bless": "npm explore backstopjs -- npm run bless",
 "openReport": "npm explore backstopjs -- npm run openReport",
 "reference": "npm explore backstopjs -- npm run reference",
 "test": "npm explore backstopjs -- npm run test"
}

Who should be using this?

I’ve shown you briefly how to use it, but who exactly should be running these visual regression tests?

Short answer: everyone! I believe that visual regression testing can help:

  • a front end developer — giving sanity checks that you know changes haven’t cascaded down to unknown areas of a website
  • a back end developer — giving comfort and peace of mind that they haven’t inadvertently changed an existing component
  • a designer — again, making changes to a CSS codebase that they don’t necessarily know may affect other parts of a website
  • a QA engineer — automating this kind of regression test will save time and allow them to carry on with more robust manual testing

In summary:

  • Visual regression testing is fairly straightforward to set up and get using right away
  • It will definitely give you some peace of mind that yourself and other developers aren’t breaking a website when building new features
  • It is a no-brainer to reducing the amount of regressions when many people are working on an existing codebase
  • Demonstrating to your team and stakeholders that you have a website fully tested can be a quick win to improve confidence

I hope some of you can go away and set this up for your projects and teams straight away!

Let me know how you get on via the article commments or on Twitter. Thanks.

Thanks! You rock.

Further reading

If you didn’t get it all here:


Please feel free to send me an email or Tweet me if you’d like me to give this talk at your event.