You are currently offline, so some features of this site may not work fully.

Your browser doesn’t support some of the latest CSS features used to build this site.

Read more about my usage of new CSS features.

I’ve been writing a lot of tasks to automate and track webpage speed and performance.

What is page speed performance?

What we generally mean by page speed performance is the time it takes the page to fully load. The time it takes for a web page to load is critical, as these days people, with our short short attention spans, will leave a website if it’s not fast enough, which can lead to loss of sales or goal completion etc.

Some stats from Web Performance Today show that after just 1 second users may start thinking about other tasks and mentally switch off, and after 10 seconds they will likely abandon the task they're on — browsing your website.

In an experiment by Amazon, they found that a page speed reduction of just 1 second could cost them 1% in sales. When Google attempted a similar experiment they found that by slowing search results loading by just 500ms traffic dropped by 20%.

As shown in these case studies and myriad more, time = money. A heck of a lot of money for these huge companies — but it can cost all websites visitors and engagement if it’s too slow to load.

There may also be SEO implications to having a slower loading website that could affect your search engine ranking, but who really knows about the great mystery of search engine ranking factors? Either way we know that having a fast performing website is essential.


How to measure it

The most popular services online to measure performance and get suggestions on how to speed it up are Webpage Test and Google’s PageSpeed Insights.

Using these services is straightforward: enter the URL of the page you’d like to test and hit the big button! The services then come back with speed results and suggestions on how to improve your performace.

A screenshot of Webpage Test in action
A screenshot of Google PageSpeed Insights in action

Alas, those who know me know that I hate boring repetitive tasks such as going to these websites, entering multiple URLs, waiting for results, analysing them, etc, etc… Enter our good friend, automation ♥.


How to automate it

Set up

This example uses Gulp to run the tasks. In your gulpfile.js import the Node modules webpagetest by Marcel Duran and psi by Addy Osmani, as well as gulp-util to help with logging messages out to the console.

import gutil from 'gulp-util'

import Webpagetest from 'webpagetest'
import psi from 'psi'

The Webpage Test Gulp task

I got the majority of this code example from a Gist by Adam Simpson and modified it for my own use, so credit where credit’s due there.

In this snippet replace config.performance.wptAPIKey with your generated WebPagetest API key, update config.performance.parameters.location with where you’d like the test to be ran from, and fill in the relevant details in the firstView objects with your performance criteria.

gulp.task('webpagetest', function () {
  var wpt = new Webpagetest('www.webpagetest.org', 'config.performance.wptAPIKey')

  var parameters = {
    'disableHTTPHeaders': true,
    'private': true,
    'video': false,
    'location': config.performance.parameters.location,
    'login': 'admin',
    'password': 'password'
  }
  var testSpecs = {
    runs: {
      1: {
        firstView: {
          SpeedIndex: config.performance.speedIndex
        }
      }
    },
    median: {
      firstView: {
        bytesIn: config.performance.bytesIn,
        visualComplete: config.performance.visualComplete
      }
    }
  }

  return wpt.runTest(config.performance.urls[0], parameters, function (err, data) {
    var testID = data.data.testId
    var checkStatus = function () {
      return wpt.getTestStatus(testID, function (err, data) {
        gutil.log(gutil.colors.blue(`Status for ${testID}: ${data.data.statusText}`))

        if (!data.data.completeTime) {
          return setTimeout(checkStatus, 5000)
        } else {
          return wpt.getTestResults(testID, {
            specs: testSpecs
          }, function (err, data) {
            gutil.log(gutil.colors.blue(`Finished test at http://www.webpagetest.org/result/${testID}/`))

            if (err > 0) {
              return process.exit(1)
            }
          })
        }
      })
    }
    return checkStatus()
  })
})

Within the task, we’re using gutil.log( ) and gutil.colors( ) to log out coloured message to your console during the process.

This task can sometimes take a long time to complete depending on how big a queue your test is put in on WebPagetest, so the next steps I’m going to take with these performance tests is to look into building a private instance of WebPagetest.

At the moment I’ve not got this task running on the CI build due to the time implications, but will do so in the future.

The mobile PageSpeed Insights task

Using the psi node module, this task hits the PageSpeed Insights API and comes back with the data that you’d get from using the online UI. This means that we can automate the process and hit that service each build, and eventually build reports over time of the page speed data that comes back.

I’ve adapted the implementation on the example project to take multiple URLs from the configuration file, and log out some pretty green or angry red messages on a successful or failed perf test.

gulp.task('psi:mobile', function () {
  config.performance.urls.forEach(function (url) {
    return psi(url, {
      nokey: 'true',
      strategy: 'mobile'
    }).then(function (data) {
      gutil.log(gutil.colors.blue(`Testing ${url} on mobile`))

      let speedScore = data.ruleGroups.SPEED.score
      let color = returnColour(speedScore)
      gutil.log(gutil.colors[color](`Speed score: ${speedScore}`))

      let usabilityScore = data.ruleGroups.USABILITY.score
      color = returnColour(usabilityScore)
      gutil.log(gutil.colors[color](`Usability score: ${usabilityScore}`))

      if (speedScore < 60 || usabilityScore < 60) {
        gutil.log(gutil.colors.red(`You seriously need to do some perf improvements…`))
        process.exit(1)
      }
    })
  })
})

A short demo of the task (I can’t make it go red - you’ll have to implement it and test some slow sites to see that!)

The desktop PageSpeed Insights task

The desktop task is pretty similar, other than the data that you’ll receive back from PSI.

At this point it’s worth noting that an API key is recommended for frequent use such as running this every time your project builds. You can read more about getting a PageSpeed Insights API key on the PSI getting started guide.

gulp.task('psi:desktop', function () {
    config.performance.urls.forEach(function (url) {
      return psi(config.performance.url, {
        nokey: 'true',
        strategy: 'desktop'
      }).then(function (data) {
        gutil.log(gutil.colors.blue(`Testing ${url} on desktop`))

        let score = data.ruleGroups.SPEED.score
        let color = returnColour(score)

        gutil.log(gutil.colors[color](`Speed score: ${score}`))

        if (score < 60) {
          gutil.log(gutil.colors.red(`You seriously need to do some perf improvements…`))
          process.exit(1)
        }
      })
    })
  })
}

A further addition to the example demo I’ve added is to exit the process when a test fails — with process.exit(1) you can error out and fail a build, warning you that you were about to deploy a change that would’ve made your website unreasonably slow.

How to record your stats

Finally — for this blog post anyway — I’ll quickly introduce the idea of writing these stats to JSON, so that they can be used to draw graphs of web page performance over time.

I started with a JSON file with a structure like this:

{
  "psi": {
    "mobile": {

    },
    "desktop": {

    }
  }
}

The code for this is fairly easy to implement, reading the JSON file and updating the relevant objects with a newly created timestamp, the strategy (mobile/desktop) and the URL being test and page speed result of it.

const filename = './reports/performance/performance.json'
const timestamp = '_' + new Date().valueOf()

function updateStats (strategy, timestamp, url, stats) {
  fs.readJson(filename, function (err, currentStats) {
    if (err !== null) {
      gutil.log(gutil.colors.red(err))
    } else {
      let newStats = currentStats
      newStats.psi[strategy][timestamp] = newStats.psi[strategy][timestamp] || []

      let obj = {}
      obj[url] = stats
      newStats.psi[strategy][timestamp].push(obj)

      fs.writeFile(filename, JSON.stringify(newStats, null, 2), function (err) {
        if (err !== null) {
          gutil.log(gutil.colors.red(err))
        } else {
          gutil.log(gutil.colors.green(`Success! ${filename} written.`))
        }
      })
    }
  })
}

This outputs something like the following:

{
  "psi": {
    "mobile": {
      "_1462473126762": [
        {
          "http://danielfurze.co.uk": {
            "speedScore": 96,
            "usabilityScore": 97
          }
        }
      ]
    },
    "desktop": {
      "_1462473126762": [
        {
          "http://danielfurze.co.uk": {
            "speedScore": 95
          }
        }
      ]
    }
  }
}

Next steps…

Showing stats over time

Next up I’m going to work on publishing this file out and exposing the JSON as an API to hook into for graph building, to track these stats over time and see trends.

Extend the task

It’d be good if the task could take multiple arguments from a config file — location, connection speed etc… for WebPagetest, and also utilise some more of the suggestions that come back from PageSpeed Insights.

I’ve already started extracting this code out to implement in client projects at work, and at some point I’ll extract the rest of the code from my website out into an open source project, so help out if you can!