Getting your pet to sit still for a picture is hard. They would much rather be scratching, sniffing or chewing up things they should not be chewing on. Standing up your app in a test environment before taking an image snapshot can similarly be a struggle, especially if the app is driven by many disparate data sources.

In this article we’re going to share one of several testing recipes we have adopted at American Express that uses jest-image-snapshot and Parrot to write image snapshot tests in a consistent and reliable way.

In order to explain the recipe, let’s create a sample app that answers a simple question.

Is it hot in Phoenix?

Our sample app queries the OpenWeatherMap API for the current temperature in Phoenix, AZ. Depending on the response, the app displays in one of three states:

Sample app states

Great! Now we just need to make sure that we guard against future regressions by writing some image snapshot tests.

Add image snapshots

For our image snapshot tests, we’ll use Puppeteer and jest-puppeteer-preset to drive a headless Chrome browser, along with jest-image-snapshot to assert on screenshots, in order to catch visual regressions. To run our sample app during the tests, we will add a jest-puppeteer.config.js to our project’s root.

module.exports = {
  server: {
    command: 'npm run start', // webpack-dev-server --config ./webpack.config.js
    port: 3000,
  },
}

Then we’ll use the global variable browser provided by jest-puppeteer-preset to access the headless browser and take a screenshot of our app.

it('is hot in Phoenix', async () => {
  // create a new browser tab 
  const page = await browser.newPage();

  // navigate to our running app
  await page.goto('http://localhost:3000', { waitUntil : 'networkidle0' });

  // take a screenshot
  const screenshot = await page.screenshot();

  // expect that the screenshot has not changed
  expect(screenshot).toMatchImageSnapshot();
});

After running our tests, we should now have a screenshot of our app stored in an __image_snapshots__ directory.

─── src
    ├── App.js
    ├── App.browser.test.js
    └── __image_snapshots__
        └── app-browser-test-js-is-hot-in-phoenix-1-snap.png

Image snapshot

Simple enough right?

A few days later we decide to refactor our sample app. Before committing our changes, we run our tests again to ensure that the functionality has not changed, and we realize that one of our image snapshots no longer matches, so the tests are failing.

FAIL  src/App.browser.test.js (5.241s)
  ✕ is hot in Phoenix (3688ms)

  ● is hot in Phoenix

    Expected image to match or be a close match to snapshot but was 78.97625000000001% different from snapshot (379086 differing pixels).
    See diff for details: /Users/user/dev/is-it-hot-in-phx/src/__image_snapshots__/__diff_output__/app-browser-test-js-is-hot-in-phoenix-1-diff.png

jest-image-snapshot creates a __diff_output__ folder containing the image diff from our stored snapshot. Let’s take a look at it and see why it is failing.

─── src
    ├── App.js
    ├── App.browser.test.js
    ├── __image_snapshots__
    │   └── app-browser-test-js-is-hot-in-phoenix-1-snap.png
    └── __diff_output__
    	└── app-browser-test-js-is-hot-in-phoenix-1-diff.png

Image snapshot diff

After examining the image diff and digging around to see why the tests failed, it hits us! Over the last few days the temperature in Phoenix has dropped, and that is causing our app to render in a different state during the tests. Since we are using a response from the live API, our tests are about as predictable as the weather. 🥁

Let’s make it hot in Phoenix

In order to make our tests more reliable, we’ll mock our API responses using Parrot. Parrot is a scenario-based mocking tool that will allow us to create several different data scenarios and easily switch between them. First, we’ll write a few scenarios that we can use in our tests.

In parrot.scenarios.js we define the scenarios:

const scenarios = {
  'is hot in phoenix': [
    {
      request: '/weather',
      response: { main: { temp: '97' } },
    },
  ],
  'is cool in Phoenix': [
    {
      request: '/weather',
      response: { main: { temp: '72' } },
    },
  ],
  'cannot fetch weather data': [
    {
      request: '/weather',
      response: {
        status: 500,
      },
    },
  ],
  'is a random temperature in Phoenix': [
    {
      request: '/weather',
      response: () => ({
        main: {
          temp: Math.floor(Math.random() * 110).toString(),
        },
      }),
    },
  ],
};

export default scenarios;

Next, we’ll update our jest-puppeteer.config.js to start our sample app and also an instance of parrot-server to serve up mock responses.

module.exports = {
  server: {
    command: 'npm run start:parrot', // concurrently --kill-others 'npm run start -- --env.mock' 'parrot-server --port 3001 --scenarios ./parrot.scenarios.js'
    port: 3000,
  },
}

Finally, we’ll switch to the correct data scenario before taking a screenshot of our app. To this, we’ll implement a helper function to set Parrot scenarios…

import fetch from 'node-fetch';

const setParrotScenario = async (scenario, parrotServerUrl) => {
  const requestBody = JSON.stringify({
    scenario,
  });
  const contentType = 'application/json';
  const response = await fetch(`${parrotServerUrl}/parrot/scenario`, {
    method: 'POST',
    headers: {
      Accept: contentType,
      'Content-Type': contentType,
    },
    body: requestBody,
  });
  if (!response.ok) { throw Error(`HTTP error ${response.status} for post to ${parrotServerUrl}/parrot/scenario with request body ${requestBody}`); }
  return response;
};

export default setParrotScenario;

…which we can then use within our tests to quickly and easily switch between Parrot scenarios.

import setParrotScenario from '../test-utils/set-parrot-scenario';

it('is hot in Phoenix', async () => {
  // make it hot in Phoenix
  await setParrotScenario('is hot in Phoenix', 'http://localhost:3001');  
    
  const page = await browser.newPage();
  await page.goto('http://localhost:3000', { waitUntil : 'networkidle0' });
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchImageSnapshot();
});

Now our tests should pass, and we will have consistent image snapshots, regardless of the weather!

Conclusion

This is a simplified and contrived example. However, at American Express we employ the same patterns and tools to test some of our most heavily trafficked applications and have found it gives us the confidence to move faster than we have ever moved before, without sacrificing on quality.

The ability to map Parrot scenarios to tests and switch between them during the tests is not only useful for visual regression tests, as shown in this article, but also for integration tests in general. Being able to create a fully controllable and deterministic test environment with the help of Parrot has been huge for us, and we hope it also can be useful to you as well as part of your testing tool belt.

For more details on jest-image-snapshot and Parrot, feel free to check out their documentation, as well as the code for our sample app.