Snapshot testing is an extremely fast way to add regression testing to an existing project. You simply take some example inputs and then snapshot the resulting outputs. From then on, you can have a high degree of confidence that any changes you make have not affected backwards compatibility (as this would have been detected as a change in a snapshot).

However there are many pitfalls you can run into as I found when writing cupaloy, a snapshot testing library for Go.

Non-deterministic snapshots

The most obvious problem you run into is that the output you’re snapshotting has to be completely deterministic for this to work.

Take for example this CLI output from Hugo:

Built site for language en:
1 of 1 draft rendered
1 regular pages created
8 other pages created
1 paginator pages created
total in 10 ms

Here we have a line reporting how long the command took. This is a useful feature to the user but will make snapshot testing practically impossible: your tests are likely going to fail unless run on exactly the same machine used to create the snapshot.

Instead we’ll need to remove this non-deterministic output:

A similar problem will arise if you timestamp your log outputs, again either add an option to disable this or use something to set the clock at a fixed time.

Diffable snapshots

Snapshots are only useful if they help you identify what has changed about your output. It’s no use snapshotting an opaque blob of bytes.

For text output this means you want your snapshots formatted nicely so that your snapshot library can easily show you a diff of the changes.

For snapshotting more complex data structures you want to ensure that they’re pretty-printed in order to make comparing changes easy.

Updating snapshots must fail your tests

Most importantly: updating your snapshots must fail that test run!

It’s far too easy for UPDATE_SNAPSHOTS=true to sneak into your environment variables in CI and from then on, all your snapshot tests will pass regardless of what has changed.

To prevent this you can: