Why Smoke Tests are Great

July 28, 2020

What is a smoke test?


A smoke tests is a test that makes sure your service isn't on fire by checking for smoke, hence the name smoke test. Your code can be broken and the smoke tests will pass and thats fine, as long as its not on fire. Smoke tests are always quick, you just need to look and see 'is there smoke'.

You can run smoke tests whenever you like but typically it makes sense to run them before you push code or before an expensive end-to-end test suite is ran, and if they fail then the end-to-end tests don't need to run.

Why would you want to use a smoke test?


Tests take time, and there is nothing worse than waiting a long time for tests and then they fail. While it is great that the tests failed and caught a bug, it sucks for a few reasons. The failing tests might be blocking the continuous integration pipeline which may mean that developers can't be productive. When the tests fail you might have already moved onto the next tasks you have to put down what you are doing and switch back.

Having a smoke test means that if your code is catastrophically broken, you get immediate feedback. Hopefully you find out before you push so the pipeline isn't blocked but if it does get to the pipe you find out immediately which means you didn't wait about and you can get your fix out straight away.

What does a smoke test look like?


Smoke tests can take a few different forms, but the main one would be requesting a resource from http server and checking it responds with the correct http status code.

There are a lot of great libraries to perform smoke tests but one that I like is shisha because it is so simple. You define a .smoke file in the root of your project that lists URLs and expected status code, it looks like:


Then all you have to do is run `shisha` and it will request the given URLs, and pass or fail depending on whether or not they match the given status code. So if for some reason your server couldn't start up or crashes your tests will fail.

You can even make different parts of this file configurable, for example you might want to change the top level domain you are running against. You do this like so:


Now you can run `shisha --domain1 github.com --domain2 twitter.com --path api` which can be configured by whatever build system you are using.

Summary


Save yourself and others time and pain by failing as fast as you can.

Tags: