Fix Your Automation Hourglass

You have no doubt heard about the Test Automation Pyramid, where the recommendation is that your code has many unit tests, a smaller number of integration tests, and an even smaller number of UI tests. You may have also heard of the anti-pattern called the Test Automation Ice Cream Cone: this is where the code has many UI tests, a smaller number of integration tests, and an even smaller number of unit tests.

But have you heard of the Test Automation Hourglass? As you can probably guess, this is a situation where the developers have written a lot of unit tests and the testers have written a lot of UI tests, but nobody has written any integration tests. This is often a symptom of having test automation silos, which I wrote about a few weeks ago. Usually the Test Automation Hourglass means that you have too many UI tests. And as I’ve written about before, having too many UI tests means slower test runs. Below are three ways to fix this issue.

Step One: Look at your entire test suite
The first step in correcting your Test Automation Hourglass is to meet with your developers and take a look at the entire test suite. What unit tests are you running? What UI tests are you running? Do you have any integration tests, and if so, what are they testing?

Identify any duplicated efforts. For example, do you have a UI test that’s already being covered by a unit test? For any duplicated tests, delete the test that’s closest to the UI.

Take a look at your UI tests. Are there any that are testing code logic that would be better tested at the unit or integration level? Make a list of those tests and begin to consider them as tech debt. You can address that tech debt by creating a replacement unit or integration test. When the new test is created, you can delete the old test.

Identify missing tests. Are you fully exercising your business logic? There may be many API tests you can write to fix this. Think about what you could do with Create, Read, Update, and Delete requests that you may not be currently testing. Make a list of these missing tests and add them to your product backlog.

Step Two: Identify ways you can run integration tests
There are many different types of integration tests, and many different ways to run them. Here are some examples:

Testing directly in the code: this is a good place for tests that just want to validate that it’s possible to connect to a datastore or to a dependency such as a third-party API.

Using unit test tools such as Jest, Mocha, or Jasmine: these tools are commonly used by developers for unit tests, but they are great for integration tests as well. Calls to create, update, or delete data can be tested here.

Using API test tools such as Postman: APIs can be tested directly in the code or through unit testing tools, but it’s especially easy to set up API tests with tools specifically designed for that purpose. API test tools typically come with some kind of command-line runner, which makes it easy to automate the tests. For example, Postman uses Newman to automate API test execution.

Step Three: Start converting your tests
Once you have a list of the tests that you need to convert or write, you can start working on them. If you have chosen a test tool that both developers and testers feel comfortable with, you can all work on your test backlog.

Don’t try to change everything at once! Set some goals for the sprint, the month, or the quarter. Order your test backlog by importance, just as you would for other tech debt. As you gradually move your tests from the UI level to the integration level, make sure to celebrate your successes! You’ll be rewarded with faster-running, more reliable tests.