Book Review: Performance Testing- A Practical Guide and Approach

It’s book review time once again, and this month I read “Performance Testing- A Practical Guide and Approach” by Albert Witteveen. I’ve been looking for a resource on performance testing for a long time, because I’ve found that most of the articles and presentations on performance testing either assume a lot of prior knowledge, or focus on using a specific tool without explaining the reasoning behind what is being tested.

This book was definitely what I needed! It explains very clearly why we should be doing performance testing, what kinds of tests we should run, how we should set the tests up, how to run them, and how to report on the results.

Here are some of the things I learned in this book:

Performance testing simply means testing a system to see if it performs its functions within given constraints such as processing time and throughput rate.

Load testing is measuring the performance of an application when we apply a load that is the same as what we would expect in the production environment.

Stress testing is finding out at what point an application will break when we apply an increasing load, and determining what will happen when it breaks.

Endurance testing is about testing with a load over an extended period of time. This can be helpful in discovering problems such as memory leaks.

Virtual users refers to the number of simulated users we are using to test the system.

Performance tests need to be planned thoughtfully; it’s not just a matter of throwing load on every web page or API call. The tester needs to think about where the potential risks of load are and design tests to investigate those risks.

How to create a load test:

• Generally load tests are created by recording an activity in order to create a test script. Next, you’ll need to add a data set for use in testing.

• You should run your load test first with just one user. You’ll need to build in some kind of check in your script to make sure that the script is working as you are expecting it to. For example, if you are load testing the login page, you’ll want to see that the user was able to log in successfully instead of getting an error message.

• Once your script is working well with one user, try it with three users, and make sure that it’s using unique values from your test data instead of using the same values three times.

• When you have validated that your script is working correctly, you can execute your load test by adding the number of virtual users and ramp-up time that are appropriate to what you would expect your application to be able to handle in production.

It’s very important to monitor things such as CPU usage, memory usage, database activities, and network activity when you are running a load test. Just measuring response times isn’t going to give you the whole picture.

It’s also important to know what kind of queuing your application is using so you can locate bottlenecks in performance. The author uses an easy to understand analogy with a supermarket:

• A small market with just one checkout lane is like a system with a single CPU. Every customer has to go through this queue, and it’s first come, first served.

• A larger market with more than one checkout lane is like a system with multiple web servers. Every customer (or in the case of the web servers, the load balancer) picks one checkout lane, and waits to go through that lane.

• A deli where the customer takes a number and waits their turn is like Web server software where multiple workers can process the request. The customer waits their turn and can be picked up by any one of the worker processes.

Load testing tools themselves generate load when they are running! For that reason, it’s best to keep test scripts simple.

This is just a small sampling of what I learned from this book. It’s a short book, but filled with great explanations and advice. One thing worth mentioning is that there are a number of grammatical errors in the book, and even a few chapters that are missing the last words in the last sentence. It makes reading the book a little slower, since the reader sometimes has to guess at what was meant.

But in spite of these issues, it’s a great book for getting started with performance testing! I recommend it to anyone who would like to understand performance testing or run load tests on their application.