When to Automate

The first step in automation success is knowing when to automate and when not to automate.  Here are some guidelines:

  • DO automate repetitive tasks.

Recently I needed to add over 600 records to our application in order to fully test a feature.  Rather than add just a few records to partially test it, I created a simple test that would add as many records as I specified.  Another great thing to automate is a feature with a number of different permutations.

  • DON’T automate brand-new features.

Yes, you will probably want to automate the feature eventually, but not the instant it’s coded.  A good QA tester will spend a lot of time doing manual, exploratory testing to make sure that she fully understands the feature and knows its limitations.

  • DO automate basic smoke-level tests.

I like to think of smoke-level tests as tests that we would be really embarrassed by if they failed in the field.  One company where I worked had a search feature broken for weeks, and no one noticed because we hadn’t run a test on it.  Unfortunately, the bug was pushed out to production and seen by customers.  Automating these tests and running them at every build or early in the morning can help catch major problems quickly.

  • DON’T automate fragile features or works in progress.

There may be a feature of your company’s software that seems to break every time someone looks at it funny.  Automating tests of this feature practically guarantees that the tests will fail daily.  Similarly, a feature that is still in progress will have many changes that will cause you to make frequent code changes in your tests.  It’s better to keep testing the feature manually and reporting bugs on it until it becomes more stable.

  • DO automate things users will do every day.

What is the primary function of your software?  What is the typical path that a user will take when using your software?  This is the kind of thing that should be automated.   Rather than running through this manual path every morning at 9 AM, you can set your automated test to do it at 8 AM and you’ll know right away if there is a problem.  

  • DON’T automate weird edge cases.

There will always be bugs in software, but some will be more likely to be seen by users than others.  You may be fascinated by the bug that is caused by going to a specific sequence of pages, entering non-UTF-8 characters, and then clicking the Back button three times in a row, but since it’s very unlikely that an end-user will do this, it’s not worth your time to design an automated test for it.

Why Automate?

For many of you, your initial answer to the question “Why automate?” may be “Because management wants me to.”  This is, of course, a valid reason.  But after you have a few automated tests written, you will discover other, more important reasons:

  • It saves you time.

At first it will seem as if automation is a huge time-sink, because it will take several weeks to get your automation system up and running.  But once it’s in place, it will save you valuable testing time.  As I am writing this blog, all of my automated smoke tests are running.  Even without looking at our application, I will know if something went wrong with the build this morning.

  • It frees you from some repetitive tasks.

We have a search form in our application that has four editable fields: Last Name, Company, Email, and Phone Number.  In order to test this feature well, I would need to test with every possible permutation: just one field at a time, two fields at a time, three fields at a time, and finally four fields at a time.  This adds up to fifteen searches.  It’s rather tedious to do this every time I need to test the search function.  But my automated test is happy to run through these fifteen searches, as many times as I tell it to.

  • It helps you to think more critically about your testing.
When I’m manually testing, I tend to follow a written plan and add in a few ad hoc tests as well.  But when I’m writing an automated test, I need to think about why I’m running the test.  The test engine will not be exploring, so I’m going to need to tell it to do something very specific.  What are the minimum steps required to ensure that the feature works?  Will I need to vary those steps for broader coverage, and what will be the most efficient way to do that?

  • It helps you to understand your development team.
Before I started writing automated tests, I knew a bit about coding from my college courses, but I’d never had real hands-on experience.  Using an IDE and writing in Java helped me have more sympathy for what developers go through in order to code, and it also taught me how programs are structured and how the build process works.  This has helped me become a better tester.

The path to learning automation won’t be easy, but it will teach you important skills, streamline your testing process, help you understand the development life-cycle, and improve your thinking about your tests.

Welcome!

This blog is aimed at Software QA Engineers who know nothing about test automation.  I was just like you six months ago.  After much trial and error, I have successfully automated many tests, and I would like to share what I learned with you and hopefully save you some time and trouble!

In the next few weeks I will be covering:

  • Why automate?
  • When to use automated tests and when to use manual tests
  • Introduction to Java
  • Setting up Eclipse
  • Setting up JUnit
  • Introduction to WebDriver
  • Setting up Maven
  • Simple Maven commands
  • Setting up Sauce
  • Setting up Bamboo
  • Creating your Base Test
  • Introduction to JQuery
  • Troubleshooting navigation issues