Think Like a Tester

• Make sure you have completely read a feature’s Acceptance Criteria and all available notes and documents.  This can help prevent unnecessary and time-consuming misunderstandings.
• Ask questions if there is anything in the feature that you don’t understand.  Don’t make potentially incorrect assumptions.
• Document your work.  This is especially helpful when you have found an issue and the developer needs to know what browser you were using or what server you are pointing to.
• Check twice to make sure that you really have a bug.  Perhaps what you are seeing is a configuration problem, a connection problem, or simply user error.

• Learn the preferred communication style of your dev and use it.  For example, some developers like to hear about issues immediately, and testing and bug fixing become a collaboration.  Other developers prefer to hear right away only if the issues are big ones, and would rather have you document the smaller issues for a later conversation.
• Ask your dev to walk you through any confusing features.  He or she will be happy to explain things to you, because they know that any information they give you at the outset of testing will save misunderstandings later.
• Be kind when reporting issues.  Your dev has worked hard on the feature he or she has delivered, and we all know it’s no fun to have our work criticized.
• Give feedback in the form of a question.  This can soften the blow of finding a bug.  For example, “I noticed that when I clicked the Save button, I wasn’t taken to the next page.  Is this as designed?”
• Let your dev know what he or she can do to help you do your job more efficiently.  A good example of this is asking them to not assign an issue to you until it is actually in the test environment, so you won’t inadvertently start testing it before the code is there.

In my last two positions, I have been the sole Software Engineer in Test for a brand-new software platform.  This means that I have been able to set up an entire arsenal of automated tests for both the backend and the front-end.  This article will describe the suite of tools I used to test a web application.

API Tests:
First, I set up tests that checked that every API call worked correctly.  I did this through Runscope, an API testing tool I highly recommend.  I was fortunate enough to have engineers who set up Swagger, a framework that lists API calls in a really easy-to-read format.  I used the information from Swagger to set up my Runscope tests.  For testing against our development environment, I created tests for every GET, POST, PUT, and DELETE request.  I set this suite of tests to run once a day as a general health check for the development environment.  For testing against our production environment, I set up tests for every GET request that ran hourly.  This way we would be notified quickly if any of our GET requests stopped working.  Runscope is also good for monitoring the response times of API calls.

Backend Tests:
The backend of our platform was written in Java, and we developed a suite of integration tests using JUnit and AssertJ.  Generally the developers would create a few basic tests outlining the functionality of whatever new feature they had coded, and I would create the rest of the tests, using the strategy that I outlined in my previous post.

Front-End Tests:
The front-end of our platform was written in Angular, so I used Protractor to write a series of end-to-end tests.  The UI developers were responsible for writing unit tests, so I focused on testing typical user interactions.  For example, in my Contact-Info tests, I tested adding and editing a phone number, adding and editing an email address, and adding and editing a mailing address.  If it was possible to delete contact info in our UI, I would have added those tests to the suite as well.  I set up two suites of tests: a smoke suite and a full suite.  The smoke suite was for basic feature testing and was integrated into the build process, running whenever a developer pushed new code.  If they pushed a change that broke a feature, they would be notified immediately and the code would not be added to the build.  The full suite was set to run every test, and was configured in Jenkins to run overnight against the development environment using SauceLabs.  If anything happened to cause the tests to fail, I’d be notified by email immediately.

Since the UI also included uploaded images, I integrated Applitools visual testing into my Protractor tests.  Their platform is extremely easy to integrate and use.  I wrote tests to check that the image I added to a page appeared on the screen, and tests to check that if I changed an image, the new image would appear on the screen.

Load Tests:
In my previous position, I set up a series of load tests using SoapUI.  This tool is similar to Runscope in that it tests API calls, but it has the added functionality of being able to configure load testing.  I set up the following test scenarios: a Baseline test, where a small group of simulated users would run through a series of API requests; a Load test, where a small group of users would make a more frequent series of requests; a Stress test, where a large group of users would make a frequent series of requests; and a Soak test, where a small group of users would make a frequent series of requests for a long test time.  We were able to use this suite of tests to monitor response times, check for memory leaks, and test our load balance setup.

Every application is different, so you may find that you need a different set of tools for your test toolbox!  But I hope that this blog post gives you a jumping-off point to think about how to test your application thoroughly.

In this post, I’ll be switching gears from discussing automated tests with Webdriver to discussing integration tests that are included as part of an application’s back-end Java code. 

In my new Software Engineer in Test position, I have been writing integration tests to test every API call we have.  This means that I have dozens of test classes, and over a thousand possible individual tests.  Today I’m sharing with you the strategy I came up with for organizing the tests. 

I sorted out the tests into four test types: 

Resource Tests:  these are the “Happy Path” tests- the tests where you expect things to work correctly.  Examples include:  POST requests with valid parameters, GET requests where the correct object is returned,  PUT requests where the new parameters are valid , and DELETE requests where the correct object is deleted.

Security Tests:  this is where I put all the tests that should fail because the user is not logged in.  For every method that is tested in the Resource Tests, there should be a similar test where the method is called, but the user is not logged in.

Validation Tests:  this is where I put all the tests that should fail because a value being sent is invalid.  Examples include: sending an null value where a value is required, sending an empty value where a value is required, sending a value with invalid characters (such as letters in a numeric field), and sending a value with too many or too few characters.  It’s important to include tests for both Create and Update requests. 

Conflict Tests:  this is where I put tests that should fail because the request has been incorrectly formed.  Examples include: 
any POST, GET, PUT, or DELETE request where the required id is null
any POST, GET, PUT, or DELETE request where the required id is non-existent
any POST request where an id of the object about to be created is included
any PUT request where the id in the request does not match the id of the existing object

I created a package for each of these test types.  Then for each domain object, I created four test classes: one for each package.  For example, our code has a domain object called Phone.  Here are the test classes I created:

PhoneResourceTest- verifies that it is possible to add, get, edit, and delete a phone number

PhoneSecurityTest- verifies that it is not possible to add, get, edit, or delete a phone number if the user is not logged in

PhoneValidationTest- verifies that the phone number being added or edited is in a valid format (ten digits, no letters, etc.)

PhoneConflictTest- verifies that it is not possible to add or edit a phone number if the id of the associated contact is missing or incorrect, that it is not possible to add a phone number if there is a phone number id in the request, that it is not possible to edit a phone number if the phone number id in the request does not match the id of the phone number being edited, etc.

I hope this post will help you think about how to best organize your own integration tests!

Today we will tackle the most difficult ways to find an element: by css and xpath.  These are the most difficult because it’s so easy to miss a slash or a period in the description.  They are also the most difficult because they are so brittle- every time a developer makes a change to the webpage structure, the navigation path will change and your findElement command will no longer work.

However, finding an Element by css or xpath is often very needed, because developers often don’t take the time to give their elements easy and unique ids or names.  I would recommend using these strategies as a last resort to find the element you need, and I would try css first before resorting to xpath.

First, let’s find an element with css.  Take a look at the table that was described in the last blog post:

In order to find and use web elements using By.cssSelector or By.xpath, it is very helpful to understand the DOM.  The DOM (Document Object Model) is simply the interface that is used to interact with HTML and XML documents.  When a JavaScript program manipulates elements on a page, it finds them using the DOM.  If you already understand HTML, it will be very easy to understand how the DOM is organized.  Rather than explaining the differences between the DOM and HMTL source code, I’ll simply direct you to this very helpful webpage:  

https://css-tricks.com/dom.  

If you have little knowledge of HTML, here’s an example of how web elements are organized on a page:

<html>

<head>

<title>My Web Page</title>

</head>

<h1>My Web Page</h1>

<p>This is a paragraph</p>

<a href=”http://www.google.com” >Here’s a link to Google</a>

<table border=”2″>

<tr>

<td>Row 1, Column 1</td>

<td>Row 1, Column 2</td>

</tr>

<tr>

<td>Row 2, Column 1</td>

<td>Row 2, Column 2</td>

</tr>

</table>

</html>

If you’d like to see what this webpage would look like, you can simply copy and paste this html code into a notepad file and save it with the .html extension.  Then find the file in the folder where you’ve saved it and double-click on it.  It should open up as a webpage.  

Now let’s take a look at the various elements on the page.  Note that each element has a beginning and ending tag.  For example, the line that says “This is a paragraph” has a <p> tag to indicate the beginning of the paragraph and a </p> tag to indicate the end of the paragraph. Similarly, the title of the page has a beginning tag: <title> and an ending tag: </title>.  

Notice that elements can be nested within each other.  Look at the <table> tag, and then find the </table> tag several lines below it.  In between the tags, you will see row tags (<tr>) and table data tags (<td>).  Everything in between the <table> and </table> tags is part of the table.  Now look at the first <tr> tag and the first </tr> tag.  Notice that there are two pairs of <td></td> tags in between. Everything between the first <tr> tag and the first </tr> tag is a row of the table.  The <td></td> pairs in the row are elements of data in the row.

Now imagine that this data is organized into tree form:

If you were going to traverse the DOM in order to get at the data in Row 1, Column 1, you’d start by finding the <table> element, then by finding the first <row> element, then you’d find the first <data> element.  This is how we will use css selectors and the xpath to find elements in the next blog post.

If you’d like to find out more about HTML and CSS, I highly recommend the w3schools website.  

In my last post, I discussed how to find an element with Webdriver using the element id, class name, or name.  In this post, I will discuss how to find an element using link text, partial link text, or tag name.

Return to www.google.com, and look at the links on the bottom of the page.  Right-click on the link that says “Advertising” and select “Inspect Element”.  You should see something like this:
<a class=”_Gs” href=”//www.google.com/intl/en/ads/?fg=1″>Advertising</a>

The link text is what is found between the >reverse brackets<.  In this case, the link text is “Advertising”.

Return to myFirstTest in Eclipse and replace the findElement command in your test with:
driver.findElement(By.linkText(“Advertising”)).click();

This command tells Webdriver to:
1.  Find the element with the link text “Advertising”
2.  Click on the link

If you followed the instructions in the last post, you should still have a breakpoint set in your test. If not, right-click in the left margin of the “driver.quit();” command and select “Insert Breakpoint”. Then run the test by right-clicking on the test and selecting Debug As->JUnit Test.  The test should run and will stop just after it clicks on the Advertising link.  If the test has run correctly, you should now be on the Google Ads page.  Click on the green arrow to finish the test.  

Now we will find an element using a partial link text.  Return to www.google.com, right-click on the link at the bottom of the page that says “Business”, and select “Inspect Element”.  You should see something like this: 
<a class=”_Gs” href=”//www.google.com/services/?fg=1″>Business</a>
The link text is “Business”.  We will find this link using just a portion of the text: “ness”.  

Return to myFirstTest in Eclipse and replace the findElement command in your test with:
driver.findElement(By.partialLinkText(“ness”)).click();

Run the test by right-clicking on the test and selecting Debug As->JUnit Test.  The test should run and will stop just after it clicks on the Business link.  If the test has run correctly, you should now be on the Google Business Solutions page.  Click on the green arrow to finish the test.  

Now we will find an element using a tag name.  Go to www.pinterest.com, right-click on the button that says “Continue with Facebook” and select “Inspect Element”.  You should see something like this:
<button class=”Button Module btn hasText large rounded unAuthFacebookConnect registerLoginButton unauthHomeRegisterButton multiStepRedesign” type=”button”>    

Note that the first word that appears after the “<” is “button”.  This is the tag name of the element.

Return to myFirstTest in Eclipse and replace the String URL line with:
String URL = ‘http://www.pinterest.com’
Next, replace the findElement command in your test with:
driver.findElement(By.tagName(“button”)).click();

This command tells Webdriver to:
1.  Find the element with the tag name “button”
2.  Click on the button

Run the test by right-clicking on the test and selecting Debug As->JUnit Test.  The test should run and will stop just after clicking on the button.  If the test has run correctly, you will see a popup prompting you to log in with Facebook.  Click on the green arrow to finish the test.

You have successfully located a web element by using an link text, a partial link text, and a tag name! Before we proceed to finding a web element by css and xpath, it is first necessary to understand the DOM.  This will be the topic of my next blog post.  

The most useful method of Webdriver is the findElement() method.  It can also be the most frustrating:  you can see the element you want on your webpage, but it’s often difficult to figure out how to tell Webdriver to find it!

There are many different ways to find an element with findElement():

By.id
By.className
By.name
By.linkText
By.partialLinkText
By.tagName
By.cssSelector
By.xpath

I will include examples of these in this blog entry, and in the two entries following.  But first you need to know how to inspect an element in order to make the best choice about which By method to use.  Go to www.google.com, right-click in the search text field, and choose “Inspect Element”.  A new window will open up at the bottom of the screen, and you will see the element highlighted:

<input id=”lst-ib” class=”gsfi” type=”text” value=”” autocomplete=”off” name=”q” style=”border: medium none; padding: 0px; margin: 0px; height: auto…; width: 100%; background: url(‘” dir=”ltr” spellcheck=”false”></input>

Look at the id attribute:  id = “lst-ib”.  This is what we will use for our first findElement() exercise.
(Note that the id for this element changes periodically.  Substitute whatever you see listed as the element id in the steps below.)

Return to myFirstTest in Eclipse and add the following text below the driver.get(URL) command:
driver.findElement(By.id(“lst-ib”)).sendKeys(“Hello World”);

This command tells Webdriver to:
1.  Find the element with the id “lst-ib”
2.  Type “Hello World” into the element

You will also need to add the following to the import section of your test:
import org.openqa.selenium.By;

Insert a breakpoint into your test so that you can see if the commands were executed correctly.  To do this, right-click in the left margin of the “driver.quit();” command and select “Insert Breakpoint”.  Then to run the test, right-click on the test, and select Debug As->JUnit Test.  The test should run and will stop just after it has typed “Hello World” into the text field.  If this is what you see, then you have used findElement() correctly!  Click on the green arrow at the top of the screen to finish the test.

We now have an example of findElement(By.id()).  Now let’s try By.className.  Look back at the html that describes the text field.  You should see the class attribute:  class=”gsfi”.  (Note that the class name for this element changes periodically.  Substitute whatever you see listed as the class in the steps below.)Let’s change our test to find the element by className.  Replace the findElement command in your test with:
driver.findElement(By.className(“gsfi”)).sendKeys(“Hello World”);

Run the test in debug mode again, and you should again see that “Hello World” has been typed into the text field.  The result is the same, but this time the element was located by the class name instead of by the id! Click on the green arrow to finish the test.

Finally, let’s try to find the element by its name.  Look at the html again and see the attribute name=”q”. Replace the findElement command in your test with:
driver.findElement(By.name(“q”)).sendKeys(“Hello World”);

Run the test in debug mode again, and you should see that once again, “Hello World” has been typed into the text field.  Click on the green arrow to finish the test.

You have successfully located a web element by using an id, a class name, and a name!  The next blog entry will discuss finding elements by link text, partial link text, and tag name.

Webdriver is an API that enables testers to find and use web elements in their automated tests.  To get set up, you will need to find out what the latest version of Webdriver is.  Navigate to this link:  http://docs.seleniumhq.org/download and in the section called Selenium Client and Webdriver Language Bindings, find the Java entry and make a note of the latest client version.

Next, you will need to add this text to your project’s POM file:
<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-java</artifactId>
   <version>2.39.0</version>  //replace the version number here with the most current version number
 </dependency>

Add the text after the JUnit dependency, so that the dependencies section now looks like this:

<dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.8.1</version>
    </dependency>
    <dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.39.0</version>
</dependency>
  </dependencies>

In the command line, navigate to your project folder.  Then type:
clean install
This will download all of the dependencies needed to run Webdriver in your project.

Let’s set up a new test using Webdriver:

1. Right-click on the myFirstTests package and select New->Class
2. Give the class a name of secondTest
3. The SecondTest class should open in the center pane, and should look like this:
package myFirstTests;
public class SecondTest {
}

4. In between the brackets, add the following:
@Test
public void test() {
}

5. Add the following to the import section:
import org.junit.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

6.  In between the brackets following test(), add this code:
driver = (WebDriver) new FirefoxDriver();
String URL = “http://www.google.com”;
driver.get(URL);
driver.quit();

7.  Right-click on the test name secondTest and choose Run As-> JUnit Test.  You should see a Firefox browser open, navigate to Google, and close again, and on the JUnit tab of Eclipse, you should see that the test has passed.

Congratulations!  You are now all set up to begin writing your own automated Webdriver tests!  In my next post, I will discuss the Webdriver syntax.