If you have ever made a REST request or looked in the developer tools section of a browser, you have likely seen the three-digit response code that is returned with an HTTP request. This week, we’ll be talking about the different types of response codes you might receive when doing API testing, and what those codes mean.
Creating a Postman Collection
This week we’ll be finishing up our discussion of REST request types with an introduction to the DELETE request and how to test it. We will also be looking at how to chain REST tests together in a Postman collection.
A DELETE request removes an entire record from a database table. To see a DELETE request in action, let’s return to the Swagger Pet Store. First let’s open up the GET /pet/{petId} request. Click on the “Try it out” button, and put “100” in the petId field. Then click the Execute button. Take a look at the server response. Was a pet returned? If not, try a different petId. Keep looking until you find an existing record.
Now that you have an existing record, open up the DELETE /pet/{petId} request and click on the “Try it out” button. In the petId field, enter the same id that you used for the GET request. Click the Execute button. In the server response, you should see a success code of 200.
How can you tell that the record was really deleted? Simply return to the GET request, put the same id in the petId field, and click the Execute button. You should now get a “Pet not found” response. You have successfully deleted a pet from the database!
Testing a DELETE request is fairly straightforward. Since the only information you are passing into the request is the id of the record you want to delete, there’s not much room for variation. You can test what happens when you enter an id for a record that doesn’t exist (you should get a 404 response) and what happens when you enter an invalid id such as “FOO”. If the DELETE functionality in your application is limited to users with authorization, you can try deleting a record without the token or cookie that gives you the required permission. You can try sending the request with an http URL instead of an https (or vice versa), or you can try sending a URL that doesn’t specify an id to delete.
Now let’s go to Postman and try chaining some requests together! Imagine that you are putting together a test suite for the Swagger Pet Store. You’d like to check to make sure that you can add a pet, retrieve a pet, edit a pet, and delete a pet from the database. When you add a pet to the database, you’d like to make sure the record has really been added; so you’d like to have a GET request to verify that. Similarly, when you edit the pet, you’d like to do a GET request to verify that the pet has been changed. And finally, when you delete the pet, you’d like to do a GET request to verify that the pet has really been deleted. Also, you’d like to have your test suite clean up after itself, so you aren’t adding more and more records to your database each day. Chaining your requests together in a Postman collection will accomplish all of this. This is the order of requests that you will make:
POST pet- to add the pet to the database
GET pet- to verify that the pet has been added
PUT pet- to change the existing pet
GET pet- to verify that the pet has been changed
DELETE pet- to remove the pet from the database
GET pet- to verify that the pet has been removed
To create a Postman collection, first click on the Collections tab on the top left of the screen. Then click on the icon with the folder and plus sign:
Give your collection a name, such as “Pet Store”, and click the Create button. You will see a folder with your collection name in the left column of the screen. Now click on the “+” tab in the main window of the screen to add a new request. We’ll start with the POST pet request. Create the request like this:
Request type: POST
URL: http://petstore.swagger.io/v2/pet
Headers: Content-Type: application/json
Body:
{
“id”: 100,
“category”: {
“id”: 1,
“name”: “cat”
},
“name”: “Grumpy Cat”,
“photoUrls”: [
“https://pbs.twimg.com/profile_images/948294484596375552/RyGNqDEM_400x400.jpg”
],
“tags”: [
{
“id”: 1,
“name”: “blue eyes”
}
],
“status”: “sold”
}
If any of this seems unfamiliar, you can see more detailed instructions in my post on POST requests.
Once the request is ready and working, you can save it to your collection. Click on the Save button in the upper right of the screen. A popup window will appear. Give your request a name, such as “Add Pet”, and then select the collection that you created earlier. Click the Save button, and you should now see your request in the collection folder on the left of the screen.
We can repeat this process with the GET request:
Request type: GET
URL: http://petstore.swagger.io/v2/pet/100
As soon as this request is working, save it to your collection with a name like “Verify Add Pet”. We are calling the request this because we will be using it to verify that the previous POST request worked correctly.
Next, we’ll edit the pet with a PUT request:
Request type: PUT
URL: http://petstore.swagger.io/v2/pet
Headers: Content-Type: application/json
Body:
Once this request is working, save it to your collection with a name like “Update Pet”.
Now we’ll want to do another GET request, to verify that the PUT request worked correctly. We already have a GET request saved, so we can just duplicate it in our collection. Hover over the “Verify Add Pet” request on the left of the screen, and you’ll see three dots appear to the right of the request name. Click on the three dots and choose “Duplicate”. You should now have a request below the original “Verify Add Pet” request that says “Verify Add Pet copy”. Click on this request and then on the three dots beside it and choose “Rename”. Rename your request to “Verify Update Pet”, and click return. Since we will be using this GET request to verify that the PUT request worked correctly, click and hold on the request and drag it below the PUT request.
Next we’ll want to create a DELETE request:
Request type: DELETE
URL: http://petstore.swagger.io/v2/pet/100
Save this request to your collection with a name like “Delete Pet”.
Finally, we’ll want to do one more GET request, to verify that the DELETE request removed the pet correctly. Duplicate the “Verify Update Pet” request, rename the copied request to “Verify Delete Pet”, and move it below the DELETE request. Remember that when you run this request after your delete, you should get a “Pet not found” response.
We now have a collection with six requests that can be used to test the pet function of the Pet Store! It should look like this:
Try clicking on each of these requests in order and using the blue Send button in the top right of the screen to run each one. You should be able to add a pet, verify that it has been added, update the pet, verify that it has been updated, delete the pet, and verify that it has been deleted.
We will use this collection in the next couple of weeks to create assertions that will turn these requests into true tests. Next week, we’ll discuss response codes and what they mean; then we’ll put some response code assertions into our requests. Until then, have fun playing around with your Pet Store collection!
Testing PATCH Requests
Like PUT requests, PATCH requests modify an existing record. But PATCH requests are much tricker to test! This is because a PUT request modifies an entire record, whereas a PATCH request modifies only one part of the record. There are many different operations that you can do within a PATCH request: you can add, replace, remove, copy, and move a value in your record. I’ll describe some examples of each and discuss the various ways you can test them.
- Happy Path- patching where there is currently a null value
- patching over the null value with an empty value of “”- this should add an empty string
- patching over an empty value of “”- this should replace the existing empty string
- patching over an existing value- this will replace the existing value, but ideally the replace operation should be used here instead
- adding a value with too many or too fewer characters than allowed- an appropriate error message should be returned
- adding a value with characters that are not allowed- an appropriate error message should be returned
- adding a value of the wrong type, such as adding an integer when a string is expected- an appropriate error message should be returned
Next, let’s take a look at the replace operation. Let’s use the second record this time, and we’ll replace the home phone. This is what the record currently looks like:
- Happy Path- replacing one value with another
- replacing the value with null- a remove operation would be better for this, but this should still work
- replacing the value with the empty string “”- this should work
- replacing a null value with a value- this will probably work, but it would be better to use the add operation
- replacing with a value with too many or too fewer characters than allowed- an appropriate error message should be returned
- replacing with a value with characters that are not allowed- an appropriate error message should be returned
- replacing with a value of the wrong type, such as adding an integer when a string is expected- an appropriate error message should be returned
- replacing where the existing value is bad in some way, such as having too many characters or having the wrong format- in this case the new good value should be allowed to replace the bad old value
- Happy Path- moving an existing value to a location where the value is currently null
- moving an existing value from location A to location B, where there is already a value in location B- in this case, the value in location B will be replaced with the value that was in location A
- doing a move from A to B where the value of A is null- this should return an appropriate error message, because there’s nothing to move
- doing a move from A to B where the validation constraints are different in location B, so A’s value should not be allowed- this should return an appropriate error message
- doing a move from A to B where the value in A is bad- this should return an appropriate error message
- moving a value from one location to a location that does not exist: this should return an appropriate error message
- Happy Path: copying a value from one location to another where the value is currently null
- copying a value from location A to location B, where location B currently has a value; this value will be replaced by the value in location A
- copying from location A where the value of A is null- this should return an error message
- copying from location A to a location that does not exist- this should return an error message
- copying from A to B where the validation constraints are different in location B- this should return an error message
- copying a bad value from location A to location B- this should return an error message
Testing PUT Requests
In last week’s blog post, we discussed how to create and test POST requests. This week, we will tackle testing PUT requests. A PUT request is actually very similar to a POST request; the major difference is that POST requests are intended to create a new record, and PUT requests are intended to replace an existing record.
Let’s return to the Swagger Pet Store to learn how to create a PUT request. Click on the PUT /pet request to open it:
Testing POST Requests
Last week, I introduced the concept of the GET request and how to test it. This week we’ll move to POST requests. POST requests are perhaps the most important of the RESTful requests, because they are what adds new records to your application’s database. It’s very important to test your POST requests well, because they will have a direct impact on the quality of data in your database.
To learn about POST requests, we’ll once again use the Swagger Pet Store and Postman. Navigate to the Pet Store (http://petstore.swagger.io) and click on the first POST request listed: “/pet”. This POST request will add a pet to the pet store. Take a look at the Example Value shown:
This is the body of the POST request. Unlike GET requests, which usually don’t have a body, you will usually find some json or xml in a POST request. The body represents the data that you are adding to the database. Now click on the Model link:
This describes a bit about what all of the values in the body are. You can click on the “>” icons to open up each section of the model. I find this model to be a bit vague in terms of defining what a category is and what tags are, so I’m going to do a little guesswork. Let’s walk through each section of the Pet model:
id: this is the id of the pet, which can be used in the GET request we tested last week
category: this represents what kind of animal the pet is. The id is the unique identifier for the category, and the name is the word describing the animal.
name: this is the pet’s name
photoUrls: these are strings that link to pictures of the pet
tags: these are descriptive phrases that can be added to the pet. The id is the unique identifier for the tag, and the name is the descriptive word describing something about the pet
status: this is the status of the pet in the store. The status can be available, pending, or sold.
We can use the information in the model to create a POST to make a new pet. Click on the “Try it out” link, and replace the body of the request with this information:
{
“id”: 102,
“category”: {
“id”: 1,
“name”: “cat”
},
“name”: “Grumpy Cat”,
“photoUrls”: [
“https://pbs.twimg.com/profile_images/948294484596375552/RyGNqDEM_400x400.jpg”
],
“tags”: [
{
“id”: 1,
“name”: “blue eyes”
}
],
“status”: “sold”
}
Testing GET Requests
Last week I introduced the concept of RESTful API requests, and discussed why it’s crucial that we test them in our applications. This week, we will begin our discussion of RESTful request types with the GET request. This is usually the easiest request to test, because all we are doing is retrieving data from the database. We don’t need to worry about whether we are manipulating data correctly; we just need to retrieve it and check that we get a correct response.
In my opinion, the two best tools to learn about REST requests are Swagger and Postman. Swagger is an open-source framework that allows developers to create documentation for RESTful APIs. If your API has a Swagger file, it’s easy to see what kinds of requests your API allows, and what sorts of parameters those requests are expecting. The developers of Swagger have also created a sample application that you can use to practice making REST requests: petstore.swagger.io. This website simulates an online pet store, where users can enter and retrieve information about their pets.
When you go to the petstore.swagger.io website, you can see that there are a variety of requests available for the pet endpoint. Let’s take a look at the GET /pet/{petId} endpoint. Click on that request, then click on the “Try it out” button. You will notice that a petId is a required parameter. Enter “1” into the parameter text field, and click the “Execute” button. Scroll down to the Response Body section, and you will see that a pet has been returned! Scroll up a bit and take a look at the Request URL: http://petstore.swagger.io/v2/pet/1. We will use this URL to learn about making requests in Postman.
Postman is the best tool that I have found for testing REST requests. It is available for free at https://www.getpostman.com, and there is also a paid Team version. If you don’t have it already, download Postman and start it up.
Postman should start up with an open tab, set to use a GET request. All you have to do now is enter the request URL that we used before, and click the “Send” button. You should get the same response that you did when you made the request in Swagger.
Now that we understand how GET requests work, let’s think about how to test them! We have obviously tested a Happy Path scenario where we are getting a pet with an id of 1. What happens if we change the id parameter to 2? What happens if we change the id parameter to -1? What if it’s 0? What if it’s “foo”? I should mention here that because the Swagger Pet Store is a test application, you may see results or behavior that would not be wanted in an application. For instance, when I tested with a parameter of -1, I got a result. It’s a fairly common expectation that the ids of data objects not be negative numbers, so this would be a bug in a real application.
Let’s find out what happens when we don’t enter any parameter at all, so our request URL is just http://petstore.swagger.io/v2/pet. We get an xml response that says “unknown”. You may also notice that the status code returned is “405 Method Not Allowed”.
The status codes you get after you make a REST request tell you a bit about the behavior of your application. We’ll discuss status codes further in a later post, but for now, let’s note that a status code of 200 is good, and a status code that begins with 4 generally indicates that something has gone wrong. You may get a 404 status when you search for a pet id that doesn’t exist. This means that the record was not found.
Let’s return now to the Swagger Pet Store and take a look at another GET request. This time, we will look at the pets/findByStatus request. Click on the request, and then click the “Try it out” button.
The question mark in the URL indicates that we are using a query parameter. This is a little different from the path parameter we saw when we were doing a GET by pet id. Query parameters always begin with a question mark, and then have the parameter name, an equals sign, and the value of the parameter we want to use.
Let’s copy this request and run it in Postman. Simply use the + button near the top of the screen to open a tab and create a new request. It should be a GET request by default. Enter the URL and click “Send”. You should get the same result that you saw when you ran the request in Swagger.
We have now tested one Happy Path for this request. You can also run the request using the “available” parameter and the “pending” parameter. You could run this request: http://petstore.swagger.io/v2/pet/findByStatus?status=pending,sold and get all of the pending pets and all of the sold pets! What happens if you send a value of “foo” for the status parameter? You will get an empty set [ ] as a result. This is different from what we saw when we sent in “foo” as the parameter for the GET pet by pet ID request. This demonstrates the difference between a path parameter and a query parameter. When a path parameter does not exist, you will generally get a 400-level response code. When a query parameter does not exist, you will generally get a 200 response code and an empty set in the response body.
Now that you have tried out two different GET requests, you can test further by manipulating the request URL in any number of ways. For example, you could see what happens if you make the request an https request instead of http. You could find out what happens if you change the “io” in the request to “com”. You could change the “v2” to “v1”. You could remove the “/pet” endpoint. You could try changing the GET request type to a POST or a DELETE type. Seeing what happens when you manipulate the URL and the request type will give you a sense of how REST requests work, and will also give you ideas for what you could test in an API that you are responsible for.
I hope that this post has demonstrated just how easy API testing is, and also how versatile it can be. With API testing, you can test many scenarios that would not be possible to test through the UI. This is a valuable strategy to use in finding potential design and security flaws earlier in development. Next week, we’ll move on to POST requests!
Introduction to REST Requests
More and more companies are moving toward a microservices model for their applications. This means that different sections of their application can have a separate datastore and separate commands for interacting with that datastore. The advantage to this is that it’s easier to deploy changes to a small component of the application rather than the entire application; it also means that if one microservice goes down, the rest of the application can continue to function. For example, let’s imagine you have a website for a bike rental service. The site has a microservice for the reservation system, and a second microservice for the inventory. If the microservice for the inventory goes down, users will still be able to make reservations for bike rentals, using cached data from the inventory microservice.
Most microservices are using APIs, or Application Programming Interfaces, which are a set of commands for how a service can be used. And most APIs are using REST requests, or Representational State Transfers, through HTTP to request and send data.
Yet in spite of the common usage of RESTful APIs in today’s applications, many testers do not know just how easy it is to test them! This post will serve as a gentle introduction to REST requests for use in API testing.
Why would you want to test REST requests, rather than just wait and test through the UI? Here are a few good reasons:
- testing REST requests means that you can find bugs earlier in the development process, sometimes even before the UI has been created!
- malicious users know how to make REST requests, and can use them to exploit security flaws in your application by making requests the UI doesn’t allow
- it’s easy to automate REST requests, and they run MUCH faster than UI automation (see my earlier blog post for further discussion of API vs. UI automation)
- the https is specifying that this is a secure request
- the www.foobank.com is the domain, which says that you want to go to the Foobank website
- the customers is the first part of the path, which says that you are a customer and therefore want to go the Customers section of the website
- the login is the last part of the path, which says that you want to go the login screen
- a POST request adds a new record to the database
- a GET request retrieves a record from the database
- a PUT request takes a record from the database and replaces it with a new record
- a PATCH request modifies an existing record in the database
- a DELETE request removes a record from the database
Testing the Login Screen
The login screen is the first line of defense between your application and a malicious unauthorized user. But it’s so easy to forget about testing the login screen! This is because as a tester, you are in the application every single day. Getting past the login screen is a step you take dozens of times a day to get to whatever feature you are currently testing.
Let’s take some time to look at the various ways we should be testing login screens.
First, make sure that the password field masks text as you type in it. Next, check to make sure that both the username and the password fields are expecting case-sensitive text. In other words, if your username is “FOO” and your password is “bar”, you should not be able to log in with “Foo” and “Bar”.
Next, find out what the validation requirements are for the username and password fields. How short can the values be? What’s the longest they can be? What characters will be accepted? Verify that the length requirements are respected in both fields, then test with every non-accepted character that you can think of. Be sure to test with non-UTF-8 characters and emojis as well. You may have heard a story several years back of a bank customer who broke the application by setting her password to have an emoji in it!
Also be sure to test for SQL injection. For example, if you enter 1′ or ‘1’ = ‘1 into the password field, you may be telling the database to return ‘true’ and allow logging in, because of course ‘1’ always equals ‘1’.
Now let’s test with every possible combination of username and password states. Each field can be either empty, incorrect, correct but with the wrong case, and correct. This generates sixteen possible combinations:
It may seem like overkill to test all these combinations, but I have seen situations (fortunately that were only in the earliest stages of development) where I could log in with an empty username and password, or log in where both the username and password were incorrect!
Another helpful thing to test is putting an empty character or two at the beginning or end of the fields. When the user submits their login request, empty characters should be stripped from these values. If this does not happen, the user may wind up with a situation where they have typed their correct password: “bar “, and the password is not accepted because of the space at the end. This is very frustrating for an end user who doesn’t know the empty character is there.
Next, let’s think about the kind of error message you are getting when you try to login with invalid credentials. You don’t want to give a malicious user any kind of hints about whether they have guessed a username or a password correctly. If the malicious user types in “FOO” for the username and “password” for the password, you don’t want to return a message that says “Invalid password”, because this will let the malicious user know that the username is correct! Now all they have to do is keep the username the same and start guessing at the password. It is much better to return a message like “Invalid credentials”.
Now let’s take a look at what is passed into the server when you are making a login request. Open up the developer tools for the browser you are using, and watch the request that goes through. Do you see the username or the password in clear text anywhere? If you do, this should be fixed! Usernames and passwords should both be encrypted in the request. You can also check the request by using an interception tool like Fiddler or Burp Suite.
Once you have logged in with correct credentials, take a look at the cookie or session id that has been set for your user. Do you see the username and password anywhere? All cookies, web tokens, and session ids should not include the user credentials, even if they are encrypted.
Finally, be sure to test logging out! When you log out, the username and password fields on the login screen should be cleared, unless there is a feature to remember the credentials. If there is such a feature, the password should definitely be masked and should give no indication of how many characters are in it. Upon logging out, any session ids or web tokens should be deleted. If there is no feature to remember credentials, the cookie should be deleted as well. There should be nothing that you can grab from developer tools or interception tools to use to pretend that you are logged in.
This is a lot to test; fortunately, we can take advantage of automation for regression testing once we have done an initial test pass. Here are a few ideas for what to automate:
API login calls using all the combinations of username and password listed above
UI logins with too many characters and invalid characters in the username and password fields; verify that the error message returned is the correct one
Visual testing of the login screen to verify that the password is not displayed in clear text (a good tool for visual testing can be found at https://www.applitools.com)
Security is such an important part of any application! By running through all of these login scenarios, you can help bring peace of mind to your product team and to your end users.
Testing Back Buttons
The back button is so ubiquitous that it is easily overlooked when it comes to web and application testing. The first thing to know when testing back buttons is that there are many different types. The two major categories are: those that come natively, and those that are added into the application.
For native buttons, there are those that are embedded in the browser, those that are embedded in a mobile application, and those that are included in the hardware of a mobile device. In the photo above, the back arrow is the back button that comes with the Chrome browser. An Android device generally has a back button at the bottom that can be used with any application. And most iPhone apps have a back button built in at the top of every page.
Added back buttons are used when the designer wants to have more control over the user’s navigation. In the example above, the Home button is used to go back to the W3 Schools home page. (I should mention here that W3 Schools is an awesome way to learn HTML, CSS, Javascript, and much more: https://www.w3schools.com/default.asp)
When you are testing websites and applications, it’s important to test the behavior of ALL of your back buttons, even those that your team didn’t add. The first thing to do is to think about where exactly you would like those buttons to go. This seems obvious, but sometimes you do not want your application to go back to just the previous page. An example of this would be when a user is editing their contact information on an Edit screen. When the user is done editing, and has gone to the Summary page, and they click the Back button, they shouldn’t be taken back to the Edit screen, because then they’ll think they will have to edit their information all over again. Instead, they should go back to the page before.
Another thing to consider is how you would like back buttons to behave when the user has logged out of the application. In this case, you don’t want to be able to back into the application and have the user logged in again, because this is a security concern. What if the user was on a public computer? The next user would have access to the previous user’s information.
For mobile device buttons, think about what behavior you would like the button to have when you are in your application. A user will be frustrated if they are expecting the back button to take them elsewhere in your app, and it instead takes them out of the app entirely.
If your application has a number of added back buttons, be sure to follow them all in the largest chain you can create. Look for errors in logic, where the application takes you to someplace you weren’t expecting, or for memory errors caused by saving too many pages in the application’s path.
You can also check whether the back button is enabled and disabled at appropriate times. You don’t want your user trying to click on an enabled back button that doesn’t go anywhere!
In summary, whenever you are testing a web page or an application, be sure to make note of all of the back buttons that are available and all of the behaviors those buttons should have. Then test those behaviors and make sure that your users will have a positive and helpful experience.
Automated Form Testing
Now that we have looked at all the different ways we can manually test forms, it’s time to think about automating those tests so you don’t have to run through all the tests again every time you need to do a regression test! But before you jump into automation, think for a while about what tests you really want to run. Let’s take last week’s form as our example again.
What sorts of things would we want to make sure still work when we do our regression test?
- We’ll want to make sure that every field can be populated with data and saved
- We’ll want to make sure that all of the required fields are still required, and that we get an appropriate error message when we leave a required field blank
- We’ll want to make sure that validation rules are respected in each field
- We’ll want to verify that both the Save and the Cancel buttons work correctly
Then I verify that all of the fields are displayed on the page