Think Like a Tester

In the last two posts, we’ve been going deep, learning about how information is transmitted over the Internet.  In Part I, we learned how data is divided into packets, sent to its destination and reassembled.  In Part II, we learned how data sent over the Internet can be encrypted and protected.  But how does data know how to get from one IP address to another?  This is the subject of today’s post.

Every device that can be connected to the Internet has a network interface card (NIC), which is a piece of hardware installed on the device that allows it to make a network connection.  Each NIC has a media access control (MAC) address that is specific to only that device.

A modem is a device that connects to the Internet.  Other devices can connect to the modem in order to receive Internet transmissions.  A wireless router is capable of receiving wifi transmissions.  The router connects to the modem, and other devices connect wirelessly to the router in order to receive data from the Internet.  Many Internet service providers (ISPs) provide customers with a combination modem/router, which connects to the Internet and sends and receives wireless signals.

In Part I, we learned that every device connected to the Internet has an IP address.  An IP address is different from a MAC address in that the MAC address is assigned by the manufacturer of the device, and an IP address is assigned by the network.  An IP address has two sections, called the subnet and the host.  The subnet refers to one subsection of the entire Internet.  The host is the unique identifier for the device on the network.  The IP address combines these two numeric values using bitwise operations.  You can’t look at an IP address and say that the numbers on the left make up the subnet and the numbers on the right make up the host; it doesn’t work that way.  The IP address is more like a sum of two long numbers.

As mentioned in Part I, when a packet of data is sent from a server to a client, it is sent with the IP address of the destination.  In order to get to that destination, the packet will hop from one network to another.  The routing protocol of the Internet is called the Border Gateway Protocol (BGP).  This is a system that helps determine a route that will traverse the least number of networks to get to the destination.  Every router in a network has a series of routing tables, which are sets of directions for how to get from one network to another.

When a packet of information is first sent to the network’s router, it looks at the IP address of the destination and determines if the directions to the destination are available in the routing tables.  If they are not, the BGP is used to determine the next logical network where the packet should be sent.  A gateway is an entrance to a network, and a default gateway is the address that the request is sent to if there’s no knowledge of a specific address in that network.  When a packet arrives at the new gateway, the BGP calculates the next appropriate destination.

After traversing through networks in this way, eventually the packet arrives at the router for the network that contains the IP address of the destination.  The router determines the MAC address of the destination and sends the packet to that address.

One more important feature of networking is the use of a proxy server.  A proxy server is a server that is positioned between the client and the destination server.  It is configured so that any requests your client makes will go through the server before it gets to its destination.  There are many uses for a proxy server; the main use is to keep the actual address of a site or a router private.  Proxy servers can also be used by hackers to intercept requests, especially on a public network.  Finally, proxy servers are a great way to do security testing!  Using a tool like Fiddler or Burp Suite, you can intercept the requests that you make to your application and the responses you receive in return.  You can learn more about how to use Burp Suite in this post.

This concludes my three-part series on how the Internet works.  I hope that you have found it helpful, and that you can apply these concepts when testing!

In last week’s post, we talked about how HTTP works to pass information from a server to a browser.  But when information is passed back and forth between systems, we need to make sure that it’s protected from being intercepted by others for whom it was not intended.  That’s why HTTPS was created.  In this week’s post, we’ll talk about how encryption is used in HTTPS, what the difference is between cookies and tokens, the different types of cookies, and how cookies can be protected.

• Secure flag: ensures that the cookie can only be transmitted over HTTPS requests and not over HTTP requests.
• HttpOnly flag: keeps a cookie from being accessed via JavaScript, which helps protect it from Cross-Site Scripting (XSS) attacks.  
• SameSite flag: ensures that the cookie can only be sent from the original client that requested the cookie, which helps protect it from Cross-Site Request Forgery attacks.  

Recently, an astute reader of my blog pointed me to a great post about the importance of having technical skills as a software tester.  The author makes an excellent analogy: a software tester who doesn’t understand technical concepts is like a surgeon who doesn’t understand anatomy.  If we are going to test our applications thoroughly, we should understand the underlying systems that make them work.

I freely admit that I am not an expert in networking, or even in how the Internet works.  But I’m willing to learn, and pass that information on to you!  So let’s go deep!  We’ll begin this week with how HTTP requests work.

I’ve been testing file uploads lately, which is always fun.  It’s also important, because uploading a malicious file is one of the ways that a bad actor can exploit your application, either by taking down your application, or by extracting sensitive data from it.  In this week’s post, I’ll offer six tips and four tools to help you be successful with testing file uploads.

Anyone who has been reading my blog for a while knows that I’m passionate about API testing.  I’m also passionate about using Postman for API testing, because in my opinion it’s the easiest way to test API requests.  So it makes me sad when I see people testing an API and only asserting that they got a 200 response!  In this week’s post, I’m going to take a simple GET request and show examples of 16 assertions that could be run on the request.

For the GET request, I’ll be using the wonderful Restful-Booker application created by Mark Winteringham to teach people how to do API testing.  The request URL I’ll be using is https://restful-booker.herokuapp.com/booking/1 which will retrieve the booking with the ID of 1. If you run this GET request in Postman, you’ll get a response like this:

The response has the first and last name of a hotel guest, the total price of their room, whether or not their deposit has been paid, and what their checkin and checkout dates are.

You may see different details in the response from those in the above image, because this is a site that is under frequent use as people practice making API requests to change booking information.

Let’s look at all the different assertions we can do on this response. To add these assertions yourself, click on the “Tests” tab just underneath the request URL.

1. First, we can add the most common assertion: asserting that the response code is correct.

pm.test(“Status code is 200”, function () { pm.response.to.have.status(200); });

The pm.test refers to “Postman test”. “Status code is 200” is the name of our assertion. And pm.response.to.have.status(200) is what we are expecting from the response.

2. We can also assert that the response comes back within a reasonable time frame, like this:

pm.test(“Response time is less than 1000ms”, function () { pm.expect(pm.response.responseTime).to.be.below(1000); });

This asserts that the response time is less than one second.

3-8. Next we can assert that expected headers are present:

pm.test(“Server header is present”, function () { pm.response.to.have.header(“Server”); }); pm.test(“Connection header is present”, function () { pm.response.to.have.header(“Connection”); });

pm.test(“Content-Length header is present”, function () { pm.response.to.have.header(“Content-Length”); }); pm.test(“Etag header is present”, function () { pm.response.to.have.header(“Etag”); }); pm.test(“Date header is present”, function () { pm.response.to.have.header(“Date”); }); pm.test(“Via header is present”, function () { pm.response.to.have.header(“Via”); });

These assertions are looking for the presence of a particular header rather than checking that the header has a specific value. The specific assertions above might not be particularly necessary, but sometimes it’s a good idea to assert that certain headers are in place, such as X-XSS-Protection (not in this API), which indicates that there are measures in place to protect from cross-site scripting.  

9-10.  We can also have assertions that validate that a specific header value is being returned:

pm.test(“X-Powered-By header value is Express”, function () {
    pm.response.to.be.header(“X-Powered-By”, “Express”);
});

pm.test(“Content-Type header value is application/json”, function () {
    pm.response.to.be.header(“Content-Type”, “application/json; charset=utf-8”);
});

The first assertion is validating that the server used is Express, and the second assertion is validating that the Content-Type of the response is application/json.

11.  We can assert that certain text is included in the body of the response:

pm.test(“Response contains last name”, function () {
    pm.expect(pm.response.text()).to.include(“lastname”);
});

In this assertion, I am validating that the text in the body of the response contains “lastname”.

12-13.  I can also assert that specific json fields exist in the response.  To do this, I first need to parse out the json in the response and save it to a variable:

var jsonData = pm.response.json();

Then I can assert on specific fields in the jsonData variable.  Here I am checking to make sure that the response has both a checkin field and a checkout field:

pm.test(“Response contains checkin date”, function () {
    pm.expect(jsonData.bookingdates.checkin).to.exist;
})

pm.test(“Response contains checkout date”, function () {
    pm.expect(jsonData.bookingdates.checkout).to.exist;

})

14-15. I can also assert that specific values are returned in the response:

var jsonData = pm.response.json();

pm.test(“Correct first name is returned in response”, function () { pm.expect(jsonData.firstname).to.eql(“Mark”); });

pm.test(“Deposit has been paid”, function () { pm.expect(jsonData.depositpaid).to.be.true; });

In this first assertion, I am validating that the value of the first name returned in the response is “Mark”. In the second assertion, I’m validating that “depositpaid” is true. Keep in mind that these values may actually change because others are using this API. When you add specific value assertions, you’ll want to be sure that your test data does not change; otherwise your tests will be flaky!

16. Finally, I can assert that a value is greater or less than a certain number:

var jsonData = pm.response.json();

pm.test(“Total price is greater than 100”, function () { pm.expect(jsonData.totalprice).to.be.above(100); });

In this case, I am asserting that the total price paid is greater than 100.

The types of assertions that you run on your API responses will vary depending on what is important for you to test. While you may not decide that all types are appropriate for your API, I hope that these examples show that there are many more things to assert on than simply “200 OK”!

If you were to guess the importance of various types of automated tests by looking at the number of tutorials and articles about them on the Web, you’d think that UI tests were the most important.  But this is not the case- so much of an application can be tested through other means, especially API tests.  API tests are faster and much less flaky than UI tests, and they’re easier to write as well!  Below are four types of tests that are better suited for API testing.

• sending in an inaccurate URL
• trying a request without appropriate authentication
• testing for IDOR
• sending in incorrect headers
• sending in a request without a required field
• trying a request with a field value that violates type or length constraints
• verifying that the correct 400-level error is displayed when a request is invalid

MongoDB is one of the most popular non-relational databases in use today.  Its versatility, speed, and scalability make it popular with applications that need to store data in a JSON-like format.  It’s very easy to install MongoDB and create a database, but the query language it uses is quite different from the SQL query language.  When I first started using MongoDB, I was frustrated by the documentation I found on queries; either they tried to explain too much or the examples were too complicated.  More than once I said things in frustration like “How can I simply ask for ‘select lastName from contacts where id = 3?!!'”.

It is because of this frustration that I have created this tutorial.  In the tutorial, I’ll be including several really simple examples of queries that you are probably used to running in SQL. 

Because this post is really about writing queries, I’m going to skip the installation instructions, and instead send you here for MacOSX and here for Windows.  Once you have finished the installation instructions, you should have a command window open that is running mongo.

Creating a Database:
Creating a new database is unbelievably easy.  We’re going to name our new database tutorial.  To create it, simply type use tutorial.  You’ll get the response switched to db tutorial.  Tada!  Your database is created.

Adding a Document:
Of course, right now your database is completely empty.  Let’s change that by typing
db.tutorial.insertOne( { _id: 1, firstName: “Prunella”, lastName: “Prunewhip” } ).  
You will get a response of 
{ “acknowledged” : true, “insertedId” : 1 }. 
You have just added your first document!  Note that a “document” is the equivalent of a “record” in a SQL database.  Your document has an id (which is preceded by an underscore, by convention), a first name, and a last name.

Retrieving All Documents:
Let’s make sure that your document was really added by asking for it.  Type 
db.tutorial.find() 
and you should get this as a result: 
{ “_id” : 1, “firstName” : “Prunella”, “lastName” : “Prunewhip” }. 
The empty find() command will find all of the documents in the database.  At the moment, we only have one document, so that’s all that was returned.

Adding Multiple Documents:
To add several documents at the same time, use the InsertMany command, like this:

db.tutorial.insertMany([ { _id: 2, firstName: “Joe”, lastName: “Schmoe” }, { _id: 3, firstName: “Amy”, lastName: “Smith” }, { _id: 4, firstName: “John”, lastName: “Smith” }, { _id: 5, firstName: “Joe”, lastName: “Bagadonuts” }, { _id: 6, firstName: “Carol”, lastName: “Jones” }, { _id: 7, firstName: “Robert”, lastName: “Johnson” } ]). 

Note that each document is wrapped in curly braces, separated by commas.  You’ll get a response like this: 
{ “acknowledged” : true, “insertedIds” : [ 2, 3, 4, 5, 6, 7 ] }. 
Now you have seven records in your database.

If you retrieve all documents at this point using db.tutorial.find(), you’ll get a result like this:
{ “_id” : 1, “firstName” : “Prunella”, “lastName” : “Prunewhip” }
{ “_id” : 2, “firstName” : “Joe”, “lastName” : “Schmoe” }
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Smith” }
{ “_id” : 4, “firstName” : “John”, “lastName” : “Smith” }
{ “_id” : 5, “firstName” : “Joe”, “lastName” : “Bagadonuts” }
{ “_id” : 6, “firstName” : “Carol”, “lastName” : “Jones” }
{ “_id” : 7, “firstName” : “Robert”, “lastName” : “Johnson” }

Retrieving a Single Document:
To retrieve a single document, use this syntax:
db.tutorial.find( { _id: 1 } ).  
This will return the document with the id of 1: 
{ “_id” : 1, “firstName” : “Prunella”, “lastName” : “Prunewhip” }

Search for All Documents With a Single Value:
The previous search on id will always return just one document, because the id is unique.  If you want to search for all documents that have a particular value, you can use 
db.tutorial.find({ lastName: “Smith”}).  
This will return all documents that have the last name Smith:
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Smith” }
{ “_id” : 4, “firstName” : “John”, “lastName” : “Smith” }

Search for One Value in One Document:
Let’s say you want to find the last name of the document with the id of 3.  To do this, type:
db.tutorial.find({ _id: 3}, {lastName:1, _id:0}).  
You will get this result:  
{ “lastName” : “Smith” }. 
The _id:0 is there to specify that you don’t want the id returned in the response; returning the id in the response is a default behavior in MongoDB.

Return All the Values for a Specific Field:
If you wanted to get a list of all the last names in your database, you would use
db.tutorial.find({}, {lastName:1, _id:0}).  
This would return
{ “lastName” : “Prunewhip” }
{ “lastName” : “Schmoe” }
{ “lastName” : “Smith” }
{ “lastName” : “Smith” }
{ “lastName” : “Bagadonuts” }
{ “lastName” : “Jones” }
{ “lastName” : “Johnson” }

Search with “Starts With”:
MongoDB uses regex to search on field values.  To search for all the documents that have last names that begin with S, you’d do this search:
db.tutorial.find({ lastName: /^S/}).  
This will return 
{ “_id” : 2, “firstName” : “Joe”, “lastName” : “Schmoe” }
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Smith” }
{ “_id” : 4, “firstName” : “John”, “lastName” : “Smith” }

Search with “And”:
If you wanted to search for a document that had a specific first name AND a specific last name, you’d search like this:
db.tutorial.find( {$and: [{ lastName: “Smith” },{ firstName: “Amy”}]} )
which would return 
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Smith” }.

Search with “In”:
To search for all the documents that have either the last name Smith or the last name Jones, you’d use:
db.tutorial.find({lastName :{$in :[“Smith”,”Jones”]}}).  
This will return
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Smith” }
{ “_id” : 4, “firstName” : “John”, “lastName” : “Smith” }
{ “_id” : 6, “firstName” : “Carol”, “lastName” : “Jones” }

Update a Document:
If you’d like to change an existing document, you can use the Update command.  For example, to change the last name of the third document from Smith to Jones, you would type:
db.tutorial.updateOne({_id: 3 }, {$set: {lastName: “Jones”}}).  
You’ll get this response: 
{ “acknowledged” : true, “matchedCount” : 1, “modifiedCount” : 1 }.

To verify that the record was updated correctly, you can use db.tutorial.find( { _id: 3 } ), which will return { “_id” : 3, “firstName” : “Amy”, “lastName” : “Jones” }.

Delete a Document:
Finally, there may be times where you want to delete a document.  This can be done with
db.tutorial.deleteOne({_id: 4 })
which will return a response of 
{ “acknowledged” : true, “deletedCount” : 1 }.

To verify that the document has been deleted, you can run db.tutorial.find() and get this response:
{ “_id” : 1, “firstName” : “Prunella”, “lastName” : “Prunewhip” }
{ “_id” : 2, “firstName” : “Joe”, “lastName” : “Schmoe” }
{ “_id” : 3, “firstName” : “Amy”, “lastName” : “Jones” }
{ “_id” : 5, “firstName” : “Joe”, “lastName” : “Bagadonuts” }
{ “_id” : 6, “firstName” : “Carol”, “lastName” : “Jones” }
{ “_id” : 7, “firstName” : “Robert”, “lastName” : “Johnson” }
and you can see that the document with the id of 4 is no longer in the database.

This is by no means a complete record of everything that you can do with MongoDB, but it should be enough to get you started.  You can also refer to last week’s post to get a few examples of interacting with nested values in MongoDB.  I hope that you will find today’s post helpful in understanding how Mongo works, and that you will use it as a reference whenever you need it.  Happy querying!

Last week, I took a look at ways to query relational databases for testing.  This week I’m going to look at non-relational databases, describe how they are different from relational databases, and discuss how to query them in your testing.  Non-relational databases, such as MongoDB and DynamoDB, are sometimes called “NoSQL” databases, and are becoming increasingly popular in software applications.

The main difference between relational and non-relational databases is that relational databases use tables to store their data, where non-relational tables use documents.  The documents are often in JSON format.  Let’s take a look at what the records in the Contacts table from last week’s post would look like if they were in a non-relational database:

In last week’s post, I discussed various ways to test your application’s database.  In order to verify that your data has been saved correctly, you’ll need to query the database, and the way to query the database will depend on what type of database you have.  In the past, most databases were relational, but in recent years there has been a trend towards using non-relational databases.  In this week’s post, I’ll address relational databases, and in next week’s post, I’ll talk about non-relational databases.

Relational databases, such as MySQL and Microsoft SQL Server, are based on tables.  Each table relies on a schema, which defines what columns will be in the table, what data types they will have, and which columns will accept null values.  Here’s an example of a typical SQL table:

Note that there are seven different columns in the table.  The first column in the table, contactId, is the primary key for the table. This will be a unique value; there will never be two contactIds with the same value. 

With a relational database, the schema remains unchangeable, so when Joe Schmoe is added to the database without a phone or city, those places in the table need to be filled with NULL.

Tables in a relational database can connect to each other.  Here is a table in the same database that shows the contact’s favorite foods:

In this table the primary key is the foodId.  But notice that the contactId is present in this table.  The contactId here is the same as the contactId in the first table.  So we can see in this table that Prunella has two different favorite foods, pizza and ice cream, and Joe’s favorite food is sushi.

When testing a relational database, you can use SQL query language to verify that the values you are looking for are present in the database.  For example, if you had just added a new contact with the name of Amy Smith to the Contacts table, you could query the database to see if it had been added, like this:

select * from Contacts where lastName = ‘Smith’ and firstName = ‘Amy’

and the query would return a table row in response:

In the above query, the asterisk * tells SQL that we want all of the columns for the record returned.

Because this is a relational database, you could also do a query with a join.  A SQL join combines the data from two tables, joining on a column that they have in common.  

In the example above, both columns have the contactId column.  Let’s say that you have given your new contact Amy a favorite food (chocolate), and you want to verify that it saved to the database correctly, but you don’t know what Amy’s contactId is.  You can’t just query the Food table for “Amy Smith” because her first and last names aren’t in there.  And you can’t query the Contacts table for the food, because it’s not in that table.  But you could query the Contacts table with that information, get the contactId from that, and then use the contactId to query the Food table for the favorite food.  

This is what such a query would look like:

select food from Foods 
inner join on Contacts 
where Foods.contactId = Contacts.contactId 
and Contacts.firstName  = ‘Amy’
and Contacts.lastName = ‘Smith’ 

and the query will return this response:

  
Let’s walk through what happens in the query.
select food from Foods – this tells SQL to return just the food column from the Foods table
inner join on Contacts – this tells SQL that the query will be joining information from the Foods table with information from the Contacts table
where Foods.contactId = Contacts.contactId – this is instructing SQL to find the contactIds in the Foods table and match them up with the contactIds from the Contacts table
and Contacts.firstName  = ‘Amy’ and Contacts.lastName = ‘Smith’  – these last two lines are telling SQL that we are only interested in the record with the first name Amy and the last name Smith 

There are many more complicated ways to query a relational database, but with these two query types you will be able to do most of your data validation.  

Be sure to check out next week’s post, where I’ll talk about how to test with non-relational databases!

As software testers we often take for granted the fact that our application has a database behind it.  We tend to focus on the visible, such as the user interface, or the application logic of the API when we are testing.  But it’s important to test the database as well!  Below are six ways to test your application’s database.

1. Verify that the data fields are the correct type

Each data field in the database will be a specific type, such as int, float, string, or datetime.  Verify that each field’s datatype is appropriate.  For example, a date field in the application should be saved as a datetime type rather than as a string.  This might not seem like a big deal, but if sorting functionality is added to the application, you’ll find that April 10 is sorted before January 11 because the dates are being sorted alphabetically.

2. Verify that the fields that are required in the database are also required in the API and the UI

Generally each record in a database will have some fields that are required.  If a field is required in the database, it should also be required in the API and the UI.  If the UI doesn’t set the last name field as required, but the database has set it as required, a user could submit a new record without a last name and the database will return an error.

3. Verify that the parameters set on the fields in the database match the parameters in the API and the UI

Data fields in a database will have specific limits set, such as a character limit for a string, or a maximum value for an int. The fields in the API and the UI should have the same limits.  If the limits do not match, confusion can ensue.  Let’s say for example that your application has a field for a street address.  In the UI the developer has set the character limit to 50 characters, but in the database, the limit is set to 40.  If a user types in a street address with 45 characters, the UI will accept that value, but the database will not.  This will result in the record not being saved, and it won’t be obvious to the user what the problem is.

4. Verify that sensitive data, such as user passwords, are encrypted in the database

I once worked for a company that did not encrypt their user passwords.  I had access to the production database, which meant that I could see the username and password for every single customer of the company.  It should be pretty obvious that this was a huge security risk!  Passwords should always be encrypted in the database so there is no way for anyone but the user to know what their password is.

5. Verify that your database supports all your API’s operations

Just because your POST request returns a 200 doesn’t mean that the data was saved to the database correctly.  When you make an API request, make sure that the database saved every field correctly.  I have seen situations where a PUT request did not result in every new value being saved to the database.  I’ve also seen a situation where a PATCH request updated the correct fields, but set every other field to null!  Be sure to test every available CRUD operation, and check every field for accuracy.  Also make sure to change field values from null to an entry, and from an entry to null.  If a field you are testing is a string, determine with your team whether empty strings will be allowed, and test going from null to an empty string, an empty string to a value, from a value to an empty string, and so on.

In addition, if your API is going to support the DELETE operation, find out from your team whether those deletes will be hard-deletes, meaning that the record will be completely removed from the database, or soft-deletes, meaning that the record stays in the database, but is moved to another table or stays in the existing table with a “deleted” flag set.  Then test the DELETE operation to verify that it is behaving as expected.

6. Verify that leading and trailing spaces are removed when saving to the database

Have you ever had trouble logging in to an application where you are sure you have the right username?  The problem could be that when the username was originally created, a trailing space was accidentally added, and it was saved to the database with the extra space.  When you tried to log in as “doglvr49”, the system was expecting “doglvr49 “.  Similarly, if you entered a contact’s last name as ” Jones” instead of “Jones” and the leading space wasn’t trimmed, and you tried to sort the contacts alphabetically, you’d find ” Jones” before “Allan”.  When you are testing text fields in your application, try testing them with leading and trailing whitespaces, and then verify in the database that those spaces have been trimmed.

Issues such as having a mismatch between character limits in the database and the UI, or having a string instead of an int, or having trailing spaces in a record, can seem like small issues, but they can result in big problems.  By following these six steps in database testing, you will help ensure that your end users will have a good experience entering and retrieving data.