Testing With Non-Relational Databases

Last week, I took a look at ways to query relational databases for testing.  This week I’m going to look at non-relational databases, describe how they are different from relational databases, and discuss how to query them in your testing.  Non-relational databases, such as MongoDB and DynamoDB, are sometimes called “NoSQL” databases, and are becoming increasingly popular in software applications.

The main difference between relational and non-relational databases is that relational databases use tables to store their data, where non-relational tables use documents.  The documents are often in JSON format.  Let’s take a look at what the records in the Contacts table from last week’s post would look like if they were in a non-relational database:

{
              contactId: “10000”,
              firstName: “Prunella”,
              lastName: “Prunewhip”,
              email: “[email protected]”,
              phone: “8005551000”,
              city: “Phoenix”,
              state: “AZ”
}
{
              contactId: “10001”,
              firstName: “Joe”,
              lastName: “Schmoe”,
              email: “[email protected]”,
              state: “RI”,
}
Note that Joe does not have a value for phone or city entered, so they are not included in his document.  This is different from relational databases, which are required to include a value for every field. Instead of having a NULL value for phone and city as Joe’s record did in the SQL table, those fields are simply not listed.
Another key difference between relational and non-relational databases is that it’s possible to add a new field into a table without adding it in for every document.  Let’s imagine that we are adding a new record to the table, and we want that record to include a spouse’s name.  When that record is added, it will look like this:

{
              contactId: “10002”,
              firstName: “Amy”,
              lastName: “Smith”,
              email: “[email protected]”,
              phone: “8885551001”,
              city: “Boise”,
              state: “ID”,
              spouse: “John”
}
The original documents, 10000 and 10001, don’t need to have this spouse value.  In a relational database if a new field is added, the entire schema of the table needs to be altered, and Prunella and Joe will need to have spouse values or NULL entered in for those fields.

With a non-relational database, it’s not possible to do joins on table data as you saw in last week’s post.  Each record should be treated as its own separate document, and you can do queries to retrieve the documents you want.  What that query language looks like depends on the type of the database used.  The examples below are using MongoDB’s query language, which is JavaScript-based, and are querying on the documents listed above:

db.contacts.find() – this will return all the contacts in the table
db.contacts.find( { contactId: “10001” } ) – this will return the document for Joe Schmoe

To make the responses easier to read, you can append the command .pretty(), which will organize the data returned in JSON format rather than a single line of values. 

You can also run a query to return a single field for each document:

db.contacts.find({}, {firstName:1, _id:0}) – this will return just the first name for each contact

Because the documents in a non-relational database have a JSON-like structure, it’s possible to have documents with arrays.  For example, our Contacts table could have a document that lists the contact’s favorite foods:

{
              contactId: “10000”,
              firstName: “Prunella”,
              lastName: “Prunewhip”,
              email: “[email protected]”,
              phone: “8005551000”,
              city: “Phoenix”,
              state: “AZ”,
              foods: [ “pizza”, “ice cream” ]
}

It’s even possible to have objects within arrays, as follows:

{
              contactId: “10001”,
              firstName: “Joe”,
              lastName: “Schmoe”,
              email: “[email protected]”,
              state: “RI”,
              pets: [ { type: “dog”, name: “fido” }, { type: “cat”, name: “fluffy” } ]
}

You can see how this type of data storage might be advantageous for your application’s data.  Nesting data in this fashion makes it easier to read at a glance than it would be in a relational database, where the pets might be in their own separate table.

To run a query that will return all the contacts that have cats, you would simply request:

db.contacts.find( {“pets.type”:”cat”} )

To run a query that will return all the contacts that have cats named Fluffy, you would request:

db.contacts.find( {$and: [{“pets.type”:”cat”},{“pets.name”:”fluffy”}]} )

These are just a few simple examples of how to query data with a non-relational database, and they should be enough to get you started in your testing.  To learn more, be sure to read the documentation for the type of database you are using.  As non-relational databases become increasingly popular, this knowledge will be extremely useful.  

Testing With Relational Databases

In last week’s post, I discussed various ways to test your application’s database.  In order to verify that your data has been saved correctly, you’ll need to query the database, and the way to query the database will depend on what type of database you have.  In the past, most databases were relational, but in recent years there has been a trend towards using non-relational databases.  In this week’s post, I’ll address relational databases, and in next week’s post, I’ll talk about non-relational databases.

Relational databases, such as MySQL and Microsoft SQL Server, are based on tables.  Each table relies on a schema, which defines what columns will be in the table, what data types they will have, and which columns will accept null values.  Here’s an example of a typical SQL table:

contactId
firstName
lastName
email
phone
city
state
10000
Prunella
Prunewhip
8005551000
Phoenix
AZ
10001
Joe
Schmoe
NULL
NULL
RI

Note that there are seven different columns in the table.  The first column in the table, contactId, is the primary key for the table. This will be a unique value; there will never be two contactIds with the same value. 

With a relational database, the schema remains unchangeable, so when Joe Schmoe is added to the database without a phone or city, those places in the table need to be filled with NULL.

Tables in a relational database can connect to each other.  Here is a table in the same database that shows the contact’s favorite foods:

foodId
contactId
food
1
10000
Pizza
2
10000
Ice cream
3
10001
Sushi

In this table the primary key is the foodId.  But notice that the contactId is present in this table.  The contactId here is the same as the contactId in the first table.  So we can see in this table that Prunella has two different favorite foods, pizza and ice cream, and Joe’s favorite food is sushi.

When testing a relational database, you can use SQL query language to verify that the values you are looking for are present in the database.  For example, if you had just added a new contact with the name of Amy Smith to the Contacts table, you could query the database to see if it had been added, like this:

select * from Contacts where lastName = ‘Smith’ and firstName = ‘Amy’

and the query would return a table row in response:

contactId
firstName
lastName
email
phone
city
state
10003
Amy
Smith
8885551001
Boise
ID

In the above query, the asterisk * tells SQL that we want all of the columns for the record returned.

Because this is a relational database, you could also do a query with a join.  A SQL join combines the data from two tables, joining on a column that they have in common.  

In the example above, both columns have the contactId column.  Let’s say that you have given your new contact Amy a favorite food (chocolate), and you want to verify that it saved to the database correctly, but you don’t know what Amy’s contactId is.  You can’t just query the Food table for “Amy Smith” because her first and last names aren’t in there.  And you can’t query the Contacts table for the food, because it’s not in that table.  But you could query the Contacts table with that information, get the contactId from that, and then use the contactId to query the Food table for the favorite food.  

This is what such a query would look like:

select food from Foods 
inner join on Contacts 
where Foods.contactId = Contacts.contactId 
and Contacts.firstName  = ‘Amy’
and Contacts.lastName = ‘Smith’ 

and the query will return this response:

food
Chocolate

  
Let’s walk through what happens in the query.
select food from Foods – this tells SQL to return just the food column from the Foods table
inner join on Contacts – this tells SQL that the query will be joining information from the Foods table with information from the Contacts table
where Foods.contactId = Contacts.contactId – this is instructing SQL to find the contactIds in the Foods table and match them up with the contactIds from the Contacts table
and Contacts.firstName  = ‘Amy’ and Contacts.lastName = ‘Smith’  – these last two lines are telling SQL that we are only interested in the record with the first name Amy and the last name Smith 

There are many more complicated ways to query a relational database, but with these two query types you will be able to do most of your data validation.  

Be sure to check out next week’s post, where I’ll talk about how to test with non-relational databases!

Database Testing

As software testers we often take for granted the fact that our application has a database behind it.  We tend to focus on the visible, such as the user interface, or the application logic of the API when we are testing.  But it’s important to test the database as well!  Below are six ways to test your application’s database.

1. Verify that the data fields are the correct type

Each data field in the database will be a specific type, such as int, float, string, or datetime.  Verify that each field’s datatype is appropriate.  For example, a date field in the application should be saved as a datetime type rather than as a string.  This might not seem like a big deal, but if sorting functionality is added to the application, you’ll find that April 10 is sorted before January 11 because the dates are being sorted alphabetically.

2. Verify that the fields that are required in the database are also required in the API and the UI

Generally each record in a database will have some fields that are required.  If a field is required in the database, it should also be required in the API and the UI.  If the UI doesn’t set the last name field as required, but the database has set it as required, a user could submit a new record without a last name and the database will return an error.

3. Verify that the parameters set on the fields in the database match the parameters in the API and the UI

Data fields in a database will have specific limits set, such as a character limit for a string, or a maximum value for an int. The fields in the API and the UI should have the same limits.  If the limits do not match, confusion can ensue.  Let’s say for example that your application has a field for a street address.  In the UI the developer has set the character limit to 50 characters, but in the database, the limit is set to 40.  If a user types in a street address with 45 characters, the UI will accept that value, but the database will not.  This will result in the record not being saved, and it won’t be obvious to the user what the problem is.

4. Verify that sensitive data, such as user passwords, are encrypted in the database

I once worked for a company that did not encrypt their user passwords.  I had access to the production database, which meant that I could see the username and password for every single customer of the company.  It should be pretty obvious that this was a huge security risk!  Passwords should always be encrypted in the database so there is no way for anyone but the user to know what their password is.

5. Verify that your database supports all your API’s operations

Just because your POST request returns a 200 doesn’t mean that the data was saved to the database correctly.  When you make an API request, make sure that the database saved every field correctly.  I have seen situations where a PUT request did not result in every new value being saved to the database.  I’ve also seen a situation where a PATCH request updated the correct fields, but set every other field to null!  Be sure to test every available CRUD operation, and check every field for accuracy.  Also make sure to change field values from null to an entry, and from an entry to null.  If a field you are testing is a string, determine with your team whether empty strings will be allowed, and test going from null to an empty string, an empty string to a value, from a value to an empty string, and so on.

In addition, if your API is going to support the DELETE operation, find out from your team whether those deletes will be hard-deletes, meaning that the record will be completely removed from the database, or soft-deletes, meaning that the record stays in the database, but is moved to another table or stays in the existing table with a “deleted” flag set.  Then test the DELETE operation to verify that it is behaving as expected.

6. Verify that leading and trailing spaces are removed when saving to the database

Have you ever had trouble logging in to an application where you are sure you have the right username?  The problem could be that when the username was originally created, a trailing space was accidentally added, and it was saved to the database with the extra space.  When you tried to log in as “doglvr49”, the system was expecting “doglvr49 “.  Similarly, if you entered a contact’s last name as ” Jones” instead of “Jones” and the leading space wasn’t trimmed, and you tried to sort the contacts alphabetically, you’d find ” Jones” before “Allan”.  When you are testing text fields in your application, try testing them with leading and trailing whitespaces, and then verify in the database that those spaces have been trimmed.

Issues such as having a mismatch between character limits in the database and the UI, or having a string instead of an int, or having trailing spaces in a record, can seem like small issues, but they can result in big problems.  By following these six steps in database testing, you will help ensure that your end users will have a good experience entering and retrieving data.

Why The Manual vs. Automation Debate is Wrong

I don’t generally editorialize in my blog- I prefer to focus on what to test rather than theories of testing- but I feel compelled to say that I’m tired of the whole “manual vs. automated testing” discussion.  Some people describe automated testing as the cure for bad code everywhere, and others lament the poor manual tester who has no technical skills.  Meanwhile, there are advocates who say that automation is merely a panacea, and that automation code should be used only for simple tools that will aid the manual tester, who is the one who really knows the product.

In my opinion, this debate is unnecessary for two reasons:
1) “Manual” and “automated” are arbitrary designations that don’t really mean anything.  If I write a Python script that will generate some test data for me, am I now an automation engineer?  If I log into an application and click around for a while before I write a Selenium test, am I now a manual tester?  
2) The whole point of software testing- to put it bluntly- is to do as much as we can to ensure that our software doesn’t suck.  We often have limited time in which to do this.  So we should use whatever strategies we have available to test as thoroughly as we can, as quickly as possible.  
Let’s take a look at three software testers: Marcia, Cindy, and Jan.  Each of them is asked to test the Superball Sorter (a hypothetical feature I created, described in this post).  
Marcia is very proud of her role as a “Software Developer in Test”.  When she’s asked to test the Superball Sorter, she thinks it would be really great to create a tool that would randomly generate sorting rules for each child.  She spends a couple of days working on this, then writes a Selenium test that will set those generated rules, run the sorter, and verify that the balls were sorted as expected.  Then she sets her test to run nightly and with every build.  
Unfortunately, Marcia didn’t take much time to read the acceptance criteria, and she didn’t do any exploratory testing.  She completely missed the fact that it’s possible to have an invalid set of rules, so there are times when her randomly generated rules are invalid.  When this happens, the sorter returns an error, and because she didn’t account for this, her Selenium test fails.  Moreover, it takes a long time for the test to run because the rules need to be set with each test and she needed to build in many explicit waits for the browser to respond to her requests.  
Cindy is often referred to as a “Manual Tester”.  She doesn’t have any interest in learning to code, but she’s careful to read the acceptance criteria for the Superball Sorter feature, and she asks good questions of the developers.  She creates a huge test plan that accounts for many different variations of the sorting rules, and she comes up with a number of edge cases to test.  As a result, she finds a couple of bugs, which the developers then fix.
After she does her initial testing, she creates a regression test plan, which she faithfully executes at every software release.  Unfortunately, the test plan takes an hour to run, and combined with the other features that she is manually testing, it now takes her three hours to run a full regression suite.  When the team releases software, they are often held up by the time it takes for her to run these tests.  Moreover, there’s no way she can run these tests whenever the developers do a build, so they are often introducing bugs that don’t get caught until a few days later.
Jan is a software tester who doesn’t concern herself with what label she has.  She pays attention during feature meetings to understand how the Superball Sorter will work long before it’s ready for testing.  Like Cindy, she creates a huge test plan with lots of permutations of sorting rules.  But she also familiarizes herself with the API call that’s used to set the sorting rules, and she starts setting up a collection of requests that will allow her to create rules quickly.  With this collection, she’s able to run through all her manual test cases in record time, and she finds a couple of bugs along the way.
She also learns about the API call that triggers the sorting process, and the call that returns data about what balls each child has after sorting.  With these three API calls and the use of environment variables, she’s able to set up a collection of requests that sets the rules, triggers the sorting, and verifies that the children receive the correct balls.  
She now combines features from her two collections to create test suites for build testing, nightly regression testing, and deployment testing.  She sets up scripts that will trigger the tests through her company’s CI tool.  Finally, she writes a couple of UI tests with Selenium that will verify that the Sorter’s page elements appear in the browser correctly, and sets those to run nightly and with every deployment.
With Jan’s work, the developers are able to discover quickly if they’ve made any changes in logic that cause the Superball Sorter to behave differently.  With each deployment, Jan can rest assured that the feature is working correctly as long as her API and UI tests are passing.  This frees up Jan to do exploratory testing on the next feature.
Which of these testers came up with a process that more efficiently tested the quality of the software?  Which one is more likely to catch any bugs that come up in the future?  My money’s on Jan!  Jan isn’t simply a “manual tester”, but she isn’t a “software developer in test” either.  Jan spends time learning about the features her team is writing, and about the best tools for testing them.  She doesn’t code for coding’s sake, but she doesn’t shy away from code either.  The tools and skills she utilizes are a means to ensure the highest quality product for her team.  

What To Do When There’s a Bug in Production

There is nothing quite as bone-chilling to a software tester than the realization that a bug has been found in Production!  In this post, I’ll walk through a series of steps testers can take to handle Production bugs and prevent them in the future.

Step One: Remain Calm

Because we are the ones who are testing the product and signing off on the release, it’s easy to panic when a bug is found in Production.  We ask ourselves “How could this have happened?” and we can be tempted to thrash around looking for answers.  But this is not productive.  Our top priority should be to make sure that the bug is fixed, and if we don’t stay calm, we may not investigate the issue properly or test the fix properly.

Step Two: Reproduce the Issue

If the issue came from a customer, or from another person in your company, the first thing to do is to see if you can reproduce the issue.  It’s possible that the issue is simply user error or a configuration problem.  But don’t jump to those conclusions too quickly!  Make sure to follow any steps described by the user as carefully as you can, and wherever possible, make sure you are using the same software and hardware as the user: for example, use the same type of mobile device and the same build; or the same type of operating system and the same browser.  If you are unable to reproduce the issue, ask the user for more information and keep trying.  See How to Reproduce A Bug for more tips.

Step Three: Gather More Information

Now that you have reproduced the issue, gather more information about it.  Is the issue happening in your test environment as well?  Is the issue present in the previous build?  If so, can you find a build where the issue is not present?  Does it only happen with one specific operating system or browser?  Are there certain configuration settings that need to be in place to see the issue?  The more information you can gather, the faster your developer will be able to fix the problem.

Step Four: Understand the Root Cause

At this point, your developer will be working to figure out what is causing the bug.  When he or she figures it out, make sure they tell you what the problem was, and make sure that you understand it.  This will help you figure out how to test the fix, and also to determine what regression tests should be done.

Step Five: Decide When to Fix the Issue

When there’s a bug in Production, you will want to fix it immediately, but that is not always the best course of action.  I’m sure you’ve encountered situations where fixing a bug created new ones.  You will want to take some time to test any areas that might have been impacted by the fix.

When trying to decide when to fix a bug, think about these two things:

  • how many users are affected by the issue?
  • how severe is the issue?

You may have an issue where less than one percent of your users are affected.  But if the bug is so severe that the users can’t use the application, you may want to fix the bug right away.

Or, you may have an issue that affects all of your users, but the issue is so minor that their experience won’t be impacted.  For example, a misaligned button might not look nice, but it’s not stopping your users from using the application.  In this case, you might want to wait until your next scheduled release to fix the issue.

Step Six: Test the Fix

When you test the bug fix, don’t check it just once.  Be sure to check the fix on all supported browsers and devices.  Then run regression tests in any areas affected by the code change.  If you followed Step Four, you’ll know which areas to test.  Finally, do a quick smoke test to make sure no important functionality is broken.

Step Seven: Analyze What Went Wrong

It’s tempting to breathe a big sigh of relief and then move on to other things when a Production bug is fixed.  But it’s very important to take the time to figure out exactly how the bug got into Production in the first place.  This is not about finger-pointing and blame; everybody makes mistakes, whether they are developers, testers, product owners, managers, or release engineers.  This is about finding out what happened so that you can make changes to avoid the problem next time.

Perhaps your developers made a code change and forgot to tell you about it, so it wasn’t tested.  Perhaps you tested a feature, but forgot to test it in all browsers, and one browser had an issue.  Maybe your product owner forgot to tell you about an important use case, so you left it out of your test plan.

Whatever the issue, be sure to communicate it clearly to your team.  Don’t be afraid to take responsibility for whatever your part was in the issue.  See my post on Extreme Ownership for more details.

Step Eight: Brainstorm Ways to Prevent Similar Issues in the Future

Now that you know what went wrong, how can you prevent it from happening again?  Discuss the issue with your team and see if you can come up with some strategies.

You may need to change a process: for example, having your product owner sign off on any new features in order to make sure that nothing is missing.  Or you could make sure that your developers let you know about any code refactoring so you can run a regression test, even if they are sure they haven’t changed anything.

You may need to change your strategy: you could have two testers on your team look at each feature so it’s less likely that something will be overlooked.  Or you could create test plans which automatically include a step to test in every browser.

You may need to change both your process and your strategy!  Whatever the situation, you can now view the bug that was found in Production as a blessing, because it has resulted in you being a wiser tester, and your team being stronger.

Merge Conflict Resolution for the Confused

Anyone working with version control software such as Git will eventually come across a merge conflict.  If you are new to working with Git, here is a simple example of a merge conflict:

The master branch contains a file with this text:
Kristin Jackvony was here on May 22, 2019

Prunella and Joe each check out a version of this master branch.  Prunella makes a branch called “Prunella” and Joe makes a branch called “Joe”.  

Joe updates the file in his branch to read:
Kristin Jackvony was here on May 22, 2019
Joe Schmoe was here on May 23, 2019

Joe does a pull request for his file changes, and they are approved and merged into the master branch.

Shortly thereafter, Prunella updates the file in her branch to read:
Kristin Jackvony was here on May 22, 2019
Prunella Prunewhip was here on May 23, 2019

She also does a pull request for her file changes, but because she no longer has the latest version of the master branch, there is a merge conflict.  Git sees that she wants to add her name to the second line of the file, but her version of the master branch doesn’t have anything on the second line, whereas the new version of the master branch already has Joe’s name on it.  Prunella will need to resolve the conflict before her changes can be merged.

I’ll be honest; I really wanted to master resolving Git merge conflicts for this post. However, resolving them from the command line still confounds me!  Fortunately, I’ve found a number of different ways to handle merge conflicts so that they no longer fill me with dread.  Below I’ll discuss six steps for handling a conflict.

Step Zero: Avoid Having a Merge Conflict In The First Place

By using the helpful hints in last week’s post, especially tips one, two, and six, you can avoid creating a merge conflict.  Doing a pull from master is always a good idea before you do anything code-related.

Step One: Don’t Panic

When you have a merge conflict, it’s important not to thrash around trying “git this” and “git that”.  You might make things more confusing this way.  A merge conflict will eventually be solved, even if you have to resort to asking for help (Step Seven).  And you can’t possibly have done anything irreversible; that’s the beauty of version control!

Step Two: Resolve the Conflict from Within GitHub 

GitHub has an easy interface that will allow you to resolve merge conflicts.  Simply click the Resolve Conflicts button and observe the conflict.  It will look something like this:

>>>>>HEAD
Joe Schmoe was here on May 23, 2019
=====
Prunella Prunewhip was here on May 23, 2019
<<<<< Prunella

All you need to do is decide which entry you want to go on line 2, and which entry you want to go on line three, and make those edits, deleting the extraneous symbols and branch names along the way.  When you are done, your file will look like this:

Kristin Jackvony was here on May 22, 2019
Joe Schmoe was here on May 23, 2019
Prunella Prunewhip was here on May 23, 2019

Now click the Mark as Resolved button, and your pull request should be ready for merging.

Step Three: Resolve the Conflict from the Command Line

If you are using Git as a version control system, but you are not using GitHub to host your repositories, you may not have a nice UI to work with in order to resolve your conflict.  In this case, you may need to use the command line.  

What I do in this scenario is open the file with the merge conflict in a text editor such as TextEdit or Notepad++, I edit the file so that it looks the way I want it to, removing all the extraneous symbols and branch names, then I do a git add with the filename, and then a git commit.  I do this in accordance with the instructions found here: https://help.github.com/en/articles/resolving-a-merge-conflict-using-the-command-line.  However, I have only had success with this once or twice.  Usually I have to go on to Step Four or Step Five.

Step Four: Forget About Your Existing Branch and Make a New One

In this scenario, I copy the text of the entire file with all my changes, and I paste it somewhere outside the code.  Then I go to the master branch and do a git pull origin master. Now I should have all the latest changes.  Then I create a brand new branch, switch to that branch, and paste in all of my file changes.  Then I do a fresh new pull request, which won’t have a merge conflict.

Step Five: The Nuclear Option

If Step Four failed to work, I exercise the nuclear option, which is to delete the entire repository from my machine.  Before I do this, however, I make sure to make a copy of my file with all my changes and paste it somewhere outside the code, as I did in Step Four.  Then I delete the entire repository and clone it again.  Then I continue as I did in Step Four, creating a new branch and switching to it, making my changes, and doing a brand new pull request.

Step Six: Ask For Help

If all else fails, ask someone for help.  There’s no shame in asking for help with a particularly thorny merge conflict!  If you followed Step One and didn’t panic, your helper will be able to see exactly what you’ve done so far and can help you arrive at a solution.

Git purists may argue that merge conflicts should be resolved the right way (using Step Two or Step Three), and they are probably right.  But doing Step Four or Step Five can keep you from wasting time trying to resolve the conflict, and they also will keep you from defenestrating your laptop!

Six Tips for Git Success

Last week, I wrote a Gentle Introduction to Git, which was designed to give testers a general overview of how version control software works, from cloning a repository to submitting a pull request.  Even when you understand how Git works, it can still be a bit mysterious, because there is so much happening that you don’t see.  The command line does not offer a visual interface to show you what branch you are on, or when the last time was that you pulled from the master branch.  So in this post I’ll be describing six tips that make using Git easier.

Tip One:  Run git status frequently

A common mistake that Git users make is to do a bunch of work and commit while on the wrong branch.  Because I am not a Git expert and I’m never sure how to recover from this mistake, I run git status very frequently.  I always run it as soon as I’ve opened the command line and navigated to my repository, so I won’t be surprised by what branch I’m on. 

Running git status is also a good way to find out what files have been changed in your branch.  Sometimes we make a change to a file for debugging purposes and we forget to change it back.  If you run git status, it will alert you that a file you didn’t want to change has been altered.

Tip Two: Pull from the master branch before you do anything

Before you start changing code in your branch, you’ll want to make sure that your branch has the very latest code in it.  Otherwise, you may be updating code that doesn’t exist any more, or you may be duplicating work that has already been done.  By making sure that you have the latest code, you will also avoid creating a merge conflict, where your branch and the master branch have differences that the version control system doesn’t know how to handle. 

Once you have pulled from the master branch, remember to switch to your own branch!  If you are running git status frequently, you’ll notice if you’ve forgotten to make the switch.

Tip Three: Add all of your changed files at once

If you have changed a number of files, you’ll find it tedious to type
git add <insert long file name here>  over and over again.  A quick way to add all of your changed files at once is to use git add -A.  Just be sure when using this that you want to add all of your files.  I don’t know of a command that will let you add all the files except one or two.  If you know a command for this, please comment on this post and share it with everyone!

Tip Four: Name your commits something helpful

When you commit your files, adding a commit message is optional, but most companies expect their employees to do it.  Make it easier on yourself and everyone else by naming your commits something that will make sense later.  “One line code change” is not a very helpful commit message.  “Adding test for new contact info endpoint” provides much more detail. 

Tip Five: View your git logs in a single line

It can be hard to remember what you’ve done in Git, because there’s no UI to show you.  This is where git log is helpful.  The log in its full version will show you the last several commits made, who made them, and the date and time they were made.  I find it’s easier to read the logs when they are condensed to a single line; to do this, type git log –pretty=oneline. To exit the log, type q. 

Tip Six: View the diff of your files before you do a pull request

If you are running git status frequently, you probably won’t commit and push any files that you didn’t mean to push.  But it’s still possible to accidentally commit code you didn’t mean to commit in a file that has the changes you want.  So before you do a pull request and ask someone to review your code, view the “diff” of your files, which simply means looking in GitHub and comparing the files in your branch with the files in the master branch to see which lines were changed.  Make sure that there are no code changes in the file that you didn’t want to commit, such as commented-out code or debugging statements.

If you find that you’ve accidentally pushed something that you didn’t mean to, simply change the file to what you want it to be and add, commit, and push it again.

Hopefully you will find these six tips helpful as you work with Git!  In next week’s post, I’ll talk about how to handle the dreaded merge conflict.

A Gentle Introduction to Git

For a software tester who has just started writing test automation, using version control software such as Git can seem daunting and confusing.  But being able to pull down the latest code, update it, and submit a pull request is very important for any team project!  In this week’s post, I’ll provide a gentle introduction to the basics of Git.

What is Git?

Git is a version control system.  A version control system is a system that allows a group of people to collaborate on code without accidentally overwriting each other’s work.  It also allows the group to keep track of who changed the code and when it was changed, so it’s easy to trace back to the source of a problem.

Why is Git needed?

Consider what file editing is like when you don’t use a version control system.  Let’s say you have a recipe for brownies.  You send the recipe to your friend, and he decides to change the amount of cocoa in the recipe.  When he makes that change, it is only in his version of the file, not yours.  Your files are now different.  If you make a change to add more vanilla to the recipe, now your versions have diverged even further. 

You can see how this would be unacceptable for software code!  In a version control system, there is one “master version” which is the accepted version of the code.  This master version lives in GitHub (or another version control hosting service), and can be “pulled” down by any user.  When someone wants to make a change to the code, they pull down the master version of the code, create a “branch” that is a copy of the master version, make their changes to the branch, push the branch up to GitHub, and then do a “pull request”, which is asking for someone to review their code and merge it into the master branch.
Confused?  Don’t worry, this will look much simpler with an example.  Let’s imagine that we have a source code repository called “The Thinking Tester Guestbook”.  We’ll take a look at what would happen if Prunella Prunewhip wanted to add her name to the guestbook.

These instructions assume that Prunella has already installed Git on her computer, and has already created a GitHub account.)



Step One: Prunella clones the source code repository

This is often called “cloning the repo” or “pulling down the repo”.  Prunella does this by going to the URL in GitHub that has the source code, and clicking the green “Clone or download” button.  A dropdown appears with the URL she will need to clone the source code.  She clicks the little clipboard button to the right of the URL to copy the URL text.

Prunella opens up a command window and navigates to the folder where she would like put the source code.  Once she’s there, she types git clone and then pastes the URL text next to those words.  The repository is copied from GitHub into a new folder.

Now that the repository is in a folder on her computer, she can open the folder up in her file browser and take a look at what’s in there.  She sees that there is one text file, called “guestBook.txt”.  The text file reads:

Kristin Jackvony was here on May 11, 2019

Step Two: Prunella makes a new branch and adds her changes to that branch

Before Prunella makes any changes to guestBook.txt, she should create a new branch and switch to it.  So in the command line, she navigates to the new folder that was cloned earlier by typing
cd ThinkingTesterGuestBook.

She can verify that she’s in the master branch by typing git status, and she will get a response like this: On branch master.

Now she can create a new branch and switch to it by typing git checkout -b NewEntry.  The command “-b” tells Git to create a new branch.  “NewEntry” is what Prunella has chosen to name her branch.  And the command “checkout” is what causes Git to switch to the new branch.

If Prunella types git status at this point, she will get On branch NewEntry as a response.

Now that Prunella is in the correct branch, she’s going to make a change to the guestBook.txt file, by adding one line, so that the file now reads:

Kristin Jackvony was here on May 11, 2019
Prunella Prunewhip was here on May 13, 2019

Step Three: Prunella commits her changes and pushes them to GitHub

Now that Prunella has made the change she wanted, she needs to commit and push her change.  First, she can run git status and she’ll get this response: 

On branch NewEntry
modified: guestBook.txt

This shows that the guestBook.txt file has been modified. Next, Prunella needs to add the file to the commit, by typing git add guestBook.txt.  Now if she types git status, she’ll see this response:

On branch NewEntry
Changes to be committed:
     modified: guestBook.txt

Next, Prunella commits her change by typing git commit -m “Adding a new entry”.  The “-m” in this command stands for “message”.  The “Adding a new entry” text is the message that she is adding to explain what she is committing.  The command line will respond with how many files and lines were changed.

Once the change has been committed, Prunella can push the change up to the GitHub repository by typing git push origin NewEntry.  The “NewEntry” value explains that the code should go up to the NewEntry branch, which doesn’t exist yet in the GitHub repository, but it will be created with this command.  “Origin” refers to the GitHub repository (this is also referred to as “remote”).  The command line will respond with several lines, the final line of which will be
* [new branch] NewEntry -> NewEntry, which shows that a new branch called NewEntry has been created in the origin, and that it was copied from the local branch Prunella created, which was also called NewEntry.

Step Four: Prunella creates a pull request in GitHub

Now that her new branch has been pushed up to GitHub, Prunella can submit a pull request to ask that her changes are merged with the master branch.  She does this by going to the GitHub repository and clicking the “New Pull Request” button.  This takes her to the “Compare” page.  She makes sure that the left side of the comparison is the master branch, and then she chooses the NewEntry branch from the branch dropdown.  She can see how the guestBook.txt file has changed; the new line she added is highlighted in green, illustrating the difference between the two files.  (If she had deleted a line, the line she removed would be highlighted in red.)  Finally, she clicks the “Create Pull Request” button.

Step Five: Prunella’s pull request is approved and merged

The final step in the file change process is that the owner of the repository (or any teammates who have approval permissions) will review the change, approve it, and merge it.  Now if Prunella changes to the master branch by doing git checkout master, pulls down the changes by doing git pull origin master, and takes a look at guestBook.txt file, she will see that her entry has been added:

Kristin Jackvony was here on May 11, 2019
Prunella Prunewhip was here on May 13, 2019

And that’s all there is to it!  In my next post, I’ll add a few more Git tips and tricks, but these steps should be enough to get you started with committing your own code to your team’s repository.

Seven Excuses Software Testers Need to Stop Making

Last summer, I read an interesting book called Extreme Ownership.  Written by two Navy SEAL officers, it describes the concept of taking responsibility for every facet of your job, even those things that you feel that you have no control over.  If one of their soldiers made a mistake, the officers would take responsibility, because they could have trained the soldier better.  If their commander made a poor decision, the officers would take responsibility for that as well, because they could have “managed up” and provided information that would have led to a better decision.  When everyone exercises Extreme Ownership, a culture of excellence and achievement is the result.

Extreme Ownership can be applied to any career, including software testing!  Yet, there are a number of excuses that I often hear software testers make.  Excuses keep us from taking full ownership over our work, and keep us from being taken seriously.  Below are eight excuses that software testers need to stop making.

Excuse #1: I don’t know how the feature works

All too often, testers simply follow the meager directions left for them by the developer in the software story, without have any idea what they are doing.  For example, “Run this SQL query and verify the result is 1”.  Why?  What information is this query obtaining?  How do you know that this answer is the right one?  If it turns out there is a bug related to this feature, how can you possibly say that you’ve tested it?

When you are presented with a story to test that you don’t understand, start asking questions.  If the developer can’t explain the feature to you, find someone who can.  Restate the information that you are given to make absolutely sure that you understand it correctly.  Ask for the information to be presented in a way that makes sense to you.  I am a visual learner, and the developers on my team know that if I don’t understand what they are trying to explain to me, it’s time for them to draw me a diagram.

There have been many times where I have uncovered bugs in a feature even before I’ve started testing, simply by asking questions about how the feature works!

Excuse #2: There’s no way to test the feature

Really?  There’s NO way to test the feature?  How does the developer know that the feature is working then?  Are they just sending it to you and hoping for the best?  There must be SOME way for your developer to know that their code is working.  What is that way?  Can they show it to you?

There have been some features that I was unable to test myself because I didn’t have access to the back-end system that was being used in the feature.  When this is the case, I make sure to work with the developer and have them show me that the feature is working.  Then I can ask them to try various test cases while I watch, so we are effectively pair testing the feature.  In this way, we can uncover any bugs that may exist.

Excuse #3: The developer coded it wrong

I have sometimes seen instances where a developer misunderstood the requirements of the feature to be built and created it incorrectly.  This is why it’s important for everyone on the team to understand the requirements and to see to it that acceptance criteria are included in the story.  If you test the story based solely on what the developer tells you, and don’t verify exactly what was supposed to be built, then the fault lies with you.  You are the tester- usually the last line of defense before the product goes to the customer.  Make sure the customer is getting the right thing!

Excuse #4: The other tester on my team missed the bug

Even the best of software testers misses a bug now and then.  That’s why it’s important to have at least two sets of eyes on every feature.  It’s the policy on my team that when one tester is finished testing a feature, another tester tests the feature in the next environment.  The week after I instituted this policy, one of my co-workers found two bugs I missed!

If you are the only tester in your company, set up a “bug hunt” where everyone in the company looks for bugs.  Don’t be embarrassed if someone finds something you missed; when we test the same thing over and over again, we can sometimes develop inattentional blindness.

Excuse #5: There wasn’t enough time to test

Let’s face it: there will never be enough time to test everything that you want.  Software developers have time constraints too; they would probably really like to refactor their code a few more times before they hand it over to you, but they are working with a deadline just as you are.  So instead of making excuses, test the most important things, and manage your time wisely.

Excuse #6: If I log the bug I found in Production, I’ll be asked why I didn’t find it sooner

This was a new one to me when I heard it a couple of months ago.  There may be some managers who blame testers for finding bugs, but these managers are misguided.  It’s up to us to educate our team about what testers do.  We simply can’t find every bug; there are just too many ways that software can go wrong.  What we can do is report what we find as soon as we find it, and keep an eye out for similar bugs next time. If you don’t report that bug in Production, it will go unfixed, and the next person to find it will be a customer, or the CEO of your company!

Excuse #7: I don’t know how to code

Software development has changed significantly over the last two decades.  Companies used to release software every six months, so testers had tons of time to do regression testing.  Now software is released every week or two.  It’s simply not possible to manually test an entire application during that time frame.  This is why automation is necessary, and why you need to learn how to automate!

You don’t have to take a college course in Java to learn how to code.  All coding languages run on some very simple logical principles that are easy to understand.  The only tricky thing is the syntax of whatever language is being used, and the more you expose yourself to the code, the more you will understand.

If there’s no test automation at your company, see if you can get one of your developers to write some tests.  If you have software testers who are already writing automation at your company, ask them to walk you through their tests.  Learn how to make a simple change to an automated test, such as changing an assertion that says “true” to one that says “false”.  Copy a test that verifies the value of a text field, and see if you can change it so it verifies the value of a different text field.  Learn how your company’s version control system works, and see if you can submit a code change for your team’s approval.

Take small steps!  You don’t have to learn it all at once.  Think of learning code as learning a new language.  When you learn a new language, no one expects you to be fluent right away.  You learn a few phrases and keep using them, and you gradually add more.

Software testing is such a valuable profession, but too often companies take testers for granted.  By applying the principles of Extreme Ownership and eliminating excuses from your vocabulary, you will come to be seen as an indispensable asset to your company.

Time Management for Testers (and Everyone)

It’s a perennial problem: there’s so much testing to be done and not enough time in which to do it.  I’ve already written one post about this issue: What to Test When There’s Not Enough Time to Test, which talks about how to prioritize your testing and how to work with your team to avoid getting into situations where there’s not enough testing time.  But this week I’d like to take a more general view of time management: how can we structure our days so we don’t feel continually stressed by the many projects we work on?  Here are eight time management strategies that work for me:

Strategy One:  Know Your Priorities

I have a bi-weekly one-on-one meeting with my manager, and in each meeting he asks me “What’s the most important priority for you right now?”  I love this question, because it helps me focus on what’s most important.  You may have ten different things on your to-do list, but if you don’t decide which things are the most important, you will always feel like you should be working on something else, which keeps you from focusing on the task at hand.  I like to think about my first, second, and third priorities when I am planning what to work on next.

How do you decide what’s most important?  One good way is to think about impacts and deadlines.  If there is a release to Production that is going out tonight and it requires some manual testing, preparing for that release is going to be my top priority, because customers will be impacted by the quality of the release.  If I am presenting a workshop to other testers in my company, and that presentation is tomorrow, I’m going to want to make my preparation a priority.  If you evaluate each of your tasks in terms of what its impact is and what its due date is, it will become clearer how your priorities should be ordered.

Strategy Two: Keep a To-Do List

Keeping a to-do list means that you won’t forget about any of your tasks.  This does not mean that all of your tasks will get done.  When I finally realized that my to-do list would never be completed, I was able to stop worrying about how many items were on it.  I have found that the less I worry about how many things are in my to-do list, the more items I can accomplish.
I use a Trello board for my to-do list, and this is what I use for columns:  Today, This Week, Next Week, Soon, and Someday.  Whenever I have a new task, I add it to one of these columns.  If it’s urgent, I put it in Today or This Week.  Something less urgent will go in Next Week or Soon; the task will eventually move to This Week or Today.  Projects I’d like to work on or tech debt I’d like to address will go in Someday.  With this method, I don’t lose track of any of my tasks, and I always have something to do in those slow moments when I’m waiting for new features to test. 

Strategy Three: Use Quiet Times for Your Deepest Work

One of the best things about working for a remote-friendly company is that my team is spread over four time zones.  Since I am in Eastern Time, our daily meetings don’t start until what is late morning for me.  This means that the first couple of hours of my workday are quiet ones without interruption.  So I use those hours to work on projects that require concentration and focus.  My teammates on the West Coast do the opposite, using the end of their workday for their deepest work because the rest of us have already stopped for the day.

Even if you are in the same time zone as your co-workers, you can still carve out some quiet time to get your most challenging work done.  Maybe your co-workers take a long lunch while you choose to eat at your desk.  Or maybe you are a morning person and get into the office before they do.  Take advantage of those quiet times!

Strategy Four: Minimize Interruptions

It should come as no surprise to anyone that we are interrupted with notifications on our phones and laptops several times every hour.  Every time one of those notifications comes through, your concentration is broken as you take the time to look at the notification to see if it’s important.  But how many of those notifications do you really need?  I’ve turned off all notifications but text messages and work-related messages on my phone.  Any other notifications, such as Linked In, Facebook, and email, are not important enough to cause me to break my concentration.

On my laptop, I’ve silenced all of my notification sounds but one: the notification that I’m about to have a meeting.  I’ve set my Slack notifications to pop up, but they don’t make a sound.  That way I have fewer sounds disrupting my concentration.

Another helpful hint is to train your team to send you an entire message all at once, instead of sending messages like:

9:30 Fred: Hi
9:31 Fred: Good Morning
9:31 Fred: I have a quick question
9:33 Fred: Do you know what day our new feature is going to Production?

If you were to receive the messages in the above example, your work would be interrupted FOUR TIMES in the course of four minutes.  Instead, ask your co-workers to do this:

9:33  Fred: Good Morning!  I have a quick question for you: do you know what day our new feature is going to Production?

In this way, you are only interrupted ONCE.  You can answer the question quickly, and then get back to work.

Strategy Five: Set Aside Time for the Big Things

Sometimes projects are so big that they feel daunting.  You may have wanted to learn a new test automation platform for a long time, but you never seem to find the time to work on it.  While you know that learning the new platform will save you and your colleagues time in the long run, it’s not urgent, and the course you’d like to take will take you ten hours to complete.

Rather than trying to find a day or two to take the course, why not set aside a small amount of time every day to work on it?  When I have a course to take, I usually set aside fifteen or twenty minutes at the very beginning of my workday to work on it.  Each day I chip away at the coursework to be done, and if I keep at it consistently, I can finish a ten hour course in six to eight weeks.  That may seem like a long time, but it’s much better than never starting the course at all!

Strategy Six:  Ask For Help

We testers have a sense of personal pride when it comes to the projects we work on.  We want to make sure that we are seen as technologically savvy, and not “just a tester”.  We take pride in the automation we write.  But the fact is that developers usually have more experience working with code than we do, and they might have ideas for better or more efficient ways of doing things.

Recently, I was preparing an example project that I was going to use to teach some new hires how to write unit tests.  It was just a simple app that compared integers.  I knew exactly how to write the logic, but when I went to compile my program, I ran into a “cannot instantiate class” error.  I knew that the cause of the error was probably something tiny, but since I don’t often write apps on my own, I couldn’t remember what the issue was.  I had a choice to make at this point- I could save my pride and spend the next two hours figuring out the problem by myself, or I could ask one of my developers to look at it and have him tell me what the problem was in less than ten seconds.  The choice was obvious: I asked my developer, and he instantly solved the problem.

However, there is one caveat to this strategy: sometimes we can get into the habit of asking for information that we could easily find ourselves.  Before you interrupt a coworker and ask them for information, ask yourself if you could find it through a simple Google search or looking through your company’s wiki.  If you can find it yourself faster than it would take to ask your coworker and wait for his or her response, you’ve just saved yourself AND your coworker some time!

Strategy Seven:  Take Advantage of Your Energy Levels

I am a morning person; I am the most energetic at the start of my workday.  As the day moves along, my energy levels drop.  By the end of the work day, it’s hard for me to focus on difficult tasks.  Because of this, I organize my work so that I do my more difficult tasks in the morning, and save the afternoon for more repetitive tasks.

Your energy levels might be different; if you think about what times of the day you do your best and worst work, you will be able to figure out when you have the most energy.  Plan your most challenging and creative work for those times!

Strategy Eight: Adjust Your Environment

I am very fortunate in that I am able to work remotely.  This means I have complete control over the cleanliness, temperature, and sound in my office.  You may not be so lucky, but you can still find ways to adjust your environment so that you work more efficiently.  If your office is so warm that you find yourself falling asleep at your desk, you can bring in a small fan to cool the air around you.  If you are distracted by back and shoulder pain that comes from slumping in your chair, you can install a standing desk and stand for part of your workday.  If your coworkers are distracting you with their constant chit-chat, you can buy a pair of noise-cancelling headphones.

Experiment with what works best for you.  What works for some people might not be right for you.  You might work most efficiently with total silence, with white noise, with ambient music, with classical music, or with heavy metal playing in your ears.  You might find that facing your desk so that you can look out the window helps you relax your mind and solve problems, or you might find it so distracting that you are better off facing empty white walls.  Whatever your formula, once you’ve found it, make it work for you!

The eight strategies above can make it easier for testers to manage their time and work more efficiently.  You may find that these strategies help in other areas of your life as well: paying bills and doing house work, home improvement projects, and so on.  If these tips work for you, consider passing them on to others in your life to help them work more efficiently too!