Testing Buttons

Buttons tend to be something that it’s easy to forget about.  The “Save” button is so ubiquitous that it seems like it would just work.  But overlooking testing buttons on a page can also mean overlooking bugs.  Recently someone told me about new functionality she was testing on an existing web page.  The new feature worked great, but her team forgot to test the “Delete” button.  It turned out that the developers had forgotten to account for the delete action in their new feature, and now “Delete” did nothing! 

Here are a few things to think about when testing buttons:

1. Test the “happy path” of the button.  Usually buttons have some sort of message on them that tell you what they are supposed to do.  So try it out, and make sure that the button delivers on its promise!  Did the “Save” button really save your data?  Did the “Delete” button delete it?  Did the “Clear” button clear it?  Did the “Search” button execute a search?  (Note: a “Back” button is also common, but back buttons are tricky enough that I will be saving them for a separate post.)

2. Misuse the button.  For example, quickly press the “Save” button twice when adding data.  Were two records saved instead of one?  Does a button get confused when you press it twice?  What about when you quickly press one button and then another?  One of the most amusing bugs I have found in my career was a “Refresh” button that redrew the screen, making the button larger whenever I pressed it.

3. Is the button there when it’s supposed to be?  When we have spent a lot of time testing an application, it’s easy to get used to the page and not notice when things are missing.  Think about your user stories when looking at the page.  What would your user need when doing certain activities on that page?  Follow the path that the user would take, and check for the buttons as you go.

4. Think about when the button is enabled and when it is disabled.  Does it make sense?  For example, is the “Save” button only enabled when all the required fields on your form have been filled out?  Is the “Clear” button only enabled when a field has been dirtied?  How do you know the button is enabled?  Can you tell by looking at it?  Does the button look enabled when it isn’t?  Does the button look disabled when it’s really active?  What happens when you press the button and it’s not enabled?  Do the enabled and disabled rules for the button make sense?

5. Finally, see if you can hack your buttons!  For example, if you have a “Save” button that is disabled on your form because some of the required fields are missing, can you edit the html on the page so that it is enabled, and can you then use the button?  If you view the html and you see that there is a hidden button on the page, can you make it visible and active?  At best, that bug is an annoyance that will get inaccurate data into the database.  At worst, the bug represents a way that a hacker can infiltrate your system!  Imagine that you have a button that should only be visible and enabled if a user is an administrator.  If a malicious user can make that button appear on the screen and be active, they will have access to pages or features that only an admin should have. Your developer should include checks whenever a button is clicked on to make sure that the user has the rights to do what the button does. 

If you have never edited the html on a web page while testing, here are a few simple instructions for the Chrome browser:
a. Click on the three-button menu on the top right of your screen, and choose “More Tools -> Developer Tools”.  A new panel will appear on the bottom or the right side of your screen.
b. Right-click on the button you want to test, and click “Inspect”.  In the developer tools panel, you will now see the html for that button highlighted.
c. Right-click on that highlighted text and choose “Edit as HTML”.  An editable text window will open up. 
d. If you see text such as “‘disabled’=disabled” delete the text.  Click away from the editable field, and see if your button is now enabled. If it is, click on it and see what happens!
e. To find hidden buttons, look at the html in the developer tools, and use the search bar to search for “button”.
f. If you find a button with the markup “ng-hide”, try changing it to “ng-show”.  See if you can get the button to appear on the page!

Buttons are one of the most important things to test.  Just imagine a user’s frustration if the button they are trying to use is disabled, or doesn’t do what they expect it to.  By being diligent in our testing, we can ensure that our users will want to work with our application. 

Testing Phone Fields- Part II- International Phone Numbers

If I were to be given three wishes for the world, my third wish (after world peace and ending world hunger) would be to standardize international phone numbers, because testing them is so complicated!  Let’s take a look at some phone number patterns from around the world:

Mexico:
Phone numbers are ten digits, plus an area code of either two digits or three digits.  So the local number will be either twelve or thirteen digits.

Italy:
Landline numbers are generally nine to eleven digits, but some can be as short as six.  Mobile numbers are usually ten digits, but there are also some nine-digit numbers.

Japan:
Telephone numbers in Japan have an area code, an exchange number, and a subscriber number.  Area codes can have between two and five digits.  Generally the length of the entire number is limited to nine digits, so if the area code is longer, the exchange and subscriber numbers will be shorter.

Now let’s think about international calling codes.  Calling codes can range from one digit (such as the US’s +1) to three digits (such as +351 for Portugal).  So between the international calling codes and the phone number itself, there is a wide range of the number of digits that might be expected for a phone number.

Finally, let’s think about number separation.  While US numbers are always separated in a 3-3-4 pattern, in other countries the numbers can be grouped in various ways.  Here are just some of the ways that phone numbers are grouped in England:

(xxx) xxxx xxxx
(xxxx) xxx xxxx
(xxxxx) xxxxxx
(xxxx xx) xxxxx

Therefore, it is extremely difficult to validate on whether the pattern of numbers, parentheses, and dashes entered by a user is correct.

So how on earth are we to validate international phone numbers?  The most important thing to do is to make sure that the developer doesn’t try to come up with a validation regex on their own- that will make them (and you) go crazy!  Fortunately an international phone number formatting standard called E.164 has been developed.  Many companies such as Microsoft and Google have come up with regex patterns that can be used for E.164 validation.  For testing, there are free websites that will let you know if an international phone number is valid.  Most of these sites will require some sort of registration.  You can come up with a list of international numbers to test, verify with the website to see if they are valid, and then try them in your application.  For negative testing, simply use a sampling of invalid numbers and verify that you get an appropriate error message.

What format should a user use when entering in their phone number?  Some users might want to put parentheses, spaces, or dashes into their number.  Then there is the plus sign, which is used to indicate a country code.  For simplicity’s sake, it would be best to expect nothing but digits and a plus sign.  It would also be good to prompt the user for what format the application is expecting.  For example, in the United States you might have a message such as: “Enter your phone number with no dashes, parentheses, or spaces.  For numbers outside the US, a plus sign (+) and the country code should be used.”  Then for validation, if the number does not have a plus sign, simple US number validation rules can be applied, and if the number does have a plus sign, international validation rules can be applied.

Similarly, you may want to store the number in the database with the plus sign, to indicate that the number is an international one.  How should the numbers be displayed when retrieved in the application?  For many countries it’s nearly impossible to know how the numbers should be spaced, unless the validation code you are using returns that to you.  If your country has a standard number formation, you can display the number in that standard.  If your country does not have a standard formation, or if you are displaying a number from outside your country, you can simply display the number as is, such as: +442073238299.

When it comes to international phone numbers, there really are no easy answers, which is why I am wishing for world standardization!  There are other ways to (mostly) solve the problem, such as 1) expecting only between 6 and 20 digits, not validating the number beyond making sure that it’s within that number of digits, and displaying the number as entered, or 2) having a separate field for the country code, validating the number for that specific country, and formatting the number according to that validation.  What is most important is that you communicate closely with the developer to decide on a clear strategy and test it accordingly.

Testing Phone Fields- Part I- US Phones

Phone number fields are without a doubt the most time-consuming thing I have ever tested.  This seems so unlikely- phone numbers are standardized to ten digits in the US, so how can they cause so many headaches?  The answer is that, as we saw with Zip codes, there are so many ways to get them wrong.

If you are fortunate enough to be testing a new application without any existing data, you can fend off most phone number headaches by clearly establishing validation rules for phone number fields.  The easiest type of validation is one that accepts ten digits only, and no other characters.  The developer can rely on UI formatting to put the number in an easily-readable format rather than expecting the user to do it themselves.  The reason why this is the best course of action is that allowing the user to add in formatting creates all sorts of ambiguity.  If my phone number is 800-867-5309, how might I type that in?  Just with dashes, or will I use parentheses, such as (800) 867-5309?  What about the older format of 1-800-867-5309?  With that format, I now have eleven digits.  
Many developers try to accommodate all styles in the mistaken notion that this will make things easier for the user, but this actually makes things much more complicated.  If both parentheses and dashes are allowed, there needs to be some sort of regex used for the validation.  Otherwise, a user could type in something like ((800)–867 5309 and the number will be accepted.  If the developer you are working with has decided that he or she would like to make it easy on the user by accepting parentheses and dashes, try entering values like 800–867–5309, 800(867)5309, and 8-0-0-867-5309.  After you have demonstrated that it’s possible to enter a badly formatted phone number, the developer’s strategy may change.

As with Zip codes, it’s important to consider how the phone number is saved to the database.  The best strategy for US phone numbers is to strip out any formatting and save the number as just the ten digits.  That way the number can be displayed in any way the UI designer deems appropriate.

Next, it’s time to think about how bad phone numbers will be displayed to the user.  If a number has been saved with formatting, will it be displayed with that formatting?  Will some attempt be made to strip the formatting?  Will it be displayed at all?  I have seen all kinds of bad numbers in databases.  One of my favorites was “dad’s office number”.  No one can format that!  A good strategy would be to format any ten-digit number by stripping out all of the non-number characters, and then adding in whatever format is desired for display.  If the phone number is eleven digits and starts with a 1, the leading 1 could be stripped off and then the number could be formatted for display.  With any other values, the “number” could be displayed as-is, or it could not be displayed at all.  Whatever is decided, it’s important to test with whatever bad numbers are currently in the database. 

Another wrinkle not often considered is phone extensions.  Many offices still use extensions to direct a call to a specific person.  Sometimes phone fields allow extensions as part of the number, but this is a bad idea, for the same reasons that allowing the user to format their own number is a bad idea.  Phone extensions can be any number of digits, so allowing them as part of the phone number means that now the validation can’t expect a specific number of digits.  Also, how might the user indicate that an extension is part of the number?  Will they enter 800-867-5309 ext. 1234?  800-867-5309×1234?  Allowing these types of variations will mean allowing letters and other characters, which it makes it more difficult to validate.  A far better solution is to include a separate field for an extension.  If your developer expresses interest in including the extension as part of the phone field, some testing with entries like 8008675309extextext…1234 will probably dissuade them. 

Finally, remember to test editing phone numbers, especially numbers that are in an incorrect format.  What will happen when the user has a phone number like 800-867-530 and tries to edit another field on the form?  Will they be notified upon saving that the number is incorrect and needs fixing?  This can be a good way to involve users in cleaning their own data.  What should happen when the user tries to fix an invalid number?  Ideally they should be able to save the value with a new valid number. 

I hope that this post has emphasized the importance of having good phone field validation.  If you are fortunate enough to be testing a brand-new application with an empty database, taking good care to ensure that only valid phone numbers are saved to the database, you will prevent many testing headaches in the future! 

This post only tackled United States phone numbers.  In my next post, I will take on the even greater challenge of international phone numbers! 

Testing Date Fields

Date fields are another data type that seems simple to test.  After all, dates are standard throughout the world: there’s a month, a day of the month, and a year.  But as you will see below, there are many factors to consider and many scenarios to test.

There are three main areas to think about when testing a date field:

  • What format will be accepted?
  • How will the date be stored in the database?
  • How will the saved date be displayed?  

There are many ways to format a date when entering it into a form.  One very important factor to consider is whether the system is expecting an American format: month, day, year; or a European format: day, month, year.  For example, if I try to enter next Wednesday’s date (12/13/2017) into some forms, it won’t be accepted if the European format was what the form was expecting, since there is no 13th month.  And if I enter July 4 into this form as 7/4/18, it may be saved to the database as April 7th.  
Beyond the American/European formatting, there are still a myriad of ways to format dates.  For example, will all four digits of the year be expected, or is it possible to put in just the last two digits?  Will single-digit months and days be allowed, or is it necessary to precede those digits with zeroes, such as in 07/04/2018?  What about spelling out the month, as in December 13, 2017?  The developer should have a clear idea of what format or formats will be allowed, and should clearly communicate that to the user.  For example, a tooltip such as “mm/dd/yyyy” can be used inside the field to help the user know what format to use.  Regardless of whether the tooltip is in place, it is important to test with a number of different date formats to ensure that an appropriate error message is displayed when the expected format is not followed.  Also be sure that the date itself is checked for whether it is a valid date; for example, 2/30/2016 should never be allowed, nor should 18/18/18.  
Next, let’s consider how the date will be stored in the database.  Some developers, not knowing better, might store the date as a string, rather than as a datetime value.  This is nearly always a bad idea for a number of reasons.  If the accepted format changes over time, or if good validation was not in place in the past, there could be strings where the date is saved as “7/4/2017” and strings where the date is saved as “December 13, 2017”.  This will make it difficult to consistently display the data.  Also, if the dates are stored as strings it will be difficult to sort correctly by ascending or descending date.  “2017-12-13” might wind up coming before “3/7/2017” in an ascending sort.  As testers, we should verify that the date is stored as a datetime value, and advocate for a change if it is not. 
It’s also important to look at how dates are displayed.  When a user calls up saved information, they should be able to read the dates easily. “2017-12-13T00:00:00” might be the way the date is saved in the database, but a user won’t be able to interpret this date quickly.  The developer or designer should decide what date format would be best for display purposes and should stick to it consistently throughout the application.  It’s also important to consider what should happen in the case of bad data.  What if there is a date saved in the database that says simply “December 13”?  Should it be displayed as 12/13/0000?  Should it not be displayed at all?  These are important scenarios to consider and test.  
There is one final consideration to make when testing date fields, and that is what the upper and lower limits of the date are.  For example, are dates in the future ever allowed?  What about dates from 100 years ago?  Remember that the future and the past change every day!  Let’s say that our application doesn’t allow dates in the future.  That means that today, 12/13/17 is not an allowed date in our application.  But by next Wednesday, 12/13/17 will be allowed.  And of course, you may be reading this blog post years in the future, at which time 12/13/17 will be a thing of the past!  

Testing Postal Codes

A text field with a postal code looks so simple, and yet it can be one of the most complex things to test on a form.  There are two important questions to ask before you start testing postal codes:

1. What countries does our application support?
2. What formats will we be accepting for the postal codes?
It always comes as a bit of a relief when I find out that the application I am testing will only be supporting US Zip codes.  Zip codes traditionally come in a five-digit format, such as 10012.  But there are also Zip +4 codes to consider, such as 10012-1234, and this is where the second question comes into play.  Will the application be accepting Zip +4 codes?  Will it be requiring the hyphen between the first five digits and the next four digits, or will a space be accepted as well?  What about just nine straight digits, with no hyphen or space in between?  If nine straight digits will be accepted, it’s important to also verify that 6, 7, and 8 digits will not be accepted.  
And there is another very important thing to test with US Zip codes: the leading zero.  There are many fields which will strip leading zeros off of a number upon submission.  It’s very important to submit some Zip codes with leading zeros and verify that they have been saved correctly to the database.  
The next likely postal code type you may encounter (if you are based in the US) is the Canadian postal code.  All Canadian postal codes are six characters in this pattern of letters and numbers:  A1A 1A1.  It’s important to clarify with the developer whether the space between the two groups of three characters will be expected, or whether you can submit the code with no space.  Hopefully the validation will expect the correct letter-number pattern, and will reject postal codes like 1A1 A1A.
What if your application expects both US and Canadian postal codes?  If this is the case, the third important question to ask is:
3. Will there be a separate validation pattern for each type?
If there is a separate validation pattern, the code may first look to see what country the address contains, and then use the appropriate validation.  In this case, it’s a good idea to test that you can’t choose “United States” for your country and then add “A1A 1A1” as the postal code.  Or the validation pattern may be chosen by the number of characters submitted.  If six or seven (including the space) characters are submitted, a Canadian validation pattern could be used.  If five, nine, or ten (including the hyphen) characters are submitted, a United States validation pattern could be used.  Understanding what validations the developer is using will allow you to craft appropriate test cases.  For example, in a scenario where only US and Canadian postal codes are used, there should never be a scenario where eight characters are accepted.  
When we move into postal codes for other countries, things can get more confusing.  Many countries have five-digit codes, which are easy to validate.  Other countries, such as Russia, have six-digit codes.  But consider Great Britain, which has postcodes in two sections: the first section can have between two and four characters, and the second section always has three characters. There is a space between the two sections, and the postcode will always start with a letter, not a number.   In this case, it’s best if the developer has ways to recognize if a British postcode is being used (perhaps by looking at what country has been entered in the form) and uses a separate validation regex for those codes.  For testing, be sure to try several different valid postcodes from various places in Great Britain, with two, three, and four characters in the first section.  You can also test with codes that have the right number of characters, but have the space in the wrong place, or with a code that has a number as the first character.
Finally, remember to check that the valid postal codes that you have submitted have been saved correctly to the database, and remember to verify that retrieved postal codes are displaying correctly.  Also be sure to check that any invalid codes in the database (that may have been added before the correct validation was in place) are displaying as added, or not at all, and that it’s possible to edit both invalid and valid postal codes.  
I hope that this post has helped you see that postal codes are not as simple as they seem!  Asking the three questions listed above can help you and your developer to recognize potential issues even before testing has begun.  

Automating CRUD Testing

There are a number of different patterns we can use to automate CRUD testing.  At the very minimum, we want to test one operation of each: Create, Read, Update, and Delete.  For the purposes of this discussion, let’s assume the following:

1. We are testing the simple form used in this post
2. We are doing UI automation (API automation will be discussed in a future post)

This is the pattern I like to use when testing CRUD.  I’m writing in Specflow/Cucumber for simplicity:

Scenario: Adding a user
Given I am adding a new user
When I add a first name and save
Then I navigate to the Users page
And I verify that the first name is present

Scenario: Updating a user
Given I am updating a user
When I change the first name and save
Then I navigate to the User page
And I verify that the first name has been updated

Scenario: Deleting a user
Given I am deleting a user
When I delete the first name and save
Then I navigate to the Users page
And I verify that the first name is not present

These three tests have tested Create, Update, and Delete. The first two tests are also testing Read, because we are retrieving the user for our assertions.  Therefore, with these three tests I’m testing the basic functionality of CRUD.

Some may argue that these tests are not idempotent.  Idempotency means that a test can be run again and again with the same results.  I can’t run the third test over and over again with the same results, for example, because after the first time I run it, the user is no longer there to delete.

If we wanted to solve the idempotency issue, we could write a test like this:

Scenario: CRUD testing of user
Given I am testing CRUD
When I add a first name and save
And I verify that the first name is present
When I change the first name and save
And I verify that the first name has been updated
When I delete the first name and save
Then I verify that the first name has been deleted

This one scenario tests all the CRUD functionality of the form.  But it also has three different assertions.  I prefer to keep my UI tests down to one or two assertions.

When my original three scenarios are looked at as a group, they are idempotent together.  My tests are responsible for creating and deleting my data.

It would also be a good idea to test some negative scenarios with our CRUD testing, such as creating a user with an invalid first name, and updating a user with an invalid first name.  These tests could look like this:

Scenario: Creating a user with an invalid first name
Given I am adding a new user
When I enter an invalid first name and save
Then I verify that I receive the appropriate error message on the page
And I navigate to the User page
And I verify that the user has not been added

Scenario: Updating a user with an invalid first name
Given I am updating an existing user
When I update the first name with an invalid value and save
Then I verify that I receive the appropriate error message on the page
And I navigate to the User page
And I verify that the existing first name has not been updated

The first scenario is idempotent, because nothing is actually being added to the database.  The second scenario is also idempotent, but it requires an existing user.  We could assume that our user will always be present in the database, but if someone inadvertently deletes it, our test will fail.  In this case, it would be good to add in a Before step and an After step that will create the user at the beginning of the test suite and delete it when the suite is over.

These five scenarios- the three to test the happy path, and the two negative tests- would be a great regression suite for our simple form.  This is a very simple form, with just one field, which is not exactly a real-world scenario.  But it is a good way to start thinking about automated UI test patterns.

CRUD Testing Part II- Update and Delete

In last week’s post, we started looking at CRUD testing.  As you recall, CRUD stands for Create, Read, Update, and Delete.  Last week we discussed testing Create and Update operations, and now we will continue by looking at Update and Delete.

In our discussion of the Read operation last week, I mentioned how important it is to test scenarios where the data in the database is invalid.  This is also true for Update operations.  Just because a text field is supposed to be required and have a certain number of characters doesn’t mean that’s how it is in the database!

Below is a matrix of testing scenarios for editing a text field.  Remember from last week that our hypothetical text field has the following validation rules:  it is a required field, it must have at least two characters, it must have 40 or fewer characters, and it should only have alphanumeric characters or hyphens and apostrophes; no other symbols are allowed.  As with the Create operation, be sure to test that the newly edited field is correct in the UI and in the database after the update.

For the bad values that you are testing with, be sure to vary them so you are covering a number of different validation scenarios.  For the good values that you are testing with, make sure to test hyphens and apostrophes, numbers and letters, and the upper and lower limits of the character count. 
Finally, let’s discuss Delete.  The main thing to test with Delete is that the value has been deleted from both the UI and the database.  But just as with Read and Update, you’ll want to make sure that you can delete bad values.  For example, if your First Name field has forty-one characters in it (violating the rule that only 40 characters or less should be added), you’ll want to make sure that you can delete it in the UI.  
You may be wondering how to find all these invalid values for testing.  While it is possible to find them by searching through the existing records, the easiest way to test them is to put them in yourself!  This is why it’s important to know the appropriate query language, such as SQL, for your database.  
For those who are wondering why testing with bad values is important, I once had a scenario where I was testing contact information for a person.  There were a number of incorrectly formatted phone numbers in the database.  When I tried to update any of the contact information for the person, I got the message that the record couldn’t be updated, because there was invalid data present.  This was true even if I was trying to update the invalid phone number in question!  Please be sure to check for scenarios like these; your end-user will thank you!

CRUD Testing Part I- Create and Read

In spite of its unappealing name, CRUD testing is extremely important!  CRUD stands for Create, Read, Update, and Delete.  As any tester knows, much of our testing involves these operations.  Today we’ll discuss the best ways to test Create and Read. 

The most important thing to know about testing CRUD is that it’s not enough just to rely on what you are seeing in your UI to confirm that a field’s value has been created or changed.  This is because the UI will sometimes cache a value for more efficient loading in the browser.  What you need to do to be absolutely sure that the value has changed is to check the database where your value is stored.  So you’ll be confirming that your value is set in two places: the UI and the database.  If you are doing API testing as well, you can actually confirm in three places, but we’ll save discussing API testing for another post.

Testing a Create Operation:

This text field looks similar to the one we looked at last week, but now we know what it does!  This is a form to add a user.  We’ll enter the first name of the user into the text field and click Submit.  Next, we’ll take a look at the Users page of our imaginary application and verify that the new user is present:

And there it is!  Finally, we need to query our database to make sure that the value has saved correctly there.  In our imaginary database, this can be done by running

SELECT * from Users

This will give us a result that looks a lot like what we are seeing in the UI, so I won’t include a screenshot here.

To thoroughly test the Create function, you can use some of the same ideas that we talked about in last week’s post.  Verify that valid entries of all kinds are saved to the database.

Testing a Read Operation:

We actually started testing the Read operation when we checked the Users page to verify that our new user was added.  But there is something else that is important to test!  We need to find out what happens when bad data is in the database and we are trying to view it in the UI.

Let’s take a look at what some bad data might look like in the database:

In our imaginary application, there are some constraints in the UI for the First Name field.  It is a required field, it must have at least two characters, it must have 40 or fewer characters, and it should only have alphanumeric characters or hyphens and apostrophes; no other symbols are allowed.  As we can see in our table, we’ve got lots of bad data:

  • User 2 has no entry for First Name
  • User 3 has an empty string for a First Name
  • User 4 is violating the rule that the name must have at least two characters
  • User 5 is violating the rule that the name must have 40 or fewer characters
  • User 6 is violating the rule that only hyphens and apostrophes are allowed for symbols
What should happen when we view the Users list in our application?  That will depend on what the product designers decide.  They may choose to display bad data as long as it is not a security risk, such as the First Name for User 6, which is actually a Stored XSS attack.  Whatever the rules for display are, it’s important to test that those rules are respected.  
You may be saying to yourself (or a developer may be saying to you), “Displaying bad data won’t be an issue, because we are putting good validation in place to make sure that bad data won’t get in the database to begin with.”  While this is absolutely standard practice today, there will always be cases where bad data will slip in.  I once tested a PATCH operation where phone numbers could be inserted into a record.  I discovered that while validation was taking place when the PATCH body was formed correctly, there was an edge case where a PATCH body was formed incorrectly and accepted without validation.  Any developer that incorrectly coded the PATCH operation could inadvertently allow bad phone numbers into the database!  
A good rule of thumb for testing any Create and Read operation is to assume that anything can go wrong and test accordingly.  We’ll continue imagining what can go wrong next week, when we test Update and Delete operations.  

Testing a Text Field

A text field in an application seems so ordinary, and yet it is one of the most important things we can test.  Why?  Because text fields provide an entryway into an application and its underlying database.  Validation on a text field is what keeps lousy data from getting into the database, which can cause all sorts of problems for end users and engineers. It can also prevent cross-site scripting attacks and SQL injection attacks. 

There are a myriad of ways to test a text field, and I will be outlining several in this post.  First, let’s imagine that we are testing the text field with absolutely no information about what it does:

  • Click Submit without filling in the text field
  • Press the space bar several times in the text field and then click Submit
  • See how many characters you can fit in the text field and then click Submit (an excellent tool to count characters is https://lettercount.com)
  • Fill the field with as many numbers as you can and then click Submit
  • Add a negative sign, fill the field with as many numbers as you can and then click Submit
  • Enter every non-alphanumeric field on the keyboard and click Submit.  If you get an error, see if you can narrow down which key (or keys) is causing the error.
  • Enter in non-ASCII characters and emojis and click Submit.  If you get an error, see if you can narrow down which symbol (or symbols) is causing the error.
  • Try cross-site scripting by entering in this script: <script>alert(“I hacked this!”)</script>  If on Submit, you get a popup message, then you know the field is vulnerable to cross-site scripting.
  • Try a SQL injection attack, such as FOO’); DROP TABLE USERS; — (Don’t try this on your Production database!)
Next, let’s assume that you have some knowledge about what is supposed to be entered into this text field and what the boundaries are on the data:
  • Try putting in a value that is a different data type from what is expected; for example, if this text field is expecting a value of currency, try putting in a string or a date
  • If the field is expecting a string, try putting in a string with one fewer characters than is expected, one more character than is expected, the lower limit of characters that is expected, the upper limit of characters that is expected, and twice the maximum number of characters expected
  • If the field is expecting a numeric value, try putting the maximum value, the minimum value, a value above the maximum, a value below the minimum, and a value twice the maximum value
  • If the field is expecting an integer, try submitting a value with a decimal point
  • If the field is expecting a float, try submitting a value with two decimal points and a value that begins with a decimal point
  • If the field is expecting a value of currency, try submitting a value with more than two digits after the decimal point
  • If the field is expecting a date, try putting in the maximum date, the minimum date, one day over the maximum date, one day before the minimum date, and a date one hundred years above or below the limit
  • For date fields, try entering a date that doesn’t make sense, such as 6/31/17 or 13/13/17  (There are many more ways to test date fields; I’ll touch on this in a later post)
  • If the field is expecting a time, try entering a time that doesn’t make sense, such as 25:15
  • If the field is expecting a phone number, try entering a number that doesn’t conform to the expected format (There are many, MANY more ways to test phone numbers; I’ll touch on this in a later post as well)
For all of the above tests, find out what sort of error message you are supposed to be receiving, and verify that you are getting the correct message.
Finally, let’s think about automation.  Assuming you have very thoroughly tested your text field manually, it’s probably not necessary to automate every single one of your tests.  Moreover, most forms have more than one text field, and having many tests for each individual field could result in many time-consuming tests.  But here are some suggestions for what you might want to automate:
  • submitting a null value
  • submitting an empty string 
  • submitting a value that meets the criteria (the “happy path”)
  • submitting the maximum number of characters or maximum value
  • submitting the minimum number of characters or minimum value
  • submitting just above the maximum characters or value
  • submitting just below the minimum characters or value
This list of tests is not meant to be completely exhaustive; it’s just a way to get you to start to think about the vast number of tests you can run on a single field.  Don’t ever assume that the developer who coded the field has put in the appropriate validation; check it for yourself!  I tested a date field once where there was a limit put on the year that could be entered so it couldn’t be before 1900 or after the present year.  While I did receive the appropriate message when I entered a value of 1880, I discovered that I could enter a date from the year 1300!

Think Like a Tester

Beginning with this week’s post, my blog will be taking on a new focus!

I have renamed it from Fearless Automation to Think Like a Tester (for the moment, the URL will remain the same). There were three recent events that made me decide to shift my focus:

  • I attended a large international computing conference where there was not a single workshop or presentation focused on software testing. 
  • At this conference, I met computer science students who asked me if there were any college classes to learn to be a tester.
  • I interviewed a QA engineer who was able to create a great automated testing solution for a website, but could not think of simple manual tests for the site. 

All of these things made me realize the following:

  • There aren’t enough people talking about testing software
  • There aren’t enough resources to learn about testing software
  • The testing community has been focused for so long on how to test software that we haven’t been thinking about what to test and why we are testing it 

Testing is truly a craft, and one that requires a different skill set from software development:

  • Rather than thinking of ways to make software work, testers think of ways to make software break
  • Rather than designing things to go right, testers think of all the ways that things can go wrong
  • Rather than focusing deeply on one feature, testers focus on how all those features integrate
  • Rather than solving a problem and moving on, testers come up with ways to continually verify that features are working 

In the weeks and months to come, I will be getting back to basics and discussing all areas of software testing- manual and automated- that require thinking like a tester. Hopefully both testing newbies and seasoned testers alike will find this knowledge helpful!