Think Like a Tester

In last week’s blog post, we started taking a look at the command line, and how it’s possible to navigate through your computer’s file system using some very simple commands.  This week we’ll learn how to use the command line to create and remove folders and files.  We’ll be building upon the knowledge in last week’s post, so if you haven’t read that, I recommend starting there.

When I first started out as a software tester, I would always get nervous when I had to do anything with the command line, and I was so impressed when my coworkers could type tiny commands and get dozens of lines of text in response.  The one thing that helped me when learning the command line was a course I took in Linux.  I was confused for much of the course, but I did manage to learn some commands, and over the years I’ve been able to gradually expand my knowledge.

The command line is hugely helpful when you want to navigate through your system’s folder structure, create new folders or files, or execute runnable files.  In this post, I’ll be walking you through some simple commands that can help you get started using the command line like a pro.  Most of the commands I’ll be sharing will work in both Mac and Windows; when there are differences between the two, I’ll point them out.

First, let’s look at some useful keys:

The up arrow
The up arrow copies whatever command you just ran, and if you click on the up arrow more than once, you can cycle back through all of the commands you have run so far.  For example, if you ran these three commands:
ls
cd Documents
cd ..
and then you were to click the up arrow, you’d see cd .. . If you clicked the up arrow again, you’d see cd Documents, and if you were to click it a third time, you’d see ls.

The up arrow is really helpful for those times when you ran a complicated command and you need to run it again, but you don’t feel like typing it all over again.  Simply click the up arrow until you’ve returned to the command you want, then click Return to run the command again.

The tab key
The tab key has auto-complete functionality.  To see how this works, let’s imagine that you have a folder called MyFolder that contains three sub-folders:
LettersToDad
LettersToMom
graduationPics
If you wanted to navigate from MyFolder to graduationPics using the cd command (more on this later), you could simply type:
cd grad
and then click the tab key.  The folder name will auto-complete to graduationPics.

This command is helpful when you don’t feel like typing out an entire folder name.  Typing just the first few letters of the folder and hitting tab, then Return, is a really fast way to navigate.

In order for the auto-complete to work, you need to type enough letters that there’s only one possible option left when you click the tab key.  For example, when you type
cd LettersTo
and then click the tab key, the command line doesn’t know if you mean LettersToDad or LettersToMom.  The Windows command line will cycle through the possible options as you repeatedly click the tab key.  In Mac, if you click the tab key a second time, it will return your possible options.

Next, let’s learn some navigation skills:

The command prompt:  The command prompt is a symbol that indicates that the command line is ready to receive commands.  In Mac, the command prompt looks like this: $.  In Windows, the command prompt looks like this: >.  The command prompt is preceded by the working directory.

Working directory:
The term working directory refers to whatever directory (folder) you are in when you are using the command line.  When you first open the command line window, you’ll be in your home directory.  This is your own personal directory.  For example, in Windows, my personal directory is C:/users/kjackvony.  In Mac, my personal directory is /Users/K.Jackvony, but the directory location will display only as ~ , which means the home directory.

ls or dir
This command – ls in Mac and dir in Windows – will list all the files and folders in your working directory.

cd <folder name>
This command will change your working directory to the folder you specify.  For example, if you said cd Documents, your working directory would change to the Documents folder.

cd ..
This command moves you up one level in the directory.

Let’s look at an example.  I am using a Mac, so I’ll use ls rather than dir.

1. I begin in my home directory:
~$

2. I type ls to see what’s in my home directory, and I get this response:
Desktop
Documents
Pictures
Projects

3. I type cd Documents, and my working directory is now the Documents folder:
Documents$

4. I type ls to see what’s in my Documents folder, and I get this response:
Blog Post Notes
Images for Testing
ProfilePhoto.jpg

5. I type cd “Blog Post Notes” (I’m using quote marks because the directory name has spaces in it), and my working directory is now the Blog Post Notes folder:
Blog Post Notes$

6. I type cd .. and I’m taken back to the Documents folder:
Documents$

7.  I type cd .. again and I’m taken back to my home folder:
~$

Now that you see how the example works, take some time to try it in your own command line, with your own folders!  Remember that if you are using Windows, your prompt will look like a > rather than a $, and you’ll want to type dir instead of ls to see what’s in your working directory.

Next week we’ll continue the command line adventure by learning some more navigation skills and how to create our own files and folders.

I love New Year’s Day!  There’s something exciting about getting a fresh start and imagining all that can be accomplished in the coming year.  The new year is an opportunity to think about how we can be better testers, how we can share our knowledge with others, and how we can continue to improve the public perception of the craft of software testing.

Here are some suggestions for resolutions you could make to improve your testing and the testing skills of those around you:

Speak Up
Because testers are sometimes made to feel like second-class citizens compared to software developers, they might feel timid about voicing their opinions.  But testers often know more about the product they test than the developers, who are usually working in one small area of the application.  This year, resolve to speak up if you see an issue with the product that you think would negatively impact the end user, even if it isn’t a “bug”.  Similarly, speak up if you find a bug that the team has dismissed as unimportant, and state why you think it should be fixed.  Advocate for your user!  Make sure that the product your customers are getting makes sense and is easy to use.

Pay Attention in Product Meetings
I’m sure my Product Owner would be sad to read this (sorry, Brian!) but I find product meetings boring.  I know that the small details of the user’s experience are important, and I’m so glad that there are people who care about where a notification badge is displayed.  But listening to the discussion where that decision is being made is not very exciting to me.  However, I am so glad that I am included in these meetings, and every year I resolve to pay more attention to product decision-making than I did the year before, and to contribute when I have information that I think will be helpful.  Attending product meetings allows me to hear why certain choices are made, and also helps me think about what I need to test when a new feature comes available.

Do Some Exploratory Testing
I suspect that most of us have some area of the application we test where we have a sneaking suspicion that things aren’t working quite right.  Or there’s a really old area of the application that no one knows how to use, because the people who initially built and tested it have since left the company.  But we are often too busy testing new features and writing test automation to take the time to really get to know the old and confusing areas of an application.  This year, resolve to set aside a few hours to do exploratory testing in those areas and share your findings with the team.  You may find some long-buried bugs or features that no one knows about!

Streamline Your Operation
Are there things your team does that could be done more efficiently?  Perhaps you have test automation that uses three different standards to name variables, making the variable names difficult to remember.  Perhaps your methods of processing work items isn’t clear, so some team members are assigning testing tickets while others are leaving them for testers to pick up.  Even if it seems like a small problem, these types of inefficiencies can keep a team from moving as quickly as it could.  Resolve to notice these issues and make suggestions for how they can be improved.

Learn Something New
This year, learn a new tool or a new language.  You don’t have to become a master user; just learn enough to be able to say why you are using your current language or tool over the new one you’ve learned.  Or you could discover that the new language or tool suits your needs better, in which case you can improve your test automation.  Either way, learning something new makes you more employable the next time you are looking for a new position.

Share Your Knowledge With Your Team
Don’t be a knowledge hoarder!  Your company and your software will be better when you share your knowledge about the product you are testing and the tools you are using to test it.  Sometimes misguided people hold on to knowledge thinking it will make them indispensable.  This will not serve to keep you employed.  In today’s world, sharing information so that the whole team can be successful is the best way to be noticed and appreciated.  Resolve to hold a workshop for the other testers on your team about the test automation you are writing, or create documentation that shows everyone how to set up a tricky test configuration.  Your teammates will thank you!

Share Your Knowledge With the Wider World
If I had one wish for software testers for the year 2020, it would be that we would be seen by the wider tech community as the valuable craftspeople we are.  If you are an awesome software tester- and I’m guessing you are because you are taking the time to read a blog about testing- share your skills with the world!  Write a blog post, help someone on Stack Overflow, or present at a local testing meetup.  You don’t have to be the World’s Most Authoritative Expert on whatever it is you are talking about, nor do you have to be the Best Speaker in the World.  Just share the information you have freely!  We will all benefit from your experience.

What New Year’s resolutions do you have for your software testing?  Please share in the comments below!

Time is the one thing of which everyone gets the same amount.  Whether we are the CEO of a company or we are the intern, we all have 1440 minutes in a day.  I’ve often heard testers talk about how they don’t have enough time to test, and that can certainly happen when deadlines are imposed without input from everyone on the team.  I’ve written a blog post about time management techniques for testers, but today I’m going to tackle the question:

Is it worth my time to automate this task?

Sometimes we are tempted to create a little tool for everything, just because we can.  I usually see this happen with developers more than testers, but I do see it with some testers who love to code.  However, writing code does not always save us time.  When considering whether to do a task manually or to write automation for it, ask yourself these four questions:

1. Will I need to do this task again?

Recently my team was migrating files from one system to another system.  I ran the migration tool manually and did manual checking that the files had migrated properly.  I didn’t write any automation for this, because I knew that I was never going to need to test it again.

Contrast this with a tester from another team who is continually asked to check the UI on a page when his team makes updates.  He got really tired of doing this again and again, so he created a script that will take screenshots and compare the old and new versions of the page.  Now he can run the check with the push of a button.

2. How much time does this task take me, and how much time will it take me to write the code?

Periodically my team’s test data gets refreshed, and that means that the information we have for our test users sometimes gets changed.  When this happens, it takes about eight hours to manually update all the users.  It took me a few hours to create a SQL script that would update the users automatically, but it was totally worth my time, because now I save eight hours of work whenever the data is refreshed.

But there have been other times where I’ve needed to set up some data for testing, and a developer has offered to write a little script to do it for me.  Since I can usually set up the data faster than they can create the script, I decline the offer.

3. How much time will it take to maintain the automation I’m writing?

At a previous job, I was testing email delivery and I wanted to write an automated test that would show that the email had actually arrived in the Gmail test account.  The trouble was that there could be up to a ten minute delay for the email to appear.  I spent a lot of time adjusting the automated test to wait longer, to have retries, and so on, until finally I realized it was just faster for me to take that assertion out of the test, and manually check the email account from time to time.

However, my team’s automated API smoke tests take very little time to maintain, because the API endpoints change so infrequently that the tests rarely need to change.  The first API smoke test I set up took a few days; but once we had a working model it became very easy to set up tests for our other APIs.

4. Does the tool I’m creating already exist?

At a previous company, the web team was porting over many customers’ websites from one provider to another.  I was asked to create a tool that would crawl through the sites and locate all the pages, and then crawl through the migrated site to make sure all the pages had been ported over.  It was really fun to create this tool, and I learned a lot about coding in the process.  However, I discovered after I made the tool that web-crawling software already exists!

But in that particular month I did have the time to create the tool, and the things I learned helped me with my other test automation.  So sometimes it may be worth “reinventing the wheel” if it will help you or your team.

The Bottom Line: Are you saving or wasting time?

All of these questions come down to one major consideration, and that is whether your task is saving or wasting time.  If you are a person who enjoys coding, you may be tempted to write a fun new script for every task you need to do; but this might not always save you time.  Similarly, if you don’t enjoy coding, you might insist on doing repetitive tasks manually; but using a simple tool could save you a ton of time.  Always consider the time-saving result of your activities!

The concept of measuring quality can be a hot-button topic for many software testers.  This is because metrics can be used poorly; we’ve all heard stories about testers who were evaluated based on how many bugs they found or how many automated tests they wrote.  These measures have absolutely no bearing on software quality. A person who finds a bug in three different browsers can either write up the bug once or write up a bug for each browser; having three JIRA tickets instead of one makes no difference in what the bug is!  Similarly, writing one hundred automated tests where only thirty are needed for adequate test coverage doesn’t ensure quality and may actually slow down development time.

But measuring quality is important, and here’s why: software testers are to software what the immune system is to the human body.  When a person’s immune system is working well, they don’t think about it at all.  They get exposed to all kinds of viruses and bacteria on a daily basis, and their immune system quietly neutralizes the threats.  It’s only when a threat gets past the immune system that a person’s health breaks down, and then they pay attention to the system.  Software testers have the same problem: when they are doing their job really well, there is no visible impact in the software.  Key decision-makers in the company may see the software and praise the developers that created it without thinking about all the testing that helped ensure that the software was of high quality.

Measuring quality is a key way that we can demonstrate the value of our contributions.  But it’s important to measure well; a metric such as “There were 100 customer support calls this month” means nothing, because we don’t have a baseline to compare it to.  If we have monthly measurements of customer support calls, and they went from 300 calls in the first month, to 200 calls in the second month, to 100 calls in the third month, and daily usage statistics stayed the same, then it’s logical to conclude that customers are having fewer problems with the software.

With last week’s post about the various facets of quality in mind, let’s take a look at some ways we could measure quality.

Functionality:
How many bugs are found in production by customers?
A declining number could indicate that bugs are being caught by testers before going to production.
How many daily active users do we have? 
A rising number probably indicates that customers are happy with the software, and that new customers have joined the ranks of users.

Reliability:
What is our percentage of uptime?  
A rising number could show that the application has become more stable.
How many errors do we see in our logs?  
A declining number might show that the software operations are generally completing successfully.

Security:
How many issues were found by penetration tests and security scans?  
A declining number could show that the application is becoming more secure.

Performance:
What is our average response time?
A stable or declining number will show that the application is operating within accepted parameters.

Usability:
What are our customers saying about our product?
Metrics like survey responses or app store ratings can indicate how happy customers are with an application.
How many customer support calls are we getting?
Increased support calls from customers could indicate that it’s not clear how to operate the software.

Compatibility:
How many support calls are we getting related to browser, device, or operating system?
An increased number of support calls could indicate that the application is not working well in certain circumstances.
What browsers/devices/operating systems are using our software?
When looking at analytics related to app usage, a low participation rate by a certain device might indicate that users have had problems and stopped using the application.

Portability:
What percentage of customers upgraded to the new version of our software?
Comparing upgrade percentages with statistics of previous upgrades could indicate that the users found the upgrade process easy.
How many support calls did we get related to the upgrade?
An increased number of support calls compared to the last upgrade could indicate that the upgrade process was problematic.

Maintainability:
How long does it take to deploy our software to production?
If it is taking longer to deploy software than it did during the last few releases, then the process needs to be evaluated.
How frequently can we deploy?
If it is possible to deploy more frequently than was possible six months ago, then the process is becoming more streamlined.

There’s no one way to measure quality, and not every facet of quality can be measured with a metric.  But it’s important for software testers to be able to use metrics to demonstrate how their work contributes to the health of their company’s software, and the above examples are some ways to get started.  Just remember to think critically about what you are measuring, and establish good baselines before drawing any conclusions.

About a year ago, I wrote a post suggesting that we could think about automation in terms of a test wheel, where each section of the wheel represented a different type of automation.  A reader who works at Abstracta told me that my wheel reminded her of the wheel they use to think about all of the different facets of quality.  I thought their wheel was so great that I knew I would eventually want to write a post about it.

I’ve been thinking about the different types of quality mentioned in Abstracta’s Software Testing Wheel, and wondering what I would do if I was brought on to a project that had never had any testing and I needed to start from scratch.  Where would I begin my testing?  I thought about what the most important needs are for quality, and I was reminded of Maslow’s Hierarchy of Needs.

For those who are unfamiliar with this psychological concept, this is a theory that all human beings need to have certain basic needs met before they can grow as people.  The needs are as follows:

1. Physiological needs- food, water, shelter
2. Safety needs- security, property, employment
3. Love and belonging- friendship, family
4. Esteem- respect, self-esteem
5. Self-actualization- becoming the best person one can be

Looking at this list, it’s clear that physiological needs are the most important.  After all, it doesn’t matter if you have high self-esteem if you have no water to drink.  Each successive need builds on the more important one before it.

With this in mind, I realized that there is a Hierarchy of Quality- certain conditions of quality that need to be met before a team can move on to the next area of quality.  Here is my perception of where the different areas of the Abstracta test wheel fall in the hierarchy:

1.  Functionality and Reliability

These two areas share the most important spot.  Functionality means that the software does what it’s supposed to do.  This is critical, because without this, the application might as well not exist.  Imagine a clock app that didn’t tell time, or a calculator that didn’t add numbers.

Reliability means the software is available when it’s needed.  It doesn’t really matter if the app works if a user can’t get to it when they need it.

Once these quality needs have been met, we can move on to the next level:

2. Security and Performance

Security is important because users need to feel that their data is being protected.  Even applications that don’t have login information or don’t save sensitive data still need to be protected from things like cross-site scripting, which might allow a malicious user to gain control of someone else’s device.

Performance is also important, because no one wants to wait for sixty seconds for a web page to load.  If an application isn’t responsive enough, the users will go elsewhere.

Now that the application is secure and performant, we can go to the third level:

3. Usability and Compatibility

This is the level where we make sure that as many users as possible have a good experience with the application.  Usability means that the workflows of an application are intuitive so users don’t get confused.  It also means that the application is internationalized, so users all around the world can use it, and that it is accessible, so users with visual, auditory, or physical differences can use it as well.

Compatibility means that users with different operating systems, browsers, or devices can use the application.  Have you ever filled out a form in a browser and had it not save correctly when you clicked the button?  This has happened to me more than once, and I’ve needed to fill out the form again in a different browser to have it save correctly.  It’s important that our users have a positive experience no matter where they are using the software.

Now that we’ve made our application accessible to as many users as possible, it’s time to go on to the next level:

4. Portability

Portability covers how easy it is to move an application from one place to another.  One example of portability would be the way I can access my Google Drive files on my laptop, my tablet, and my phone.  Portability also refers to how easy an application can be installed or updated.  Also, we want our application to keep working when a device has an operating system upgrade.

Finally, we have thought about all of our users’ needs.  Now it’s time for one more level:

5. Maintainability

This is a level of quality that benefits the software team.  Maintainability refers to how easily an application can be updated.  Is it possible to add new APIs or update existing ones?  How easy is it to test the system?  Is it easy to deploy new code?  Is it easy for other teams to use the code?  Is the code clear and easy to understand?

When software is accessible and easy to use for all end users, AND is easy to work with and maintain for the development team, then truly high quality has been achieved.

I hope that this Hierarchy of Quality will help you make decisions about what areas of an application should be focused on first when there are a number of different quality areas competing for your team’s attention.

What do you think of this order?  Do you agree or disagree with where I placed items in the hierarchy?  Are there any missing quality areas?  Let me know in the comments below!

In last week’s post, I wrote about how it is important for software testers to code like a developer.  But there is a second half of the sentence “Code like a developer…”, and that is that software testers should be TESTING. 

I’ll be honest: I don’t love coding.  Don’t get me wrong, I love test automation!  I love the feeling of solving a technical challenge and coming up with a great way to automatically assert that software is doing what it’s supposed to be doing.  I love maintaining and updating my automated test suites.  But the actual writing of the code is not my favorite thing.  Whenever I find myself having to write another nested “for” loop, I sigh inwardly.

However, with all the coding I’ve done over the years, I’ve come to really appreciate the work that software developers do!  Software is complex stuff, and developers have come up with great ways to set standards, share repositories, and review each other’s work.

The test automation code we write is important; just as important as the code the software developers are writing.  Therefore, we should write our code with the same standards the developers use.  Here are a few suggestions for coding practices you should adopt:

• Keep it simple.  Always look over your code and ask yourself if there’s a simpler way of doing what it is that you are trying to do.  Sometimes the obvious solution to a testing problem only becomes clear after you have solved it in a complicated way; now it’s time to go back and solve it more elegantly. 
• Don’t repeat yourself.  If there’s something you’re doing in more than one test- for example, logging in to the application- write a method that you can call instead of putting those steps into every test.  Similarly, create a file where you save all of your variables and element locators, and have all of your tests refer to that file.  That way if a variable or a locator changes, you can make the change in one place rather than several. 
• Be consistent.  Consistent code is easier to read.  Be consistent with your casing: if you have a variable for the user’s first name called “firstName”, don’t make the variable for the user’s last name “LastName”.  Follow the conventions that your developers are using: if they indent with two spaces, you should too.  If they put their opening curly braces on a separate line, you should as well.
• Comment your code.  It’s not always obvious what test automation code is doing at first glance, and while you might be quite used to the syntax you are using for your tests, your developers might not be familiar with it.  Simple comments like “Polling the queue for the delete request” can be really helpful in explaining your intent.  Moreover, what might seem really obvious to you now might not be obvious in three months when you need to update the test!  Your future self will thank you for the comments you write today.  

Have you ever been querying a SQL table, and one of your queries seems to take forever?  And then the next query you run takes milliseconds?  This would frequently happen to me, and I thought it meant that the server that hosted the database was unreliable in some way.  But this week I learned about indexes, and that the way we structure our queries has a huge impact on how long they will take to execute!  In this post, I’ll describe what indexes are and talk about the ways we can use them to optimize our queries.

An index is a database structure that is designed to speed up queries in a table.  An easy way to understand this is to think about the index at the back of a book.  Let’s say you have a book on car repair, and you want to find information about your car’s brakes.  You could look up “brakes” in the index, or you could search through every single page of the book for the word “brakes”.  It’s pretty obvious which would take less time!

Unlike books, databases can have more than one index.  There are two different kinds of indexes: clustered and unclustered.  A clustered index is used to store a table in sorted order.  There can only be one clustered index, because the table is stored in only one order.  Unclustered indexes are stored in the original table order, but they save the location of certain fields in the table.

Let’s take a look at an example.  If we had a table like this, called the Users table:

Did you know that there is a wealth of testing tools right in your browser?  Web browsers like Chrome and Firefox have developer tools that are available for free, for everyone.  And these tools are not just for developers!  In this post, I’ll be sharing six ways that Chrome DevTools can help you with your testing.

To access Chrome DevTools, simply click on the three-dot menu in the upper right corner of your browser, choose “More Tools”, and then choose “Developer Tools”.  DevTools will open up alongside your browser window.  You can customize where you would like the tools to display by clicking on the three-dot menu in the DevTools nav bar and selecting an option for “Dock Side”.  You can choose to have the DevTools display on the left, on the right, on the bottom, or in a separate window.

Here are some of the things that Dev Tools can do:

1. Inspect an HTML Element
Have you ever been writing UI automation and you just can’t figure out how to access an element?  With DevTools, you can right-click on the element and choose “Inspect”, and the Elements pane of DevTools will show you the element in the HTML.  You can then use this information to figure out the best way to access the element.

2. Edit HTML Elements
Not only can you find an element in the HTML, you can also edit it!  This is great for security testing.  Imagine that there is a page with a button that is hidden for users who are not admins.  A malicious user could find that element using DevTools, remove the “hide” tag, and use the button.  So it’s helpful to try this while testing to verify that there’s an additional check for user permissions when the button is used.

To edit an element, right-click on it in the HTML displayed in the Elements pane, and choose “Edit as HTML”.  Make whatever edits to the element you want, then click out of the edit box.  You should see the element on the page change as a result of your edits.

3. View HTTP requests
If you click on the Network tab of DevTools, you can see all of the requests made to the server while using a web page.  This includes API calls, which you can then copy and use in a tool like Postman.  This feature is helpful for determining if your page is making the API calls that you are expecting, and it’s also great for security testing.  For example, just because the front-end of a web page doesn’t allow a user to submit a field with more than 50 characters doesn’t mean that it can’t be done.  If a malicious user copies the API call and submits it through Postman, through a curl command, or through some other tool, they may be able to send more than 50 characters directly to the server.  This is why it’s important to have both front-end and back-end validation on a website.

4. Simulate device frames
When you are testing a webpage, it’s important to make sure that the page appears correctly on both laptops and mobile devices.  But even the most well-equipped tester doesn’t have access to every single device in use today.  So DevTools comes with a simulator that shows roughly what your webpage will look like on various devices.  To access this feature, click on the device logo

in the toolbar.  This will open the simulator in the webpage side of the browser.  Then you can use the dropdown to select specific devices (which seem to be a bit obsolete), or you can choose the “Responsive” setting and then manually expand or contract the window to get the size you want.  The exact size is displayed in the navbar at the top.

5. Simulate performance on slower networks
Testing a webpage while in your office usually means you are using a great high-speed network.  But what about your users who have slower connections?  You can use DevTools to simulate slower connections and throttled CPU, which could help uncover race conditions in your application.  To use this feature, go to the Performance tab in the navbar.  In the Network dropdown, you can choose “Fast 3G”, “Slow 3G”, or “Offline”, and in the CPU dropdown, you can choose “No throttling”, “4x slowdown” or “6x slowdown”.  Don’t forget to reverse your changes when you are done testing!

6. Investigate page load errors
As I was creating this post, I was reminded of one more way that DevTools are helpful.  I was trying to upload the Chrome logo to my post, and the popup that I usually use to add an image was completely blank.  I went to the Console tab of DevTools and saw that there was a 404 “File not found” error when I clicked on the Add Images button in Blogger.  When you are testing your team’s application and you’ve found a bug on a page, checking for errors in the console can help you give more information to your developers so they can get to the root of the problem more quickly.

Sometimes the most useful testing tools are right there in front of you!  I hope this post has inspired you to take a look at DevTools to see how it can help you in your testing.