Testathon

Recently, I was talking with some technology leaders at my company about how we needed to encourage testers to do more exploratory testing. We were getting ready to hold our second annual Hackathon, and one of our group suggested “Why not have a competition for testing as well?” And thus, Testathon was born!

To prepare for Testathon, we invited people to sign up in teams of three to five people. Testathon was open to any participants, not just software testers. We had 48 participants sign up in ten teams; most of the participants were testers, but we had a few software engineers, managers, and product owners as well.

Once I knew how many teams we had, I chose ten products for the teams to test. I made sure that no person was testing their own product. I worked with the product teams to create documentation and user logins that the test teams could use during Testathon.

Testathon lasted two days; during those days the teams tested their assigned products in as many ways as they could think of. They logged the bugs they found in a special Jira board, and marked them with their team name.

Once Testathon was over, five judges- all Principal Software Test Engineers- divided up the bugs and worked for days to attempt to reproduce them. This showed the importance of being able to write up good repro steps in a bug; if a judge wasn’t able to reproduce the bug with the instruction steps provided, they moved the bug to Closed.

Bugs that were reproduced were assigned a severity and labeled by bug type, such as API, Performance, Security, Usability, and so on.

Next, each of the judges nominated which bugs they thought should win for each category. Our categories included: Most Unique Bug, Best Bug Report, Most Critical Bug, Best Security Bug, Best API Bug, Best Usability Bug, and Best Performance Bug.

The judges convened and voted on a winner for each category, and then announced the winners to the whole Engineering group. The winners received cash prizes.

We received lots of positive feedback from the Testathon participants. They really enjoyed working with testers from other teams, testing new products, and learning test strategies from each other. Moreover, some of the bugs logged were great finds! A serious security bug was identified and fixed in two days. Another tricky bug that a product team had known about, but couldn’t consistently reproduce, was finally given good repro steps so the team could fix the bug. Many API bugs were found where the response code was incorrect. And there were several small UI bugs found that when fixed will help improve the user experience.

After Testathon, all of the bugs were presented to the product teams for analysis. They sorted through the bugs for duplicates, and moved any new bugs to their Jira board.

Testathon was a great way for our company to learn the value of exploratory testing! We are already looking forward to our next Testathon. Perhaps you might find a Testathon valuable for your company!