Recently, I was talking with some technology leaders at my company about how we needed to encourage testers to do more exploratory testing. We were getting ready to hold our second annual Hackathon, and one of our group suggested “Why not have a competition for testing as well?” And thus, Testathon was born!
To prepare for Testathon, we invited people to sign up in teams of three to five people. Testathon was open to any participants, not just software testers. We had 48 participants sign up in ten teams; most of the participants were testers, but we had a few software engineers, managers, and product owners as well.
Once I knew how many teams we had, I chose ten products for the teams to test. I made sure that no person was testing their own product. I worked with the product teams to create documentation and user logins that the test teams could use during Testathon.
Testathon lasted two days; during those days the teams tested their assigned products in as many ways as they could think of. They logged the bugs they found in a special Jira board, and marked them with their team name.
Once Testathon was over, five judges- all Principal Software Test Engineers- divided up the bugs and worked for days to attempt to reproduce them. This showed the importance of being able to write up good repro steps in a bug; if a judge wasn’t able to reproduce the bug with the instruction steps provided, they moved the bug to Closed.
Bugs that were reproduced were assigned a severity and labeled by bug type, such as API, Performance, Security, Usability, and so on.
Next, each of the judges nominated which bugs they thought should win for each category. Our categories included: Most Unique Bug, Best Bug Report, Most Critical Bug, Best Security Bug, Best API Bug, Best Usability Bug, and Best Performance Bug.
The judges convened and voted on a winner for each category, and then announced the winners to the whole Engineering group. The winners received cash prizes.
We received lots of positive feedback from the Testathon participants. They really enjoyed working with testers from other teams, testing new products, and learning test strategies from each other. Moreover, some of the bugs logged were great finds! A serious security bug was identified and fixed in two days. Another tricky bug that a product team had known about, but couldn’t consistently reproduce, was finally given good repro steps so the team could fix the bug. Many API bugs were found where the response code was incorrect. And there were several small UI bugs found that when fixed will help improve the user experience.
After Testathon, all of the bugs were presented to the product teams for analysis. They sorted through the bugs for duplicates, and moved any new bugs to their Jira board.
Testathon was a great way for our company to learn the value of exploratory testing! We are already looking forward to our next Testathon. Perhaps you might find a Testathon valuable for your company!
Great idea and congratulations for executing the testathon. I would like to know about the information you provided to the teams, did you provide the source code repo? or archetecture of the application.?
Hi Moshe! We provided logins, common user journeys, and a link to the API documentation. We also gave each team the name of a person from the product team that they could contact with questions. If the team asked for it, they could get the link to the source code, and information on the architecture of the application.
Hei,
How to participate.
Hi Anastasia- this was an event that my company did, only for the employees of my company. But you could set up your own Testathon at your company.
Hello,
Thank you very much for this article.
In my company, we also tried the Testathon experience but we had faced some problems, the most important for us being the difficulty in exploiting the bugs reported due to their lack of relevance or precision.
The analysis of all the bugs identified also required a significant effort.
Have you encountered this problem? Have you carried out preventive actions to avoid this trap?
Hi Bruno- Yes, we did encounter some difficulties managing all the bugs reported. First of all, we gave a prize for “Most Bugs Found”, which meant that some teams logged lots of tiny UI bugs that product teams weren’t interested in fixing so that they could win the prize. We won’t be using that category next time! Additionally, many bugs found were in older UIs that were slated for updates, so the bugs weren’t really necessary. Next time, we’ll test only our newer products. Also, it did take five judges nearly a week to go through all of the bugs, so it was a significant investment of time. We may want to have the product teams participate in the judging next time because they are more familiar with their own product.
Great tool to empower collaboration among the team and bring the quality narrative to the forefront. It’s inspiring to see how the Testathon encouraged diverse roles to engage in exploratory testing, leading to meaningful insights and improvements in product quality. Well done!