Logical Fallacies for Testers III: Appeal to Authority

As you can no doubt guess from the title, this is the third post on my series about logical fallacies. (You can find the first two posts here and here.) Logical fallacies are important for testers to learn about because it can help keep them from making mistakes in judgment that will impact their testing speed and accuracy. This is especially true in our third fallacy: the Appeal to Authority.

The Appeal to Authority fallacy happens when someone makes the argument that because an expert said so, something must be true. While many times experts are correct in their assessments, there are also times when they are wrong. They could be blinded by their own cognitive bias, they could have a motivation for not telling the truth, or what they say might be correct in some situations but not in others.

Furthermore, sometimes an “authority” isn’t an authority in the area where they are making a pronouncement. A perfect example of this is when a famous actor, singer, or sports figure weighs in on a political situation. Being a great actress doesn’t make someone an expert in fiscal policy, or a good judge of character in a presidential race.

How is the Appeal to Authority used in testing situations? There are two main ways. One is when a tester adopts a testing framework or tool because a testing expert recommends it. To be clear, there’s absolutely nothing wrong with this! But there are times when the framework or tool might not be right for that particular situation. It’s important for testers to take stock of their testing needs, look at the pros and cons of the tool or framework, and then make an informed decision, rather than blindly following an expert.

A second, more problematic, situation occurs when a company decides to hire a testing consultant rather than listening to their own employees. Testing consultants can provide valuable guidance, but they should not be used as a replacement for listening to the company’s own testers. Let’s use our hypothetical company, Cute Kitten Photos, to examine what can happen in this scenario.

Susie is the lead tester for Cute Kitten Photos. She wants to set up test automation for the mobile app, so she does her research and determines that Tool Y will be the best tool for creating reliable mobile test automation for the company. When she presents this data to her manager, Dharmesh, however, he is unconvinced. Dharmesh thinks that Tool Y is much too expensive, and that there must be a better, cheaper way. He hires a consultant named Bob to take a look at the mobile app and make his own recommendations. Bob takes several weeks to examine the app and then submits his report: Tool Y will be the best strategy for reliable test automation for Cute Kitten Photos! Then he submits his bill for $50,000.

Susie was correct in her original assessment, but Dharmesh didn’t listen because in his mind, she wasn’t an authority. How could Susie have made a more compelling case? In this case, Dharmesh was concerned about the expense of Tool Y, which made him willing to spend extra money for an expert. Susie could have asked for a few weeks to do a proof of concept with Tool Y. She could then have asked the sales representative at Tool Y to give her a trial account for those few weeks. Dharmesh would hopefully be convinced that spending 0 dollars on a proof of concept would be better than spending 50,000 dollars for a consultant. At the end of the trial period, Susie could demonstrate her success with the tool, showing Dharmesh that paying for Tool Y would be a good decision.

It’s important for all of us, testers and managers alike, to consider whether an authority should be followed, or even needed, when we are making decisions. Examining the evidence and using clear reasoning can keep us on the right path with our testing choices.