The Hasty Generalization Fallacy is a common one in software testing. But before we look at its impact on testing, let’s learn what it is. This fallacy occurs when someone draws a conclusion based on just one example, or a few examples.
You may have fallen for the Hasty Generalization Fallacy as a child when you met someone from another country for the first time. If they were very nice, you may have concluded that everyone from that country is nice. If they were cold and unfeeling to you, you may have concluded that everyone from that country is cold and unfeeling. This is silly, because countries have millions of people and it’s unreasonable to assume that everyone in an entire country will have the same personality!
This fallacy is very dangerous in software testing, because it results in us not testing enough. If you run one or two tests on a feature, and from those tests determine that the feature is working just fine, and you stop testing, you may miss important bugs. Here are some examples where this might happen:
• Running a passing test in the QA environment, and assuming that it must work in the Production environment without actually checking
• Running a passing test with one Admin user, and assuming that it must work with other types of users, even those that have different permission levels
• Running a passing test on an Android device, and concluding that it must work on an iOS device as well
Recently, I got this question from a team manager: “Why are we expected to test in the Stage environment when we’ve already tested in the QA environment?” There are so many reasons!
• Data could be different in the Stage environment, which might expose a missed bug
• APIs that the application is dependent upon could be at a different state of release, exposing a bug that might then go on to Production
• Connections, such as those that go to a database or call an email service, could have the wrong connection strings. If the connections are wrong in Stage, they could also be wrong in Production.
Before you stop testing a feature, ask yourself whether you might be committing the Hasty Generalization Fallacy. Think about the other tests you could run to make sure that the feature is really working as expected. Always ask yourself, “What else could I test?”
I like your series of Fallacies — just thinking loud – what if you drop the qualifier ‘Logical” from “Logical fallacies” ?
What are other types of fallacies so prominent so that you had to use “logical” qualifier to distinguish from other types?
Like “Physical fallacies” or “Emotional fallacies” (like “everybody hates me” or “I am not good”) or social fallacies…
Keep writing good stuff
BTW – my blog is called “Thinking Tester” 🙂
Shrini
Hi Shrini- What a coincidence that your blog is named so similarly to mine! (And what an awesome endorsement you have from James Bach!) I think the term is “Logical Fallacies” because of the long tradition of logical thought. They can sometimes be called Syllogistic Fallacies as well. Certainly the fallacies can be used in all types of thinking, including physical and emotional thought. The most important thing is that we learn to challenge our thinking, to ensure that our conclusions are arrived at logically.
Good and simple read.
Yes! Such a good point you made.
thanks!
I like this post. I’m an old school test manager and have always called this logical fallacy as assumptions. Assumptions are unmitigated risks and need to be challenged on what happens if the assumption proves to be true and conversely what if the assumption proves to be false. They’re potential showstopper issues hidden in these innocuous statements.
“Data could be different in the Stage environment” this itself will blast over the QA processes. Data preparation is as important as test designing using test conditions. If the data on which we test does not replicate or be like the production data, it is definitely not an effective QA practice. Only smoke tests need to be a part of the Staging and Production environments tests and if time permits, then some important risk based testing.
The question “Why are we expected to test in the Stage environment when we’ve already tested in the QA environment?” asked is valid in my opinion. Just to be more specific, to ask like “Why are we expected to perform routine regression tests on the Staging environment when we’ve already tested on our QA environment?”
[[..Pingback..]]
This article was curated as a part of #93rd Issue of Software Testing Notes Newsletter.
https://softwaretestingnotes.substack.com/p/issue-93-software-testing-notes
Web: https://softwaretestingnotes.com
This is a difficult one to overcome unless you’re concentrating because it isn’t a fallacy in other areas of your life – it’s a hold-over from your survival instincts as a caveman. Your mind takes each experience and adds it to your experience library to create generalizations first and then each additional experience narrows down your brain’s categorization.
E.g.
Furry thing with big, sharp teeth = probably dangerous >> avoid!
Bigger furry thing with large teeth-like things and a big nose = possibly dangerous >> easy to avoid but tastes good with ketchup >> be careful when approaching!