PSA: Not All Research Needs to Be Statistically Significant
The world can be divided into three groups:
People who have never heard of statistical significance
People who know A LOT about statistical significance
People who have heard of statistical significance and, while they don’t exactly remember what it means, know that smart people think it’s very important
If you are in that last group, this is for you: Not all research needs to be statistically significant. In fact, there are many times when commissioning research to collect statistically significant data is exactly the wrong thing to do.
[BTW, “statistical significance” is a method of quantifying whether a result is likely due to chance or to some factor of interest. When a finding is statistically significant, it simply means you can feel confident that’s it real, not that you just got lucky (or unlucky) in choosing the sample.]
Different research goals require different research approaches. If you get your mind set on collecting statistically significant data, you have unintentionally made the decision that you are pursuing quantitative research. In doing so, you’ve eliminated the entire category of qualitative research, which is more useful when you’re trying to generate ideas, looking for themes, or seeking to understand why something is or isn’t happening.
So when is statistically significant quantitative data useful? When you have a specific hypothesis to confirm, you need to show a measurable outcome, and have access to a large sample. For example, if a small number of interviews led you to discover that most of your customers have 4 or more children living at home, you might want to validate this through a statistically significant study.
