Lecture 12: Statistical Hypotheses Testing
October 10th, 2024 (7th week of classes)Read and watch everything
Statistical Hypothesis Testing: Generating evidence-based conclusion without complete biological knowledge
One of the most crucial functions of statistics is to generate evidence that supports research hypotheses. However, the framework of statistical hypothesis testing extends beyond research-driven fields and can be applied to various areas such as election polls, marketing preferences, societal trends, and virtually any question based on sample data.
In general, ‘evidence’ refers to information, facts, or data that support (or refute) a claim, prediction, assumption, or hypothesis.
When we refer to ‘evidence from the scientific literature,’ we mean the empirical studies that have been published in peer-reviewed scholarly journals.
What is a research hypothesis:
A research hypothesis is a proposed explanation or supposition made based on limited evidence, intended to serve as a starting point for further investigation (Oxford Dictionary). It is a proposition formulated for reasoning, without assuming it to be true.
Hypotheses, often viewed as educated guesses, have not yet been confirmed or supported by data. They cannot be definitively proven right or wrong by the data. Instead, they can be said to be either refuted or supported by the evidence gathered from research.
The most used framework for statistical hypothesis testing is based on P-values, which follows a frequentist approach.
Statistical hypothesis testing, often involving statistical tests, is a quantitative method of statistical inference that generates evidence either supporting or contradicting a research hypothesis.
In this framework, the research hypothesis is translated into a statistical question. This question is then framed as two mutually exclusive hypotheses: the null hypothesis (H0) and the alternative hypothesis (HA).
The process typically involves estimating a probability value (P-value), which serves as a quantitative measure of the strength of evidence for or against the research hypothesis.
Some interesting videos that complement this lecture:
What is a P-Value and Why Does it Matter? By NOVA PBD Official. The famous Lady tasting tea problem: https://en.wikipedia.org/wiki/Lady_tasting_tea
Even scientists struggle to explain p-values—and they use them every day! Not exactly ideal, right? By the end of this lecture, let’s make sure you’re not in that club. You’ll know exactly what a p-value means (and maybe even be able to explain it to a scientist!)
Not Even Scientists Can Easily Explain P-values: https://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/
After watching the video, read this small article, P-value: What is and what is not: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5804470/
Mark Chang (2017) said it best: “A smaller p-value indicates a discrepancy between the hypothesis and the observed data. In this sense, p-value measures the strength of evidence against the null hypothesis. However, p-value is not a probability of a null hypothesis being true. Rejecting H0 at α = .05 level with p = .02 does not mean we have a 5% or 2% probability of making a mistake.”; Educational and Psychological Measurement, 77, 475-488.
Lecture
Slides: