Using P-values to make conclusions

This article was designed to provide a bit of teaching and a whole lot of practice. The questions are ordered to build your understanding as you go, so it's probably best to do them in order. Onward!

We use $p$‍-values to make conclusions in significance testing. More specifically, we compare the $p$‍-value to a significance level $\alpha$‍ to make conclusions about our hypotheses.

If the $p$‍-value is lower than the significance level we chose, then we reject the null hypothesis $H_0$‍ in favor of the alternative hypothesis $H_{\text{a}}$‍. If the $p$‍-value is greater than or equal to the significance level, then we fail to reject the null hypothesis $H_0$‍, but this doesn't mean we accept $H_0$‍. To summarize:

Let's try a few examples where we use $p$‍-values to make conclusions.

Alessandra designed an experiment where subjects tasted water from four different cups and attempted to identify which cup contained bottled water. Each subject was given three cups that contained regular tap water and one cup that contained bottled water (the order was randomized). She wanted to test if the subjects could do better than simply guessing when identifying the bottled water.

Her hypotheses were $H_0:p=0.25$‍ vs. $H_{\text{a}}:p>0.25$‍ (where $p$‍ is the true likelihood of these subjects identifying the bottled water).

The experiment showed that $20$‍ of the $60$‍ subjects correctly identified the bottle water. Alessandra calculated that the statistic $\hat{p}={\displaystyle \frac{20}{60}}=0.\overline{3}$‍ had an associated P-value of approximately $0.068$‍.

A certain bag of fertilizer advertises that it contains $7.25\text{ kg}$‍, but the amounts these bags actually contain is normally distributed with a mean of $7.4\text{ kg}$‍ and a standard deviation of $0.15\text{ kg}$‍.

The company installed new filling machines, and they wanted to perform a test to see if the mean amount in these bags had changed. Their hypotheses were $H_0:\mu =7.4\text{ kg}$‍ vs. $H_{\text{a}}:\mu \ne 7.4\text{ kg}$‍ (where $\mu$‍ is the true mean weight of these bags filled by the new machines).

They took a random sample of $50$‍ bags and observed a sample mean and standard deviation of $\overline{x}=7.36\text{ kg}$‍ and $s_x=0.12\text{ kg}$‍. They calculated that these results had a P-value of approximately $0.02$‍.

These examples demonstrate how we may arrive at different conclusions from the same data depending on what we choose as our significance level $\alpha$‍. In practice, we should make our hypotheses and set our significance level before we collect or see any data. Which specific significance level we choose depends on the consequences of various errors, and we'll cover that in videos and exercises that follow.