Random sampling vs. random assignment (scope of inference)

Hilary wants to determine if any relationship exists between Vitamin D and blood pressure.

She is considering using one of a few different designs for her study.

Hilary obtains a random sample of residents from her town. She surveys those residents on whether or not they consume Vitamin D and how much Vitamin D they get. She also measures their blood pressures.

Suppose Hilary finds that among the people sampled, those who consume higher amounts of Vitamin D had significantly lower blood pressure than those who did not.

Hilary recruits residents from her town who have physical exams scheduled in the next month with the local doctor's office. She randomly assigns the volunteers to either a Vitamin D supplement pill or a placebo pill. Participants do not know which pill they are taking. They have their blood pressures measured before the study begins and at the end of the study.

Suppose Hilary finds that the group who took the Vitamin D supplements had a significant decrease in blood pressure, while the placebo group showed no significant change in blood pressure.

Note: In the real world, we can't ethically take a random sample of people and make them participate in a study involving drugs. However, there are more advanced methods for controlling for this type of selection bias. When we rely on volunteers for testing new drugs and we see significant results, we need to be willing to assume that the volunteers are representative of the larger population. We can also repeat the study on a different group of volunteers to see if we get the same results.

Key idea: If a sample isn't randomly selected, it may not be representative of the larger population. On the AP test, be ready to apply this concept and some nuance when it comes to discussing if a sample is representative of the larger population.

The table below summarizes what type of conclusions we can make based on the study design.