Does the study design support the claim?

Question 1

A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate?

• A. They should have said walking definitely cures heart disease.
• B. Ten years is too short to study health.
• C. Even a large, careful observational study can suggest but not prove cause, so 'may help' is suitably cautious.
• D. The study was too small to say anything.

Question 2

Is the proposed answer correct? A headline says: "Eating breakfast makes children smarter." The study behind it found that children who ate breakfast had higher test scores, but it did not control for family income or sleep. What is the main weakness in the headline's claim? → The study shows a link, but other factors could explain it, so it doesn't prove breakfast causes higher scores.

• A. yes, it is correct
• B. no, it is not correct

Question 3

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. It tested words instead of numbers.
• B. Random assignment and a comparison group make other explanations less likely.
• C. It lasted exactly one month.
• D. The students all liked apps.

Question 4

Is the proposed answer correct? An article claims a herbal tea cures colds because 'most people who drank it felt better within a week'. What is the main flaw? → Most colds get better within a week anyway, so there's no comparison to show the tea did anything.

• A. yes, it is correct
• B. no, it is not correct

Question 5

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. It tested words instead of numbers.
• B. It lasted exactly one month.
• C. The students all liked apps.
• D. Random assignment and a comparison group make other explanations less likely.

Question 6

Is the proposed answer correct? A company asked 15 of its own employees whether its new chair reduced back pain. Twelve said yes. The company now advertises that "80% of users report less back pain". What is the biggest problem with this evidence? → The sample is tiny and made up of people who may want the product to succeed.

• A. yes, it is correct
• B. no, it is not correct

Question 7

A headline says: "Eating breakfast makes children smarter." The study behind it found that children who ate breakfast had higher test scores, but it did not control for family income or sleep. What is the main weakness in the headline's claim?

• A. The study shows a link, but other factors could explain it, so it doesn't prove breakfast causes higher scores.
• B. Breakfast is unhealthy.
• C. Test scores can't be measured.
• D. The sample was made up of children.

Question 8

Is the proposed answer correct? A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate? → They should have said walking definitely cures heart disease.

• A. yes, it is correct
• B. no, it is not correct

Question 9

A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate?

• A. Ten years is too short to study health.
• B. They should have said walking definitely cures heart disease.
• C. Even a large, careful observational study can suggest but not prove cause, so 'may help' is suitably cautious.
• D. The study was too small to say anything.

Question 10

Is the proposed answer correct? A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate? → Ten years is too short to study health.

• A. yes, it is correct
• B. no, it is not correct

Question 11

A company asked 15 of its own employees whether its new chair reduced back pain. Twelve said yes. The company now advertises that "80% of users report less back pain". What is the biggest problem with this evidence?

• A. 80% is not a real percentage.
• B. The sample is tiny and made up of people who may want the product to succeed.
• C. Chairs can't be tested.
• D. Back pain is not a real problem.

Question 12

Is the proposed answer correct? A company asked 15 of its own employees whether its new chair reduced back pain. Twelve said yes. The company now advertises that "80% of users report less back pain". What is the biggest problem with this evidence? → Back pain is not a real problem.

• A. yes, it is correct
• B. no, it is not correct

Question 13

An article claims a herbal tea cures colds because 'most people who drank it felt better within a week'. What is the main flaw?

• A. Most colds get better within a week anyway, so there's no comparison to show the tea did anything.
• B. A week is too long to measure.
• C. Herbal tea is too hot to drink.
• D. The article doesn't name the tea's colour.

Question 14

Is the proposed answer correct? A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate? → Even a large, careful observational study can suggest but not prove cause, so 'may help' is suitably cautious.

• A. yes, it is correct
• B. no, it is not correct

Question 15

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. It lasted exactly one month.
• B. The students all liked apps.
• C. It tested words instead of numbers.
• D. Random assignment and a comparison group make other explanations less likely.

Question 16

Is the proposed answer correct? A headline says: "Eating breakfast makes children smarter." The study behind it found that children who ate breakfast had higher test scores, but it did not control for family income or sleep. What is the main weakness in the headline's claim? → The study shows a link, but other factors could explain it, so it doesn't prove breakfast causes higher scores.

• A. yes, it is correct
• B. no, it is not correct