Does the study design support the claim?

Question 1

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. The students all liked apps.
• B. Random assignment and a comparison group make other explanations less likely.
• C. It lasted exactly one month.
• D. It tested words instead of numbers.

Question 2

Is the proposed answer correct? A company asked 15 of its own employees whether its new chair reduced back pain. Twelve said yes. The company now advertises that "80% of users report less back pain". What is the biggest problem with this evidence? → The sample is tiny and made up of people who may want the product to succeed.

• A. yes, it is correct
• B. no, it is not correct

Question 3

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. The students all liked apps.
• B. It lasted exactly one month.
• C. It tested words instead of numbers.
• D. Random assignment and a comparison group make other explanations less likely.

Question 4

Is the proposed answer correct? Researchers tested whether background music helps concentration. The same 60 students did puzzles in silence one week and with music the next. Scores were higher with music. What is a possible problem with this design? → Students may simply have improved with practice by the second week.

• A. yes, it is correct
• B. no, it is not correct

Question 5

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. The students all liked apps.
• B. Random assignment and a comparison group make other explanations less likely.
• C. It lasted exactly one month.
• D. It tested words instead of numbers.

Question 6

Is the proposed answer correct? A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate? → Even a large, careful observational study can suggest but not prove cause, so 'may help' is suitably cautious.

• A. yes, it is correct
• B. no, it is not correct

Question 7

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. It lasted exactly one month.
• B. It tested words instead of numbers.
• C. Random assignment and a comparison group make other explanations less likely.
• D. The students all liked apps.

Question 8

Is the proposed answer correct? A company asked 15 of its own employees whether its new chair reduced back pain. Twelve said yes. The company now advertises that "80% of users report less back pain". What is the biggest problem with this evidence? → Chairs can't be tested.

• A. yes, it is correct
• B. no, it is not correct

Question 9

A study tracked 50,000 adults for ten years and found that those who walked daily had fewer heart problems. The researchers adjusted for age, smoking and diet, and said walking 'may help protect' the heart. Why is the researchers' wording appropriate?

• A. Even a large, careful observational study can suggest but not prove cause, so 'may help' is suitably cautious.
• B. Ten years is too short to study health.
• C. They should have said walking definitely cures heart disease.
• D. The study was too small to say anything.

Question 10

Is the proposed answer correct? To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps? → Random assignment and a comparison group make other explanations less likely.

• A. yes, it is correct
• B. no, it is not correct

Question 11

To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps?

• A. The students all liked apps.
• B. It tested words instead of numbers.
• C. It lasted exactly one month.
• D. Random assignment and a comparison group make other explanations less likely.

Question 12

Is the proposed answer correct? A headline says: "Eating breakfast makes children smarter." The study behind it found that children who ate breakfast had higher test scores, but it did not control for family income or sleep. What is the main weakness in the headline's claim? → The sample was made up of children.

• A. yes, it is correct
• B. no, it is not correct

Question 13

Researchers tested whether background music helps concentration. The same 60 students did puzzles in silence one week and with music the next. Scores were higher with music. What is a possible problem with this design?

• A. 60 students is too many.
• B. Puzzles can't measure concentration at all.
• C. Music is always distracting.
• D. Students may simply have improved with practice by the second week.

Question 14

Is the proposed answer correct? A headline says: "Eating breakfast makes children smarter." The study behind it found that children who ate breakfast had higher test scores, but it did not control for family income or sleep. What is the main weakness in the headline's claim? → Test scores can't be measured.

• A. yes, it is correct
• B. no, it is not correct

Question 15

An article claims a herbal tea cures colds because 'most people who drank it felt better within a week'. What is the main flaw?

• A. The article doesn't name the tea's colour.
• B. Herbal tea is too hot to drink.
• C. Most colds get better within a week anyway, so there's no comparison to show the tea did anything.
• D. A week is too long to measure.

Question 16

Is the proposed answer correct? To test a new memory app, researchers randomly assigned 200 students either to use the app or to use a version with the memory games removed. After a month, the app group recalled more words. Why is this design fairly strong evidence that the app helps? → It lasted exactly one month.

• A. yes, it is correct
• B. no, it is not correct