What is the study about: The paper examines fake news and whether repeated exposure to such content creates the illusion that news is actually real. In Study 2, participants go through a set of real and fake news headlines presented in Facebook-post format. They are then presented with some new and some already presented headlines and have to assess the accuracy of each headline. A follow-up session a week later tests whether the effect persists. Additionally, the study tests whether warnings for suspicious content eliminate the effect and whether it depends on political concordance.
Suggested number of participants: 100
Can it be conducted online: YES
Mandatory data analyses: Repeated measures ANOVA, t-tests
Note: The “news” are based on the two party political system in the US (Democrats and Republicans), more specifically the 2016-era US election. Make sure the participants are old enough to remember this time period and are familiar with the American party system in general.
The CREP team advice:
Full paper link: https://psycnet.apa.org/manuscript/2018-46919-001.pdf
(The mentors are obliged to read the full paper before committing to the replication)
Structure of the survey:
- Familiarization stage: Participants see 12 headlines in Facebook format – 6 real and 6 fake. A subset of the fake headlines is shown with a “disputed by 3rd party fact-checkers” warning.
- Distractor tasks / filler
- Accuracy judgment phase: Participants rate headlines they saw earlier mixed with new ones, in random order and are asked to assess how accurate are the clams made in those headlines (1 = not at all accurate, 2 = not very accurate, 3 = somewhat accurate, 4 = very accurate). Additionally, it is asses whether they are familiar with the stories before the study.
- Follow-up session about one week later: Participants rate accuracy of the same headlines again.
Results obtained in the paper:
A single prior exposure increased perceived accuracy for both fake and real news. Roughly twice as many participants judged a given fake headline as accurate after exposure. When it comes to the fake headlines with a fact-checker warning, familiarized warned headlines were rated as more accurate than novel unwarned ones, even though the warnings lowered perceived accuracy overall. The effect also held for headlines inconsistent with the participant’s political ideology, and it persisted after a week.