Bringing the psych lab online
Ken Nakayama (left) and Laura Germine are co-authors of a study that found that the quality of data obtained by doing psychological studies online can be just as good as that from in-person experiments. "The thing I like to say about using the Web is that it’s fast, it’s cheap, but it’s not dirty," said Germine. "In experiments like ours, what you’re getting is good, reliable data.” Photo: Stephanie Mitchell/Harvard Staff Photographer
The Internet has already fundamentally changed the way that people communicate, shop, and even date, but now it is poised to revolutionize psychological studies by enabling researchers to quickly and easily recruit thousands of study volunteers from around the world, and by changing the way the public interacts with researchers.
By conducting experiments online, researchers have been able to enlist as many as 65,000 volunteers to take part in studies of cognition, a number far larger than they could bring into the lab. Such studies, however, have been dogged by questions about whether anonymous, unpaid volunteers tested online can produce data that is as high quality as that gathered through in-person lab testing.
New research conducted by Harvard scientists may put those questions to rest.
A team led by Laura Germine, a postdoctoral research associate in Harvard's Psychology Department, and made up of Ken Nakayama, Edgar Pierce Professor of Psychology and chair of the Department of Psychology, and Jeremy Wilmer of Wellesley College, has shown that data gathered exclusively through online volunteers can be just as good as that from in-person experiments. Their research is described in a forthcoming issue of Psychonomic Bulletin & Review.
"What this says is that people shouldn't be afraid of the Web as a way of conducting their research," said Germine, the paper's first author. "Despite the cost advantages, despite the time advantages, researchers continue to worry that data from Web volunteers will not be as good as data from paid lab participants. We've shown that data from self-selected Web volunteers can be very good. The thing I like to say about using the Web is that it's fast, it's cheap, but it's not dirty. In experiments like ours, what you're getting is good, reliable data."
To test whether Web studies are as valid as those done in the lab, researchers recruited thousands of volunteers through Germine's site, TestMyBrain.org. The volunteers, who navigated from search engines and social networking sites, took part in a handful of tests to learn more about themselves and contribute to scientific research. These tests were designed to assess everything from facial recognition ability to a person's capacity for remembering a long string of numbers. Researchers then compared the results of the online tests with those done in the lab.
"We looked at three basic metrics," Germine explained. "We looked at mean performance, we looked at variation in performance, and we looked at internal consistency. In some ways, I think, the third measure—the consistency, or internal reliability—is the most important, because it allows you to look at how someone's performance on one part of the test predicts how they will perform on another part of the test. If they stopped paying attention or started watching YouTube videos halfway through a test, that would be reflected in lower reliability."
On each measure, Germine said, the results from studies conducted with Web volunteers were the same as those done in the lab. The only variation researchers found came when they compared the average, or mean, performance of the general public with that of elite college students from Harvard and Wellesley on IQ-based tests.
"For tests based on visual recognition and perception that are less related to IQ, average scores were the same for Web and lab," Germine said.
While it has simplified the process of gathering data, the Web has also changed the relationship between researchers and those they study. By forming communities online, Germine said, people who suffer from a particular disorder have been able to raise awareness—and drive research in areas that might otherwise not be studied.
"The other benefit that comes from using the Web is that people can offer insights about themselves we might not think to ask," Germine said. "The best example of this is with the selective developmental disorders."
Arguably the most prominent example of such a disorder is prosopagnosia, also known as "face blindness," in which individuals have difficulty recognizing people, sometimes even in their own families. Although the condition was occasionally reported following a stroke, when people claimed to have had the disorder from birth, they were often dismissed by doctors and researchers. As tests for facial recognition were made available online, researchers began to realize that the disorder was far more common than previously thought.
"As a result, we learned a great deal," Germine said. "Because we had these tests available online, and because people were able to participate in this exploration online, the knowledge they gained validated their own experience, which encouraged them to get in contact with a researcher. So there was this circle of knowledge that became very beneficial to everyone involved.
"I think psychology, as a field, is unique in that the research questions can be made fairly accessible to the public," she added. "The Web offers us another avenue to take advantage of that, and to engage people in our science in a new way. I think there's a great opportunity for collaboration between scientists and 'citizen scientists' to advance scientific knowledge."
More information: www.psychonomic.or… thcoming.htm
Journal reference: Psychonomic Bulletin & Review
Provided by Harvard University
- Ability to recognize faces peaks in the 30s Dec 15, 2010 | not rated yet | 0
- Ability to estimate quantity increases in first 30 years of life Jun 27, 2012 | not rated yet | 0
- 'Super-recognizers,' with extraordinary face recognition ability, never forget a face May 19, 2009 | not rated yet | 0
- Should medical researchers share their results with the volunteers in their studies? Jun 06, 2008 | not rated yet | 0
- Lab in the Wild asks: What's your Internet like? Aug 06, 2012 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Why is zone 1 in liver more prone to ischemic injury?
May 23, 2013 Hi, Is it because around central vein, there is only deoxygenated blood from the vein where as in the periphery there is hepatic artery. Also why...
How can there be villous adenoma in colon, if there are no villi there
May 22, 2013 As title suggest. Thanks :smile:
How can there be a term called "intestinal metaplasia" of stomach
May 21, 2013 Hello everyone, Ok Stomach's normal epithelium is simple columnar, now in intestinal type of adenocarcinoma of stomach it undergoes "intestinal...
Pressure-volume curve: Elastic Recoil Pressure don't make sense
May 18, 2013 From pressure-volume curve of the lung and chest wall (attached photo), I don't understand why would the elastic recoil pressure of the lung is...
If you became brain-dead, would you want them to pull the plug?
May 17, 2013 I'd want the rest of me to stay alive. Sure it's a lousy way to live but it beats being all-the-way dead. Maybe if I make it 20 years they'll...
MRI bill question
May 15, 2013 Dear PFers, The hospital gave us a $12k bill for one MRI (head with contrast). The people I talked to at the hospital tell me that they do not...
- More from Physics Forums - Medical Sciences
More news stories
(HealthDay)—We've all seen them: the surfers who race to the beach when a hurricane hits, the guy who decides to ride out the storm in his overmatched boat, the tornado chasers who fearlessly steer their ...
Psychology & Psychiatry May 24, 2013 | not rated yet | 0
Women at a particular stage in their monthly menstrual cycle may be more vulnerable to some of the psychological side-effects associated with stressful experiences, according to a study from UCL.
Psychology & Psychiatry May 24, 2013 | 4 / 5 (4) | 4 |
Ernie Pyle – an iconic war correspondent in World War II – reportedly said "There are no atheists in foxholes." A new joint study between two brothers at Cornell and Virginia Wesleyan found that only ...
Psychology & Psychiatry May 24, 2013 | 2.5 / 5 (4) | 2
(Medical Xpress)—Research by Stanford scholar Emma Seppala at the Center for Compassion and Altruism Research and Education found that post-traumatic stress disorder decreased in veterans who participated ...
Psychology & Psychiatry May 24, 2013 | 5 / 5 (1) | 1
(Medical Xpress)—Patients with diabetes who are depressed are much more likely to develop episodes of dangerously low blood sugars, or hypoglycemia, than are those who are not depressed, a new study has ...
Psychology & Psychiatry May 24, 2013 | not rated yet | 0 |
(HealthDay)—Animals make great companions for senior citizens, but elderly people who always drive with a pet in the car are far more likely to crash than those who never drive with a pet, researchers have ...
1 hour ago | not rated yet | 0
Coenzyme Q10 decreases all cause mortality by half, according to the results of a multicentre randomised double blind trial presented today at Heart Failure 2013 congress. It is the first drug to improve heart failure mortality ...
9 hours ago | 5 / 5 (2) | 5
Heart failure accelerates the aging process and brings on early andropausal syndrome (AS), according to research presented today at the Heart Failure Congress 2013. AS, also referred to as male 'menopause', was four times ...
9 hours ago | not rated yet | 1
Mortality and length of stay are highest in heart failure patients admitted in January, on Friday, and overnight, according to research presented today at the Heart Failure Congress 2013. The analysis of nearly 1 million ...
9 hours ago | not rated yet | 0
(AP)—Department of Justice lawyers have again asked a federal appeals court in New York to delay lifting age restrictions and prescription requirements on an emergency contraceptive popularly known as the morning-after ...
9 hours ago | not rated yet | 0