Bringing the psych lab online

September 3, 2012 by Peter Reuell
Ken Nakayama (left) and Laura Germine are co-authors of a study that found that the quality of data obtained by doing psychological studies online can be just as good as that from in-person experiments. "The thing I like to say about using the Web is that it’s fast, it’s cheap, but it’s not dirty," said Germine. "In experiments like ours, what you’re getting is good, reliable data.” Photo: Stephanie Mitchell/Harvard Staff Photographer

The Internet has already fundamentally changed the way that people communicate, shop, and even date, but now it is poised to revolutionize psychological studies by enabling researchers to quickly and easily recruit thousands of study volunteers from around the world, and by changing the way the public interacts with researchers.

By conducting experiments online, researchers have been able to enlist as many as 65,000 volunteers to take part in studies of cognition, a number far larger than they could bring into the lab. Such studies, however, have been dogged by questions about whether anonymous, unpaid volunteers tested online can produce data that is as high quality as that gathered through in-person lab testing.

New research conducted by Harvard scientists may put those questions to rest.

A team led by Laura Germine, a postdoctoral research associate in Harvard's Psychology Department, and made up of Ken Nakayama, Edgar Pierce Professor of Psychology and chair of the Department of Psychology, and Jeremy Wilmer of Wellesley College, has shown that data gathered exclusively through online volunteers can be just as good as that from in-person experiments. Their research is described in a forthcoming issue of Psychonomic Bulletin & Review.

"What this says is that people shouldn't be afraid of the Web as a way of conducting their research," said Germine, the paper's first author. "Despite the cost advantages, despite the time advantages, researchers continue to worry that data from Web volunteers will not be as good as data from paid lab participants. We've shown that data from self-selected Web volunteers can be very good. The thing I like to say about using the Web is that it's fast, it's cheap, but it's not dirty. In experiments like ours, what you're getting is good, reliable data."

To test whether Web studies are as valid as those done in the lab, researchers recruited thousands of volunteers through Germine's site, TestMyBrain.org. The volunteers, who navigated from search engines and social networking sites, took part in a handful of tests to learn more about themselves and contribute to scientific research. These tests were designed to assess everything from facial recognition ability to a person's capacity for remembering a long string of numbers. Researchers then compared the results of the online tests with those done in the lab.

"We looked at three basic metrics," Germine explained. "We looked at mean performance, we looked at variation in performance, and we looked at internal consistency. In some ways, I think, the third measure—the consistency, or internal reliability—is the most important, because it allows you to look at how someone's performance on one part of the test predicts how they will perform on another part of the test. If they stopped paying attention or started watching YouTube videos halfway through a test, that would be reflected in lower reliability."

On each measure, Germine said, the results from studies conducted with Web volunteers were the same as those done in the lab. The only variation researchers found came when they compared the average, or mean, performance of the general public with that of elite college students from Harvard and Wellesley on IQ-based tests.

"For tests based on visual recognition and perception that are less related to IQ, average scores were the same for Web and lab," Germine said.

While it has simplified the process of gathering data, the Web has also changed the relationship between researchers and those they study. By forming communities online, Germine said, people who suffer from a particular disorder have been able to raise awareness—and drive research in areas that might otherwise not be studied.

"The other benefit that comes from using the Web is that people can offer insights about themselves we might not think to ask," Germine said. "The best example of this is with the selective developmental disorders."

Arguably the most prominent example of such a disorder is prosopagnosia, also known as "face blindness," in which individuals have difficulty recognizing people, sometimes even in their own families. Although the condition was occasionally reported following a stroke, when people claimed to have had the disorder from birth, they were often dismissed by doctors and researchers. As tests for facial recognition were made available online, researchers began to realize that the disorder was far more common than previously thought.

"As a result, we learned a great deal," Germine said. "Because we had these tests available online, and because people were able to participate in this exploration online, the knowledge they gained validated their own experience, which encouraged them to get in contact with a researcher. So there was this circle of knowledge that became very beneficial to everyone involved.

"I think psychology, as a field, is unique in that the research questions can be made fairly accessible to the public," she added. "The Web offers us another avenue to take advantage of that, and to engage people in our science in a new way. I think there's a great opportunity for collaboration between scientists and 'citizen scientists' to advance scientific knowledge."

Explore further: Ability to estimate quantity increases in first 30 years of life

More information: www.psychonomic.org/PBR/forthcoming.htm

Related Stories

Ability to estimate quantity increases in first 30 years of life

June 27, 2012
One of the basic elements of cognition―the ability to estimate quantities―grows more precise across the first 30 years or more of a person's life, according to researchers supported by the National Institutes ...

Recommended for you

Cultural values can be a strong predictor of alcohol consumption

November 20, 2017
Countries with populations that value autonomy and harmony tend to have higher average levels of alcohol consumption than countries with more traditional values, such as hierarchy and being part of a collective. This new ...

A walk at the mall or the park? New study shows, for moms and daughters, a walk in the park is best

November 17, 2017
Spending time together with family may help strengthen the family bond, but new research from the University of Illinois shows that specifically spending time outside in nature—even just a 20-minute walk—together can ...

Risk of distracted driving predicted by age, gender, personality and driving frequency

November 17, 2017
New research identifies age, gender, personality and how often people drive as potential risk factors for becoming distracted while driving. Young men, extroverted or neurotic people, and people who drive more often were ...

When male voles drink alcohol, but their partner doesn't, their relationship suffers

November 17, 2017
A study of the effect of alcohol on long-term relationships finds that when a male prairie vole has access to alcohol, but his female partner doesn't, the relationship suffers - similar to what has been observed in human ...

Spanking linked to increase in children's behavior problems

November 16, 2017
Children who have been spanked by their parents by age 5 show an increase in behavior problems at age 6 and age 8 relative to children who have never been spanked, according to new findings in Psychological Science, a journal ...

Multiplayer video games: Researchers discover link between skill and intelligence

November 15, 2017
Researchers at the University of York have discovered a link between young people's ability to perform well at two popular video games and high levels of intelligence.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.