Bringing the psych lab online

September 3, 2012 by Peter Reuell
Ken Nakayama (left) and Laura Germine are co-authors of a study that found that the quality of data obtained by doing psychological studies online can be just as good as that from in-person experiments. "The thing I like to say about using the Web is that it’s fast, it’s cheap, but it’s not dirty," said Germine. "In experiments like ours, what you’re getting is good, reliable data.” Photo: Stephanie Mitchell/Harvard Staff Photographer

The Internet has already fundamentally changed the way that people communicate, shop, and even date, but now it is poised to revolutionize psychological studies by enabling researchers to quickly and easily recruit thousands of study volunteers from around the world, and by changing the way the public interacts with researchers.

By conducting experiments online, researchers have been able to enlist as many as 65,000 volunteers to take part in studies of cognition, a number far larger than they could bring into the lab. Such studies, however, have been dogged by questions about whether anonymous, unpaid volunteers tested online can produce data that is as high quality as that gathered through in-person lab testing.

New research conducted by Harvard scientists may put those questions to rest.

A team led by Laura Germine, a postdoctoral research associate in Harvard's Psychology Department, and made up of Ken Nakayama, Edgar Pierce Professor of Psychology and chair of the Department of Psychology, and Jeremy Wilmer of Wellesley College, has shown that data gathered exclusively through online volunteers can be just as good as that from in-person experiments. Their research is described in a forthcoming issue of Psychonomic Bulletin & Review.

"What this says is that people shouldn't be afraid of the Web as a way of conducting their research," said Germine, the paper's first author. "Despite the cost advantages, despite the time advantages, researchers continue to worry that data from Web volunteers will not be as good as data from paid lab participants. We've shown that data from self-selected Web volunteers can be very good. The thing I like to say about using the Web is that it's fast, it's cheap, but it's not dirty. In experiments like ours, what you're getting is good, reliable data."

To test whether Web studies are as valid as those done in the lab, researchers recruited thousands of volunteers through Germine's site, TestMyBrain.org. The volunteers, who navigated from search engines and social networking sites, took part in a handful of tests to learn more about themselves and contribute to scientific research. These tests were designed to assess everything from facial recognition ability to a person's capacity for remembering a long string of numbers. Researchers then compared the results of the online tests with those done in the lab.

"We looked at three basic metrics," Germine explained. "We looked at mean performance, we looked at variation in performance, and we looked at internal consistency. In some ways, I think, the third measure—the consistency, or internal reliability—is the most important, because it allows you to look at how someone's performance on one part of the test predicts how they will perform on another part of the test. If they stopped paying attention or started watching YouTube videos halfway through a test, that would be reflected in lower reliability."

On each measure, Germine said, the results from studies conducted with Web volunteers were the same as those done in the lab. The only variation researchers found came when they compared the average, or mean, performance of the general public with that of elite college students from Harvard and Wellesley on IQ-based tests.

"For tests based on visual recognition and perception that are less related to IQ, average scores were the same for Web and lab," Germine said.

While it has simplified the process of gathering data, the Web has also changed the relationship between researchers and those they study. By forming communities online, Germine said, people who suffer from a particular disorder have been able to raise awareness—and drive research in areas that might otherwise not be studied.

"The other benefit that comes from using the Web is that people can offer insights about themselves we might not think to ask," Germine said. "The best example of this is with the selective developmental disorders."

Arguably the most prominent example of such a disorder is prosopagnosia, also known as "face blindness," in which individuals have difficulty recognizing people, sometimes even in their own families. Although the condition was occasionally reported following a stroke, when people claimed to have had the disorder from birth, they were often dismissed by doctors and researchers. As tests for facial recognition were made available online, researchers began to realize that the disorder was far more common than previously thought.

"As a result, we learned a great deal," Germine said. "Because we had these tests available online, and because people were able to participate in this exploration online, the knowledge they gained validated their own experience, which encouraged them to get in contact with a researcher. So there was this circle of knowledge that became very beneficial to everyone involved.

"I think psychology, as a field, is unique in that the research questions can be made fairly accessible to the public," she added. "The Web offers us another avenue to take advantage of that, and to engage people in our science in a new way. I think there's a great opportunity for collaboration between scientists and 'citizen scientists' to advance scientific knowledge."

Explore further: Ability to estimate quantity increases in first 30 years of life

More information: www.psychonomic.org/PBR/forthcoming.htm

Related Stories

Recommended for you

Depression screening rates in primary care remain low

February 20, 2017

Despite federal recommendations for depression screening, a new Rutgers study found that less than 5 percent of adults were screened for depression in primary care settings. The low screening rate suggests missed opportunities ...

What the ability to 'get the gist' says about your brain

February 17, 2017

Many who have a chronic traumatic brain injury (TBI) report struggling to solve problems, understand complex information and maintain friendships, despite scoring normally on cognitive tests. New research from the Center ...

B vitamins reduce schizophrenia symptoms, study finds

February 16, 2017

A review of worldwide studies has found that add-on treatment with high-dose b-vitamins - including B6, B8 and B12 - can significantly reduce symptoms of schizophrenia more than standard treatments alone.

Emotions are cognitive, not innate, researchers conclude

February 15, 2017

Emotions are not innately programmed into our brains, but, in fact, are cognitive states resulting from the gathering of information, New York University Professor Joseph LeDoux and Richard Brown, a professor at the City ...

People are found to be inefficient when searching for things

February 15, 2017

(Medical Xpress)—A trio of researchers at the University of Aberdeen in the U.K. has found that when people scan areas looking for something in particular, they tend to do so in a very inefficient manner. In their paper ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.