Screeners miss the really rare stuff: Commonly found objects may be crowding out identification of the unusual items

December 3, 2013, Duke University
A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's part of a handgun in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

A smartphone app that turns gamers into airport baggage screeners is showing that finding weapons and other illegal items isn't all that easy, even when you're looking for them.

Duke University researchers analyzed data from searches of 20 million virtual suitcases in the game Airport Scanner created by Kedlin Co. and found that users failed in most cases to identify objects that occurred only rarely.

"We're seeing that people are really bad at finding items that are not likely to appear," said Stephen Mitroff, Ph.D., associate professor of psychology and neuroscience and member of the Duke Institute for Brain Sciences.

In the game, players scan images that look like X-rays of carry-on luggage, trying to find hundreds of possible items, including guns and dynamite sticks, as well as over-sized bottles and scissors.

For this study – which appears online in Psychological Science – researchers evaluated gameplay data from December 2012 to March 2013 to determine how often the players found 78 different illegal items in light of how often the items appeared. Investigators used target frequencies (the appearance rate of a specific illegal object) to understand a player's success at identifying targets when they appear.

Thirty items were "ultra-rare," appearing in the game less than 0.15 percent of the time – a rate that is comparable to the presence of cancerous markers in real-life radiological screenings, according to Mitroff. Yet the study results revealed players correctly identified those 30 targets only 27 percent of the time. Targets that appeared with more than 1 percent frequency were pinpointed 92 percent of the time.

A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's a stick of dynamite in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

"This isn't a matter of overall vigilance or how frequently players responded, since half of the searches had a to-be-found item present," Mitroff said. "This effect is about being able to detect specific items and how likely you are to miss them when they occur infrequently."

For example, airport screeners are more likely to encounter pocket knives or incorrectly-packed prescription medications than they are to see a gun in a carry-on bag. So, they might become conditioned to catch the illegal, though relatively harmless, items that appear more frequently than a rarely-seen weapon.

"With the very large array of potential targets, the searchers seem to be highly sensitive to how frequently each individual target appeared, and they adjusted the focus of their searches so that targets that appeared rarely were a low priority," Mitroff said. "This situation may be what occurs in many real-world searches."

This research is important, Mitroff said, because it explores how rare objects slip past us no matter how hard we're looking for them or how critical finding them might be, something he and co-author Adam Biggs, a post-doctoral researcher from the psychology and neuroscience department in the Center for Cognitive Neuroscience, have termed the "ultra-rare item effect."

The strong relationship between the frequencies and associated detection rates surprised Mitroff's team even though they anticipated a connection. Based on these results, Mitroff said, the door is open to further research into how behaviors can be modified to catch more ultra-rare items.

The potential is also there, Mitroff said, for the Airport Scanner game to play a role in Transportation Security Administration training and standard operating procedures. The research team now has access to more than 1.5 billion trials from the for analysis.

"We want to better understand why this happens." he said. "Are you attuned to more frequent items at the cost of others? If you don't see something very often, it makes sense that you don't want to waste energy looking for it, but at the end of the day you still want to detect some of these ultra-rare items."

The anonymous game data were shared in accordance with the Standard Apple User agreement, and Duke University approved the analysis of the data. This research received no external funding support.

Explore further: Slow and steady wins the baggage search

More information: "The Ultra-Rare Item Effect: Exceedingly rare items are highly susceptible to search errors," Stephen R. Mitroff, Adam T. Biggs. Psychological Science, early online Nov. 22, 2013. DOI: 10.1177/0956797613504221

Related Stories

Slow and steady wins the baggage search

June 17, 2013
Next time you're doing a slow burn in security screening at the airport, calm yourself with the assurance that a more deliberate baggage scanner may do a better job.

Searching for tumors or handguns can be like looking for food

August 7, 2012
If past experience makes you think there's going to be one more cashew at the bottom of the bowl, you're likely to search through those mixed nuts a little longer.

Anxious searchers miss multiple objects

June 15, 2011
A person scanning baggage or X-rays stands a better chance of seeing everything they're searching for if they aren't feeling anxious, according to a new laboratory experiment.

Videogamers no better at talking while driving

June 13, 2012
No matter how much time you've spent training your brain to multitask by playing "Call of Duty," you're probably no better at talking on the phone while driving than anybody else.

Video gamers really do see more, research says

June 11, 2013
Hours spent at the video gaming console not only train a player's hands to work the buttons on the controller, they probably also train the brain to make better and faster use of visual input, according to Duke University ...

Recommended for you

Intensive behavior therapy no better than conventional support in treating teenagers with antisocial behavior

January 19, 2018
Research led by UCL has found that intensive and costly multisystemic therapy is no better than conventional therapy in treating teenagers with moderate to severe antisocial behaviour.

Babies' babbling betters brains, language

January 18, 2018
Babies are adept at getting what they need - including an education. New research shows that babies organize mothers' verbal responses, which promotes more effective language instruction, and infant babbling is the key.

College branding makes beer more salient to underage students

January 18, 2018
In recent years, major beer companies have tried to capitalize on the salience of students' university affiliations, unveiling marketing campaigns and products—such as "fan cans," store displays, and billboard ads—that ...

Inherited IQ can increase in early childhood

January 18, 2018
When it comes to intelligence, environment and education matter – more than we think.

Modulating molecules: Study shows oxytocin helps the brain to modulate social signals

January 17, 2018
Between sights, sounds, smells and other senses, the brain is flooded with stimuli on a moment-to-moment basis. How can it sort through the flood of information to decide what is important and what can be relegated to the ...

Baby brains help infants figure it out before they try it out

January 17, 2018
Babies often amaze their parents when they seemingly learn new skills overnight—how to walk, for example. But their brains were probably prepping for those tasks long before their first steps occurred, according to researchers.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.