Screeners miss the really rare stuff: Commonly found objects may be crowding out identification of the unusual items

December 3, 2013
A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's part of a handgun in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

A smartphone app that turns gamers into airport baggage screeners is showing that finding weapons and other illegal items isn't all that easy, even when you're looking for them.

Duke University researchers analyzed data from searches of 20 million virtual suitcases in the game Airport Scanner created by Kedlin Co. and found that users failed in most cases to identify objects that occurred only rarely.

"We're seeing that people are really bad at finding items that are not likely to appear," said Stephen Mitroff, Ph.D., associate professor of psychology and neuroscience and member of the Duke Institute for Brain Sciences.

In the game, players scan images that look like X-rays of carry-on luggage, trying to find hundreds of possible items, including guns and dynamite sticks, as well as over-sized bottles and scissors.

For this study – which appears online in Psychological Science – researchers evaluated gameplay data from December 2012 to March 2013 to determine how often the players found 78 different illegal items in light of how often the items appeared. Investigators used target frequencies (the appearance rate of a specific illegal object) to understand a player's success at identifying targets when they appear.

Thirty items were "ultra-rare," appearing in the game less than 0.15 percent of the time – a rate that is comparable to the presence of cancerous markers in real-life radiological screenings, according to Mitroff. Yet the study results revealed players correctly identified those 30 targets only 27 percent of the time. Targets that appeared with more than 1 percent frequency were pinpointed 92 percent of the time.

A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's a stick of dynamite in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

"This isn't a matter of overall vigilance or how frequently players responded, since half of the searches had a to-be-found item present," Mitroff said. "This effect is about being able to detect specific items and how likely you are to miss them when they occur infrequently."

For example, airport screeners are more likely to encounter pocket knives or incorrectly-packed prescription medications than they are to see a gun in a carry-on bag. So, they might become conditioned to catch the illegal, though relatively harmless, items that appear more frequently than a rarely-seen weapon.

"With the very large array of potential targets, the searchers seem to be highly sensitive to how frequently each individual target appeared, and they adjusted the focus of their searches so that targets that appeared rarely were a low priority," Mitroff said. "This situation may be what occurs in many real-world searches."

This research is important, Mitroff said, because it explores how rare objects slip past us no matter how hard we're looking for them or how critical finding them might be, something he and co-author Adam Biggs, a post-doctoral researcher from the psychology and neuroscience department in the Center for Cognitive Neuroscience, have termed the "ultra-rare item effect."

The strong relationship between the frequencies and associated detection rates surprised Mitroff's team even though they anticipated a connection. Based on these results, Mitroff said, the door is open to further research into how behaviors can be modified to catch more ultra-rare items.

The potential is also there, Mitroff said, for the Airport Scanner game to play a role in Transportation Security Administration training and standard operating procedures. The research team now has access to more than 1.5 billion trials from the for analysis.

"We want to better understand why this happens." he said. "Are you attuned to more frequent items at the cost of others? If you don't see something very often, it makes sense that you don't want to waste energy looking for it, but at the end of the day you still want to detect some of these ultra-rare items."

The anonymous game data were shared in accordance with the Standard Apple User agreement, and Duke University approved the analysis of the data. This research received no external funding support.

Explore further: Slow and steady wins the baggage search

More information: "The Ultra-Rare Item Effect: Exceedingly rare items are highly susceptible to search errors," Stephen R. Mitroff, Adam T. Biggs. Psychological Science, early online Nov. 22, 2013. DOI: 10.1177/0956797613504221

Related Stories

Slow and steady wins the baggage search

June 17, 2013
Next time you're doing a slow burn in security screening at the airport, calm yourself with the assurance that a more deliberate baggage scanner may do a better job.

Searching for tumors or handguns can be like looking for food

August 7, 2012
If past experience makes you think there's going to be one more cashew at the bottom of the bowl, you're likely to search through those mixed nuts a little longer.

Anxious searchers miss multiple objects

June 15, 2011
A person scanning baggage or X-rays stands a better chance of seeing everything they're searching for if they aren't feeling anxious, according to a new laboratory experiment.

Videogamers no better at talking while driving

June 13, 2012
No matter how much time you've spent training your brain to multitask by playing "Call of Duty," you're probably no better at talking on the phone while driving than anybody else.

Video gamers really do see more, research says

June 11, 2013
Hours spent at the video gaming console not only train a player's hands to work the buttons on the controller, they probably also train the brain to make better and faster use of visual input, according to Duke University ...

Recommended for you

Perceiving oneself as less physically active than peers is linked to a shorter lifespan

July 20, 2017
Would you say that you are physically more active, less active, or about equally active as other people your age?

Study examines effects of stopping psychiatric medication

July 20, 2017
Despite numerous obstacles and severe withdrawal effects, long-term users of psychiatric drugs can stop taking them if they choose, and mental health care professionals could be more helpful to such individuals, according ...

New study suggests that reduced insurance coverage for mental health treatment increases costs for the seriously ill

July 19, 2017
Higher out-of-pocket costs for mental health care could have the unintended consequence of increasing the use of acute and involuntary mental health care among those suffering from the most debilitating disorders, a Harvard ...

Old antibiotic could form new depression treatment

July 19, 2017
An antibiotic used mostly to treat acne has been found to improve the quality of life for people with major depression, in a world-first clinical trial conducted at Deakin University.

Wonder why those happy memories fade? You're programmed that way

July 19, 2017
We'll always have Paris." Or will we?

A child's spoken vocabulary helps them when it comes to reading new words for the first time

July 19, 2017
Children find it easier to spell a word when they've already heard it spoken, a new study led by researchers from the ARC Centre of Excellence in Cognition and its Disorders (CCD) at Macquarie University has found. The findings ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.