Screeners miss the really rare stuff: Commonly found objects may be crowding out identification of the unusual items

December 3, 2013
A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's part of a handgun in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

A smartphone app that turns gamers into airport baggage screeners is showing that finding weapons and other illegal items isn't all that easy, even when you're looking for them.

Duke University researchers analyzed data from searches of 20 million virtual suitcases in the game Airport Scanner created by Kedlin Co. and found that users failed in most cases to identify objects that occurred only rarely.

"We're seeing that people are really bad at finding items that are not likely to appear," said Stephen Mitroff, Ph.D., associate professor of psychology and neuroscience and member of the Duke Institute for Brain Sciences.

In the game, players scan images that look like X-rays of carry-on luggage, trying to find hundreds of possible items, including guns and dynamite sticks, as well as over-sized bottles and scissors.

For this study – which appears online in Psychological Science – researchers evaluated gameplay data from December 2012 to March 2013 to determine how often the players found 78 different illegal items in light of how often the items appeared. Investigators used target frequencies (the appearance rate of a specific illegal object) to understand a player's success at identifying targets when they appear.

Thirty items were "ultra-rare," appearing in the game less than 0.15 percent of the time – a rate that is comparable to the presence of cancerous markers in real-life radiological screenings, according to Mitroff. Yet the study results revealed players correctly identified those 30 targets only 27 percent of the time. Targets that appeared with more than 1 percent frequency were pinpointed 92 percent of the time.

A screen shot from the smartphone game Airport Scanner shows simulated baggage screening. There's a stick of dynamite in this bag. Duke researchers used more than 20 million bag searches from the game to discover that very rare items often go undetected. Credit: Airport Scanner by Kedlin

"This isn't a matter of overall vigilance or how frequently players responded, since half of the searches had a to-be-found item present," Mitroff said. "This effect is about being able to detect specific items and how likely you are to miss them when they occur infrequently."

For example, airport screeners are more likely to encounter pocket knives or incorrectly-packed prescription medications than they are to see a gun in a carry-on bag. So, they might become conditioned to catch the illegal, though relatively harmless, items that appear more frequently than a rarely-seen weapon.

"With the very large array of potential targets, the searchers seem to be highly sensitive to how frequently each individual target appeared, and they adjusted the focus of their searches so that targets that appeared rarely were a low priority," Mitroff said. "This situation may be what occurs in many real-world searches."

This research is important, Mitroff said, because it explores how rare objects slip past us no matter how hard we're looking for them or how critical finding them might be, something he and co-author Adam Biggs, a post-doctoral researcher from the psychology and neuroscience department in the Center for Cognitive Neuroscience, have termed the "ultra-rare item effect."

The strong relationship between the frequencies and associated detection rates surprised Mitroff's team even though they anticipated a connection. Based on these results, Mitroff said, the door is open to further research into how behaviors can be modified to catch more ultra-rare items.

The potential is also there, Mitroff said, for the Airport Scanner game to play a role in Transportation Security Administration training and standard operating procedures. The research team now has access to more than 1.5 billion trials from the for analysis.

"We want to better understand why this happens." he said. "Are you attuned to more frequent items at the cost of others? If you don't see something very often, it makes sense that you don't want to waste energy looking for it, but at the end of the day you still want to detect some of these ultra-rare items."

The anonymous game data were shared in accordance with the Standard Apple User agreement, and Duke University approved the analysis of the data. This research received no external funding support.

Explore further: Videogamers no better at talking while driving

More information: "The Ultra-Rare Item Effect: Exceedingly rare items are highly susceptible to search errors," Stephen R. Mitroff, Adam T. Biggs. Psychological Science, early online Nov. 22, 2013. DOI: 10.1177/0956797613504221

Related Stories

Videogamers no better at talking while driving

June 13, 2012

No matter how much time you've spent training your brain to multitask by playing "Call of Duty," you're probably no better at talking on the phone while driving than anybody else.

Video gamers really do see more, research says

June 11, 2013

Hours spent at the video gaming console not only train a player's hands to work the buttons on the controller, they probably also train the brain to make better and faster use of visual input, according to Duke University ...

Slow and steady wins the baggage search

June 17, 2013

Next time you're doing a slow burn in security screening at the airport, calm yourself with the assurance that a more deliberate baggage scanner may do a better job.

Recommended for you

How much video gaming is too much for kids?

September 27, 2016

(HealthDay)—Playing video games might improve a child's motor skills, reaction time and even academic performance, but new research shows that too much gaming can be linked to social and behavioral problems.

How we handle objects depends on who owns them

September 27, 2016

From scissors and staplers to car keys and cell phones, we pass objects to other people every day. We often try to pass the objects so that the handle or other useful feature is facing the appropriate direction for the person ...

Dogs ignore bad advice that humans follow

September 27, 2016

Dogs are less likely to follow bad advice than children, according to a new study conducted at the Canine Cognition Center at Yale. In contrast to children, dogs only copy a human's actions if they are absolutely necessary ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.