Crowdsourcing contest using data from people, dogs advances epileptic seizure forecasting

It might sound like a riddle: What do you get when you combine one online contest, two patients, five dogs and 654 data scientists?

The answer: Hope for patients with epilepsy that their seizures can be reliably predicted, and perhaps prevented.

A study published in the June issue of the journal Brain reports on a 2014 online contest in which more than 500 teams of data scientists from all over the world analyzed hundreds of hours of recordings of electrical activity in the brains of two people and five dogs before and during epileptic seizures.

The contest result? The wisdom of the crowd created a better-than-random prediction of epileptic seizures. In what has long been a highly contentious field of research, with limited reproducibility of results, this study has made all the data and analysis code freely available.

For patients, predictability of seizures could be life changing. "If an algorithm could detect subtle changes in the electrical activity of a person's brain before a seizure occurs, people with epilepsy could take medications only when needed and possibly reclaim the daily activities many of us take for granted," says Ben Brinkmann, Ph.D., a data scientist at Mayo Clinic and lead author of the study by Mayo Clinic, University of Pennsylvania and University of Minnesota.

During seizures, abnormal electrical activity in the brain sometimes results in loss of consciousness. For people with epilepsy, the unpredictability of seizures severely limits their ability to perform tasks where even a momentary loss of consciousness could prove disastrous—driving a car, swimming or holding an infant, for example. Approximately 50 million people worldwide have epilepsy, according to the World Health Organization.

Development of reliable seizure prediction algorithms requires data on many seizures and non-seizure brain activity as measured by implanted electroencephalography (EEG). But the large quantities of long-duration data needed are difficult to acquire and have not been available. Typically, EEG recordings lasting seven days or fewer are taken before surgery when a patient's medications are reduced to encourage seizures. These recordings offer a limited view of brain function under changing conditions.

Even when a working algorithm is created, investigators typically don't share algorithm code because of concerns related to privacy, cost of sharing large datasets, protecting their research and intellectual property. There is also a lack of incentives to share data. The result is a vast literature that is difficult to compare and impossible to reproduce.

Enter crowdsourcing and data sharing. If, as the old adage says, two heads are better than one, then crowdsourcing goes the next step by seeking online help from a large group of people to solve a problem.

"In the hope of winning up to $15,000 in prize money and bragging rights in data science circles, hundreds of algorithm developers, most with little or no experience with epilepsy or EEG, worked countless hours to build, test and rebuild algorithms for seizure forecasting," Dr. Brinkmann says.

The novel data from canines with epilepsy were obtained with the Seizure Advisory System developed by Seattle-based NeuroVista Corp., an early-stage medical device company that folded in 2014. The datasets and source code remain openly available from the Mayo Clinic Systems Electrophysiology Lab and the International Epilepsy Electrophysiology Portal for researchers to continue developing new algorithms for predicting seizures.

"These results support our effort to develop the next generation of epilepsy devices designed to continuously monitor brain activity, forecast and prevent seizures," says neurologist Greg Worrell, M.D., PhD., Mayo Clinic. "These datasets and source code also serve as a benchmark, allowing new algorithms to be compared to each other and to the algorithms developed in this competition."

"This is an incredibly dynamic and powerful way to do community research and bring new minds into the challenge of epilepsy," says Brian Litt, M.D., Perelman School of Medicine at the University of Pennsylvania, Philadelphia.

"I think this effort highlights the potential of medical devices that quantify disease burden over clinically meaningful time frames," says Kent Leyde, former chief technology officer of NeuroVista. "Long-term records, such as those collected by the NeuroVista system, help us better understand the disease process, and their availability provides a basis for collaborating with experts from other fields to devise new methods to treat and manage disease."

The approach would not be possible without making the data openly available. Data scientists from across the globe tested algorithms on nearly 350 seizures recorded over more than 1,500 days and the winners agreed to make their computer code freely available. During the four-month contest, over half of these crowdsourced algorithms performed better than random predictions. The best performing algorithms accurately predicted more than 70 percent of seizures when tested on unseen portions of the canine data.

"The dog EEG data has great bidirectional benefits, both as a comparative model for human epilepsy and also to directly benefit dogs with naturally occurring epilepsy," says Ned Patterson, Ph.D., D.V.M., University of Minnesota.

The effort to forecast seizures continues under a five-year federal grant awarded to the Mayo team in 2015 from the National Institutes of Health. The grant, part of a presidential initiative aimed at revolutionizing the understanding of the human brain, is called Brain Research Through Advancing Innovative Neurotechnologies, or BRAIN Initiative.

Working with Medtronic Inc., the Mayo team will test the safety and efficacy of seizure forecasting initially in canines, with human trials to follow.