Changes to high-risk medical devices often supported by low-quality research

August 15, 2017
Credit: CC0 Public Domain

Clinical trials that test changes in the design or use of high-risk medical devices are often poorly designed, and can rely on inadequate or potentially biased data, according to a new study by researchers at the UC San Francisco and Yale School of Medicine.

In the study, published August 15, 2017 in JAMA, the authors reviewed supporting U.S. Food and Drug Administration (FDA) approval of changes to high-risk devices over the last decade. They found that fewer than half of these studies were randomized, blinded, or controlled - the "gold standards" for clinical trials of drugs.

The FDA defines high-risk devices as those that "that support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury."

The consequences of poor testing of medical devices can be severe several of the devices investigated in the clinical trials analyzed in the new paper had been recalled for safety or efficacy issues.

"There's a lot of pressure on the FDA to speed things up, to let innovative or life-changing treatments go to market," said Rita Redberg, MD, MSc, a professor of medicine at UCSF and the study's senior author, "but you don't know if they're innovative or life-changing until you've done a careful study. These were short studies, with low-quality data, and little follow-up after approval."

The paper is published alongside another from researchers at the London School of Economics and Harvard Medical School that notes similar problems in accelerated approval paths for pharmaceuticals. In an accompanying editorial, former FDA Commissioner Robert Califf MD, MACC, calling for continued discussion and reform of both approval processes.

Investigation found brief studies, incomplete data

To mitigate the potential hazards of high-risk medical devices, the FDA requires that these devices undergo rigorous initial clinical testing, known as premarket approval (PMA), before they are sold, a process that Redberg and her collaborators have been studying for years.

In a 2009 study, her research team found that, more often than not, clinical studies to support PMA lacked blinding, randomization, or proper controls. That study was a wake-up call for device regulators. "We got a ton of emails from people thanking us for exposing this important issue that nobody was talking about," said Redberg.

But the original PMA is seldom the whole story. As device makers update their devices - modifying the design or providing for different uses than the original purpose - they must submit "supplements" to the FDA for approval.

"Many devices can have hundreds of different supplements attached to them," said Sarah Zheng, MD, a resident physician in psychiatry at UCSF, and a co-lead author on the paper. "So we wanted to check the quality of evidence supporting these important changes as well."

Redberg and Zheng were joined in the research by co-lead author Sanket S. Dhruva, MD, who received his MD from UCSF in 2009, completed his residency in Internal medicine in 2009, and is now a Robert Wood Johnson Foundation Clinical Scholar at Yale School of Medicine.

There are many paths to supplement approval, and only one, the so-called panel track, requires clinical data, so the team investigated the rigor of 78 panel track supplement approvals, supported by 83 studies. The studies supporting supplements, they found, suffered the same problems as the PMA studies.

As with PMA studies, most supplement trials weren't blinded, randomized, or controlled. But in nearly a quarter of the supplement studies, the analysis procedures were changed after the study began, and many failed to present data from all the participants, potentially biasing the studies' results. Most studies lasted less than six months, and many presented incomplete data on age and sex, making it harder for physicians to evaluate the risk of a device in particular patients.

"Physicians and patients need to be aware of how little we actually know about these devices," said Zheng. "If patients need to have the device removed because it isn't functioning properly, that can be very risky and expensive."

Little recourse if devices harm patients

The investigation takes on particular significance because patients who are harmed by FDA-approved devices have few legal options. In 2008, the U.S. Supreme Court ruled in Riegel v. Medtronic, Inc. that FDA approval protects manufacturers from lawsuits over devices' safety or efficacy; this principle is likely to hold even if approval is based on a study that did not follow best practices. "If the FDA approved it, that's that," said Redberg.

In order to improve post-approval procedures, the FDA has discussed standardizing data on medical device failures, leveraging electronic medical records and assigning a unique ID to each device to make it easy for doctors to report failures or safety threats. "The FDA says they want to do more work on post-approval, but nothing's in place yet," said Redberg. "Even when they do ask for a post-approval study, it's often not completed or done at all."

"I think there has to be more public and congressional calls to have reasonable assurance of safety and effectiveness before devices are approved," said Redberg said. "Until then, all we know is that a lot of devices on the market haven't been shown to be safe or effective."

Explore further: Safety of medical devices not often evaluated by sex, age, or race

More information: JAMA (2017). … .1001/jama.2017.9414

Related Stories

Safety of medical devices not often evaluated by sex, age, or race

July 25, 2017
Researchers at Yale and the University of California-San Francisco have found that few medical devices are analyzed to consider the influence of their users' sex, age, or race on safety and effectiveness.

Patient safety merits new review for modified medical devices, physician says

March 25, 2014
For patient safety, the U.S. Food and Drug Administration (FDA) should require that clinical data be submitted as part of a more rigorous re-evaluation of medical devices that are modified after approval, according to UC ...

A call to prevent unsafe high-risk medical devices from reaching the marketplace

January 22, 2013
Technological advancements in medicine have allowed patients suffering from musculoskeletal conditions such as hip and knee pain to regain mobility and live relatively pain-free. But some "high risk" surgical devices that ...

Study examines high-risk therapeutic medical devices receiving FDA premarket approval

August 11, 2015
Of high-risk therapeutic devices approved via the Food and Drug Administration (FDA) Premarket Approval (PMA) pathway between 2010 and 2011, there has been wide variation in both the number and quality of premarket and postmarket ...

Flawed data behind regulation of high-risk women's health devices

May 20, 2016
Some high-risk medical devices used in obstetrics and gynecology were approved by the FDA based on flawed data, according to a recent study from Northwestern Medicine.

Most high-risk cardiac devices in use today approved as modifications to previously-approved devices

January 21, 2014
The Food and Drug Administration (FDA) ensures that high-risk medical devices, such as implantable cardioverter-defibrillators and pacemakers, are safe and effective through its premarket approval (PMA) pathway, during which ...

Recommended for you

Estrogen discovery could shed new light on fertility problems

December 12, 2017
Estrogen produced in the brain is necessary for ovulation in monkeys, according to researchers at the University of Wisconsin-Madison who have upended the traditional understanding of the hormonal cascade that leads to release ...

Time of day affects severity of autoimmune disease

December 12, 2017
Insights into how the body clock and time of day influence immune responses are revealed today in a study published in leading international journal Nature Communications. Understanding the effect of the interplay between ...

3-D printed microfibers could provide structure for artificially grown body parts

December 12, 2017
Much as a frame provides structural support for a house and the chassis provides strength and shape for a car, a team of Penn State engineers believe they have a way to create the structural framework for growing living tissue ...

Team identifies DNA element that may cause rare movement disorder

December 11, 2017
A team of Massachusetts General Hospital (MGH) researchers has identified a specific genetic change that may be the cause of a rare but severe neurological disorder called X-linked dystonia parkinsonism (XDP). Occurring only ...

Protein Daple coordinates single-cell and organ-wide directionality in the inner ear

December 11, 2017
Humans inherited the capacity to hear sounds thanks to structures that evolved millions of years ago. Sensory "hair cells" in the inner ear have the amazing ability to convert sound waves into electrical signals and transmit ...

Gene therapy improves immunity in babies with 'bubble boy' disease

December 9, 2017
Early evidence suggests that gene therapy developed at St. Jude Children's Research Hospital will lead to broad protection for infants with the devastating immune disorder X-linked severe combined immunodeficiency disorder. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.