Changes to high-risk medical devices often supported by low-quality research

August 15, 2017, University of California, San Francisco
Credit: CC0 Public Domain

Clinical trials that test changes in the design or use of high-risk medical devices are often poorly designed, and can rely on inadequate or potentially biased data, according to a new study by researchers at the UC San Francisco and Yale School of Medicine.

In the study, published August 15, 2017 in JAMA, the authors reviewed supporting U.S. Food and Drug Administration (FDA) approval of changes to high-risk devices over the last decade. They found that fewer than half of these studies were randomized, blinded, or controlled - the "gold standards" for clinical trials of drugs.

The FDA defines high-risk devices as those that "that support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury."

The consequences of poor testing of medical devices can be severe several of the devices investigated in the clinical trials analyzed in the new paper had been recalled for safety or efficacy issues.

"There's a lot of pressure on the FDA to speed things up, to let innovative or life-changing treatments go to market," said Rita Redberg, MD, MSc, a professor of medicine at UCSF and the study's senior author, "but you don't know if they're innovative or life-changing until you've done a careful study. These were short studies, with low-quality data, and little follow-up after approval."

The paper is published alongside another from researchers at the London School of Economics and Harvard Medical School that notes similar problems in accelerated approval paths for pharmaceuticals. In an accompanying editorial, former FDA Commissioner Robert Califf MD, MACC, calling for continued discussion and reform of both approval processes.

Investigation found brief studies, incomplete data

To mitigate the potential hazards of high-risk medical devices, the FDA requires that these devices undergo rigorous initial clinical testing, known as premarket approval (PMA), before they are sold, a process that Redberg and her collaborators have been studying for years.

In a 2009 study, her research team found that, more often than not, clinical studies to support PMA lacked blinding, randomization, or proper controls. That study was a wake-up call for device regulators. "We got a ton of emails from people thanking us for exposing this important issue that nobody was talking about," said Redberg.

But the original PMA is seldom the whole story. As device makers update their devices - modifying the design or providing for different uses than the original purpose - they must submit "supplements" to the FDA for approval.

"Many devices can have hundreds of different supplements attached to them," said Sarah Zheng, MD, a resident physician in psychiatry at UCSF, and a co-lead author on the paper. "So we wanted to check the quality of evidence supporting these important changes as well."

Redberg and Zheng were joined in the research by co-lead author Sanket S. Dhruva, MD, who received his MD from UCSF in 2009, completed his residency in Internal medicine in 2009, and is now a Robert Wood Johnson Foundation Clinical Scholar at Yale School of Medicine.

There are many paths to supplement approval, and only one, the so-called panel track, requires clinical data, so the team investigated the rigor of 78 panel track supplement approvals, supported by 83 studies. The studies supporting supplements, they found, suffered the same problems as the PMA studies.

As with PMA studies, most supplement trials weren't blinded, randomized, or controlled. But in nearly a quarter of the supplement studies, the analysis procedures were changed after the study began, and many failed to present data from all the participants, potentially biasing the studies' results. Most studies lasted less than six months, and many presented incomplete data on age and sex, making it harder for physicians to evaluate the risk of a device in particular patients.

"Physicians and patients need to be aware of how little we actually know about these devices," said Zheng. "If patients need to have the device removed because it isn't functioning properly, that can be very risky and expensive."

Little recourse if devices harm patients

The investigation takes on particular significance because patients who are harmed by FDA-approved devices have few legal options. In 2008, the U.S. Supreme Court ruled in Riegel v. Medtronic, Inc. that FDA approval protects manufacturers from lawsuits over devices' safety or efficacy; this principle is likely to hold even if approval is based on a study that did not follow best practices. "If the FDA approved it, that's that," said Redberg.

In order to improve post-approval procedures, the FDA has discussed standardizing data on medical device failures, leveraging electronic medical records and assigning a unique ID to each device to make it easy for doctors to report failures or safety threats. "The FDA says they want to do more work on post-approval, but nothing's in place yet," said Redberg. "Even when they do ask for a post-approval study, it's often not completed or done at all."

"I think there has to be more public and congressional calls to have reasonable assurance of safety and effectiveness before devices are approved," said Redberg said. "Until then, all we know is that a lot of devices on the market haven't been shown to be safe or effective."

Explore further: Safety of medical devices not often evaluated by sex, age, or race

More information: JAMA (2017). … .1001/jama.2017.9414

Related Stories

Safety of medical devices not often evaluated by sex, age, or race

July 25, 2017
Researchers at Yale and the University of California-San Francisco have found that few medical devices are analyzed to consider the influence of their users' sex, age, or race on safety and effectiveness.

Patient safety merits new review for modified medical devices, physician says

March 25, 2014
For patient safety, the U.S. Food and Drug Administration (FDA) should require that clinical data be submitted as part of a more rigorous re-evaluation of medical devices that are modified after approval, according to UC ...

A call to prevent unsafe high-risk medical devices from reaching the marketplace

January 22, 2013
Technological advancements in medicine have allowed patients suffering from musculoskeletal conditions such as hip and knee pain to regain mobility and live relatively pain-free. But some "high risk" surgical devices that ...

Study examines high-risk therapeutic medical devices receiving FDA premarket approval

August 11, 2015
Of high-risk therapeutic devices approved via the Food and Drug Administration (FDA) Premarket Approval (PMA) pathway between 2010 and 2011, there has been wide variation in both the number and quality of premarket and postmarket ...

Flawed data behind regulation of high-risk women's health devices

May 20, 2016
Some high-risk medical devices used in obstetrics and gynecology were approved by the FDA based on flawed data, according to a recent study from Northwestern Medicine.

Most high-risk cardiac devices in use today approved as modifications to previously-approved devices

January 21, 2014
The Food and Drug Administration (FDA) ensures that high-risk medical devices, such as implantable cardioverter-defibrillators and pacemakers, are safe and effective through its premarket approval (PMA) pathway, during which ...

Recommended for you

Separated entry and exit doors for calcium keep energy production smooth in the powerhouses of heart cells

September 18, 2018
Stress demands the heart to work harder and faster. To keep pace, the muscle must make its fuel at an accelerated rate. Bursts of calcium entering mitochondria—the cell's powerhouses—normally help control energy output, ...

A new defender for your sense of smell

September 18, 2018
New research from the Monell Center increases understanding of a mysterious sensory cell located in the olfactory epithelium, the patch of nasal tissue that contains odor-detecting olfactory receptor cells. The findings suggest ...

First gut bacteria may have lasting effect on ability to fight chronic diseases

September 18, 2018
New research showing that the first bacteria introduced into the gut have a lasting impact may one day allow science to adjust microbiomes—the one-of-a-kind microbial communities that live in our gastrointestinal tracts—to ...

Small molecule plays big role in weaker bones as we age

September 18, 2018
With age, expression of a small molecule that can silence others goes way up while a key signaling molecule that helps stem cells make healthy bone goes down, scientists report.

Sperm quality study updates advice for couples trying to conceive

September 17, 2018
Could doctors at fertility clinics be giving men bad advice? Dr. Da Li and Dr. XiuXia Wang, who are clinician-researchers at the Center for Reproductive Medicine of Shengjing Hospital in Shenyang in northeast China, think ...

Antioxidant found to be effective in treating mice with osteoarthritis

September 14, 2018
A team of researchers in Belgium and the Netherlands has found that feeding a common antioxidant to test mice was effective in treating osteoarthritis. In their paper published in Science Translational Medicine, the group ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.