Changes to high-risk medical devices often supported by low-quality research

August 15, 2017, University of California, San Francisco
Credit: CC0 Public Domain

Clinical trials that test changes in the design or use of high-risk medical devices are often poorly designed, and can rely on inadequate or potentially biased data, according to a new study by researchers at the UC San Francisco and Yale School of Medicine.

In the study, published August 15, 2017 in JAMA, the authors reviewed supporting U.S. Food and Drug Administration (FDA) approval of changes to high-risk devices over the last decade. They found that fewer than half of these studies were randomized, blinded, or controlled - the "gold standards" for clinical trials of drugs.

The FDA defines high-risk devices as those that "that support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury."

The consequences of poor testing of medical devices can be severe several of the devices investigated in the clinical trials analyzed in the new paper had been recalled for safety or efficacy issues.

"There's a lot of pressure on the FDA to speed things up, to let innovative or life-changing treatments go to market," said Rita Redberg, MD, MSc, a professor of medicine at UCSF and the study's senior author, "but you don't know if they're innovative or life-changing until you've done a careful study. These were short studies, with low-quality data, and little follow-up after approval."

The paper is published alongside another from researchers at the London School of Economics and Harvard Medical School that notes similar problems in accelerated approval paths for pharmaceuticals. In an accompanying editorial, former FDA Commissioner Robert Califf MD, MACC, calling for continued discussion and reform of both approval processes.

Investigation found brief studies, incomplete data

To mitigate the potential hazards of high-risk medical devices, the FDA requires that these devices undergo rigorous initial clinical testing, known as premarket approval (PMA), before they are sold, a process that Redberg and her collaborators have been studying for years.

In a 2009 study, her research team found that, more often than not, clinical studies to support PMA lacked blinding, randomization, or proper controls. That study was a wake-up call for device regulators. "We got a ton of emails from people thanking us for exposing this important issue that nobody was talking about," said Redberg.

But the original PMA is seldom the whole story. As device makers update their devices - modifying the design or providing for different uses than the original purpose - they must submit "supplements" to the FDA for approval.

"Many devices can have hundreds of different supplements attached to them," said Sarah Zheng, MD, a resident physician in psychiatry at UCSF, and a co-lead author on the paper. "So we wanted to check the quality of evidence supporting these important changes as well."

Redberg and Zheng were joined in the research by co-lead author Sanket S. Dhruva, MD, who received his MD from UCSF in 2009, completed his residency in Internal medicine in 2009, and is now a Robert Wood Johnson Foundation Clinical Scholar at Yale School of Medicine.

There are many paths to supplement approval, and only one, the so-called panel track, requires clinical data, so the team investigated the rigor of 78 panel track supplement approvals, supported by 83 studies. The studies supporting supplements, they found, suffered the same problems as the PMA studies.

As with PMA studies, most supplement trials weren't blinded, randomized, or controlled. But in nearly a quarter of the supplement studies, the analysis procedures were changed after the study began, and many failed to present data from all the participants, potentially biasing the studies' results. Most studies lasted less than six months, and many presented incomplete data on age and sex, making it harder for physicians to evaluate the risk of a device in particular patients.

"Physicians and patients need to be aware of how little we actually know about these devices," said Zheng. "If patients need to have the device removed because it isn't functioning properly, that can be very risky and expensive."

Little recourse if devices harm patients

The investigation takes on particular significance because patients who are harmed by FDA-approved devices have few legal options. In 2008, the U.S. Supreme Court ruled in Riegel v. Medtronic, Inc. that FDA approval protects manufacturers from lawsuits over devices' safety or efficacy; this principle is likely to hold even if approval is based on a study that did not follow best practices. "If the FDA approved it, that's that," said Redberg.

In order to improve post-approval procedures, the FDA has discussed standardizing data on medical device failures, leveraging electronic medical records and assigning a unique ID to each device to make it easy for doctors to report failures or safety threats. "The FDA says they want to do more work on post-approval, but nothing's in place yet," said Redberg. "Even when they do ask for a post-approval study, it's often not completed or done at all."

"I think there has to be more public and congressional calls to have reasonable assurance of safety and effectiveness before devices are approved," said Redberg said. "Until then, all we know is that a lot of devices on the market haven't been shown to be safe or effective."

Explore further: Safety of medical devices not often evaluated by sex, age, or race

More information: JAMA (2017). … .1001/jama.2017.9414

Related Stories

Safety of medical devices not often evaluated by sex, age, or race

July 25, 2017
Researchers at Yale and the University of California-San Francisco have found that few medical devices are analyzed to consider the influence of their users' sex, age, or race on safety and effectiveness.

Patient safety merits new review for modified medical devices, physician says

March 25, 2014
For patient safety, the U.S. Food and Drug Administration (FDA) should require that clinical data be submitted as part of a more rigorous re-evaluation of medical devices that are modified after approval, according to UC ...

A call to prevent unsafe high-risk medical devices from reaching the marketplace

January 22, 2013
Technological advancements in medicine have allowed patients suffering from musculoskeletal conditions such as hip and knee pain to regain mobility and live relatively pain-free. But some "high risk" surgical devices that ...

Study examines high-risk therapeutic medical devices receiving FDA premarket approval

August 11, 2015
Of high-risk therapeutic devices approved via the Food and Drug Administration (FDA) Premarket Approval (PMA) pathway between 2010 and 2011, there has been wide variation in both the number and quality of premarket and postmarket ...

Flawed data behind regulation of high-risk women's health devices

May 20, 2016
Some high-risk medical devices used in obstetrics and gynecology were approved by the FDA based on flawed data, according to a recent study from Northwestern Medicine.

Most high-risk cardiac devices in use today approved as modifications to previously-approved devices

January 21, 2014
The Food and Drug Administration (FDA) ensures that high-risk medical devices, such as implantable cardioverter-defibrillators and pacemakers, are safe and effective through its premarket approval (PMA) pathway, during which ...

Recommended for you

Fabric imbued with optical fibers helps fight skin diseases

February 23, 2018
A team of researchers with Texinov Medical Textiles in France has announced that their PHOS-ISTOS system, called the Fluxmedicare, is on track to be made commercially available later this year. The system consists of a piece ...

Low-calorie diet enhances intestinal regeneration after injury

February 22, 2018
Dramatic calorie restriction, diets reduced by 40 percent of a normal calorie total, have long been known to extend health span, the duration of disease-free aging, in animal studies, and even to extend life span in most ...

Artificial intelligence quickly and accurately diagnoses eye diseases and pneumonia

February 22, 2018
Using artificial intelligence and machine learning techniques, researchers at Shiley Eye Institute at UC San Diego Health and University of California San Diego School of Medicine, with colleagues in China, Germany and Texas, ...

Gut microbes protect against sepsis—mouse study

February 22, 2018
Sepsis occurs when the body's response to the spread of bacteria or toxins to the bloodstream damages tissues and organs. The fight against sepsis could get a helping hand from a surprising source: gut bacteria. Researchers ...

Breakthrough could lead to better drugs to tackle diabetes and obesity

February 22, 2018
Breakthrough research at Monash University has shown how different areas of major diabetes and obesity drug targets can be 'activated', guiding future drug development and better treatment of diseases.

Fertility breakthrough: New research could extend egg health with age

February 22, 2018
Women have been told for years that if they don't have children before their mid-30s, they may not be able to. But a new study from Princeton University's Coleen Murphy has identified a drug that extends egg viability in ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.