Changes to high-risk medical devices often supported by low-quality research

August 15, 2017, University of California, San Francisco
Credit: CC0 Public Domain

Clinical trials that test changes in the design or use of high-risk medical devices are often poorly designed, and can rely on inadequate or potentially biased data, according to a new study by researchers at the UC San Francisco and Yale School of Medicine.

In the study, published August 15, 2017 in JAMA, the authors reviewed supporting U.S. Food and Drug Administration (FDA) approval of changes to high-risk devices over the last decade. They found that fewer than half of these studies were randomized, blinded, or controlled - the "gold standards" for clinical trials of drugs.

The FDA defines high-risk devices as those that "that support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury."

The consequences of poor testing of medical devices can be severe several of the devices investigated in the clinical trials analyzed in the new paper had been recalled for safety or efficacy issues.

"There's a lot of pressure on the FDA to speed things up, to let innovative or life-changing treatments go to market," said Rita Redberg, MD, MSc, a professor of medicine at UCSF and the study's senior author, "but you don't know if they're innovative or life-changing until you've done a careful study. These were short studies, with low-quality data, and little follow-up after approval."

The paper is published alongside another from researchers at the London School of Economics and Harvard Medical School that notes similar problems in accelerated approval paths for pharmaceuticals. In an accompanying editorial, former FDA Commissioner Robert Califf MD, MACC, calling for continued discussion and reform of both approval processes.

Investigation found brief studies, incomplete data

To mitigate the potential hazards of high-risk medical devices, the FDA requires that these devices undergo rigorous initial clinical testing, known as premarket approval (PMA), before they are sold, a process that Redberg and her collaborators have been studying for years.

In a 2009 study, her research team found that, more often than not, clinical studies to support PMA lacked blinding, randomization, or proper controls. That study was a wake-up call for device regulators. "We got a ton of emails from people thanking us for exposing this important issue that nobody was talking about," said Redberg.

But the original PMA is seldom the whole story. As device makers update their devices - modifying the design or providing for different uses than the original purpose - they must submit "supplements" to the FDA for approval.

"Many devices can have hundreds of different supplements attached to them," said Sarah Zheng, MD, a resident physician in psychiatry at UCSF, and a co-lead author on the paper. "So we wanted to check the quality of evidence supporting these important changes as well."

Redberg and Zheng were joined in the research by co-lead author Sanket S. Dhruva, MD, who received his MD from UCSF in 2009, completed his residency in Internal medicine in 2009, and is now a Robert Wood Johnson Foundation Clinical Scholar at Yale School of Medicine.

There are many paths to supplement approval, and only one, the so-called panel track, requires clinical data, so the team investigated the rigor of 78 panel track supplement approvals, supported by 83 studies. The studies supporting supplements, they found, suffered the same problems as the PMA studies.

As with PMA studies, most supplement trials weren't blinded, randomized, or controlled. But in nearly a quarter of the supplement studies, the analysis procedures were changed after the study began, and many failed to present data from all the participants, potentially biasing the studies' results. Most studies lasted less than six months, and many presented incomplete data on age and sex, making it harder for physicians to evaluate the risk of a device in particular patients.

"Physicians and patients need to be aware of how little we actually know about these devices," said Zheng. "If patients need to have the device removed because it isn't functioning properly, that can be very risky and expensive."

Little recourse if devices harm patients

The investigation takes on particular significance because patients who are harmed by FDA-approved devices have few legal options. In 2008, the U.S. Supreme Court ruled in Riegel v. Medtronic, Inc. that FDA approval protects manufacturers from lawsuits over devices' safety or efficacy; this principle is likely to hold even if approval is based on a study that did not follow best practices. "If the FDA approved it, that's that," said Redberg.

In order to improve post-approval procedures, the FDA has discussed standardizing data on medical device failures, leveraging electronic medical records and assigning a unique ID to each device to make it easy for doctors to report failures or safety threats. "The FDA says they want to do more work on post-approval, but nothing's in place yet," said Redberg. "Even when they do ask for a post-approval study, it's often not completed or done at all."

"I think there has to be more public and congressional calls to have reasonable assurance of safety and effectiveness before devices are approved," said Redberg said. "Until then, all we know is that a lot of devices on the market haven't been shown to be safe or effective."

Explore further: Safety of medical devices not often evaluated by sex, age, or race

More information: JAMA (2017). jamanetwork.com/journals/jama/ … .1001/jama.2017.9414

Related Stories

Safety of medical devices not often evaluated by sex, age, or race

July 25, 2017
Researchers at Yale and the University of California-San Francisco have found that few medical devices are analyzed to consider the influence of their users' sex, age, or race on safety and effectiveness.

Patient safety merits new review for modified medical devices, physician says

March 25, 2014
For patient safety, the U.S. Food and Drug Administration (FDA) should require that clinical data be submitted as part of a more rigorous re-evaluation of medical devices that are modified after approval, according to UC ...

A call to prevent unsafe high-risk medical devices from reaching the marketplace

January 22, 2013
Technological advancements in medicine have allowed patients suffering from musculoskeletal conditions such as hip and knee pain to regain mobility and live relatively pain-free. But some "high risk" surgical devices that ...

Study examines high-risk therapeutic medical devices receiving FDA premarket approval

August 11, 2015
Of high-risk therapeutic devices approved via the Food and Drug Administration (FDA) Premarket Approval (PMA) pathway between 2010 and 2011, there has been wide variation in both the number and quality of premarket and postmarket ...

Flawed data behind regulation of high-risk women's health devices

May 20, 2016
Some high-risk medical devices used in obstetrics and gynecology were approved by the FDA based on flawed data, according to a recent study from Northwestern Medicine.

Most high-risk cardiac devices in use today approved as modifications to previously-approved devices

January 21, 2014
The Food and Drug Administration (FDA) ensures that high-risk medical devices, such as implantable cardioverter-defibrillators and pacemakers, are safe and effective through its premarket approval (PMA) pathway, during which ...

Recommended for you

LincRNAs identified in human fat tissue

June 21, 2018
A large team of researchers from the U.S. and China has succeeded in identifying a number of RNA fragments found in human fat tissue. In their paper published in the journal Science Translational Medicine the group describes ...

Scientists solve the case of the missing subplate, with wide implications for brain science

June 21, 2018
The disappearance of an entire brain region should be cause for concern. Yet, for decades scientists have calmly maintained that one brain area, the subplate, simply vanishes during the course of human development. Recently, ...

Key molecule of aging discovered

June 21, 2018
Every cell and every organism ages sooner or later. But why is this so? Scientists at the German Cancer Research Center in Heidelberg have now discovered for the first time a protein that represents a central switching point ...

Compound made inside human body stops viruses from replicating

June 20, 2018
The newest antiviral drugs could take advantage of a compound made not by humans, but inside them. A team of researchers has identified the mode of action of viperin, a naturally occurring enzyme in humans and other mammals ...

Research reveals zero proof probiotics can ease your anxiety

June 20, 2018
If you're expecting probiotics to reduce your anxiety, it might be time to put down that yogurt spoon—or supplement bottle—and call a professional instead.

Long-term estrogen therapy changes microbial activity in the gut, study finds

June 20, 2018
Long-term therapy with estrogen and bazedoxifene alters the microbial composition and activity in the gut, affecting how estrogen is metabolized, a new study in mice found.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.