Spin and bias in published studies of breast cancer trials

January 9, 2013, European Society for Medical Oncology

Spin and bias exist in a high proportion of published studies of the outcomes and adverse side-effects of phase III clinical trials of breast cancer treatments, according to new research published in the cancer journal Annals of Oncology today.

In the first study to investigate how accurately outcomes and side-effects are reported in breast , researchers at the Princess Margaret Cancer Centre and University of Toronto (Toronto, Canada) found that in a third of all trials that failed to show a statistically significant benefit for the treatment under investigation, the reports focused on other, less important outcomes in order to influence positively the interpretation of the results.

In two-thirds of the reports there was bias in the way of the treatment were reported, with more serious side-effects (those with toxicities graded as III or IV) poorly reported. This was particularly the case in trials that showed a significant benefit for the treatment under investigation. Only 32% of articles gave details of the frequency of grade III or IV toxicities in the summary (known as the "abstract").

The authors of the study call for authors, journals and experts who review the articles for journals to be more rigorous in encouraging unbiased reporting of trial results and in enforcing guidelines.

Professor Ian Tannock, medical oncologist and senior scientist in the Division of and at the Princess Margaret, who led the research, said: "Better and more accurate reporting is urgently needed. and reviewers, who give their expertise on the topic, are very important in ensuring this happens. However, readers also need to critically appraise reports in order to detect potential bias. We believe guidelines are necessary to improve the reporting of both efficacy and toxicity."

Prof Tannock and his colleagues identified all randomised controlled, phase III for therapies that had been published between January 1995 and August 2011. Out of a total of 568 articles, 164 were eligible for inclusion in the analysis. trials usually evaluate the efficacy and/or the best dose for a particular therapy that has already been tested in earlier, small trials, and they usually involve more patients than phase I or II trials. Often, they are the final stage that a drug or other therapy has to pass before the treatment can be licensed for use in patients in normal clinical practice, outside of the trial setting.

Trials always have a "primary endpoint" – the specific event that is measured at the end of the trial to see whether or not the given treatment works. The primary endpoint is decided before the study begins. Often it relates to overall survival: did more patients survive or live longer on the new treatment than patients on the existing standard treatment? However, there can also be "secondary endpoints"; these are additional events that are of interest to the investigators, but which the study has not been designed specifically to address, and for this reason investigators have to be cautious in analysing and drawing conclusions from them. Secondary endpoints can include how much longer patients on the new treatment live without the disease progressing, spreading to other parts of the body or recurring, compared to patients on the standard treatment; what are the adverse side-effects and what is the quality of life.

Prof Tannock and his colleagues defined bias as "inappropriate reporting of the primary endpoint and toxicity, with emphasis on reporting of these outcomes in the abstract". They defined spin as "the use of words in the concluding statement of the abstract to suggest that a trial with a negative primary endpoint was positive based on some apparent benefit shown in one or more secondary endpoints".

They found that 54 (33%) trials were reported as positive, based on secondary endpoints, despite not finding a statistically significant benefit in the primary endpoint. "These reports were biased and used spin in attempts to conceal that bias," write the authors. They found that 58% of 92 trials that showed no benefit for patients from the experimental therapy (negative primary endpoint) used secondary endpoints to suggest benefit from the treatment.

A total of 110 (67%) of papers reported of the experimental therapy in a biased manner. If a trial showed a benefit for the treatment (positive primary endpoint), then toxicities were more likely to be under-reported.

The first author of the study, Dr Francisco Vera-Badillo, clinical research fellow at the Princess Margaret, said: "We found a high incidence of biased reporting of the outcomes of clinical trials. In those with outcomes that were either negative or did not show a statistically significant benefit, spin was used frequently to influence positively the interpretation of the results, by focusing on apparent benefits from secondary endpoints.

"Where trials showed a positive outcome, the toxicities were less likely to be reported. A possible explanation for this could be that the investigators, sponsors or both, prefer to focus on the efficacy of the experimental treatment and downplay toxicity to make the results look more attractive."

The source of funding for trials (industry or academic) was not associated with bias or spin in the reporting of results and toxicities.

In order to be published in most academic journals, it is now compulsory to register clinical trials before they start. Many countries register them on either the USA registry (ClinicalTrials.gov) or the European registry (clinicaltrialsregister.eu). Some of the studies analysed for the Annals of Oncology paper started before registration became compulsory. However, for those that were registered, the researchers found that some changed the primary endpoint between registration and the report of the outcomes being published. "Among these , there was a trend towards change of the primary endpoint being associated with positive results, suggesting that it may be a strategy to make a negative trial appear positive," write the authors. "Trial registration does not necessarily remove bias in reporting outcomes, although it does make it easier to detect."

Explore further: 'Spin' in media reports of scientific articles

More information: "Bias in reporting of endpoints of efficacy and toxicity in randomized clinical trials for women with breast cancer", by F.E. Vera-Badillo, R. Shapiro, A. Ocana, E. Amir, I.F. Tannock. Annals of Oncology. doi:10.1093/annonc/mds636

Related Stories

'Spin' in media reports of scientific articles

September 11, 2012
Press releases and news stories reporting the results of randomized controlled trials often contain "spin"—specific reporting strategies (intentional or unintentional) emphasizing the beneficial effect of the experimental ...

Industry sponsorship leads to bias in reported findings of clinical trials

December 11, 2012
Studies reporting the results of industry sponsored clinical trials present a more favourable picture of the effects of drugs and medical devices than those reporting on non-industry sponsored trials, according to a new Cochrane ...

Recommended for you

T-cells engineered to outsmart tumors induce clinical responses in relapsed Hodgkin lymphoma

January 16, 2018
WASHINGTON-(Jan. 16, 2018)-Tumors have come up with ingenious strategies that enable them to evade detection and destruction by the immune system. So, a research team that includes Children's National Health System clinician-researchers ...

Researchers identify new treatment target for melanoma

January 16, 2018
Researchers in the Perelman School of Medicine at the University of Pennsylvania have identified a new therapeutic target for the treatment of melanoma. For decades, research has associated female sex and a history of previous ...

More evidence of link between severe gum disease and cancer risk

January 16, 2018
Data collected during a long-term health study provides additional evidence for a link between increased risk of cancer in individuals with advanced gum disease, according to a new collaborative study led by epidemiologists ...

Researchers develop a remote-controlled cancer immunotherapy system

January 15, 2018
A team of researchers has developed an ultrasound-based system that can non-invasively and remotely control genetic processes in live immune T cells so that they recognize and kill cancer cells.

Dietary fat, changes in fat metabolism may promote prostate cancer metastasis

January 15, 2018
Prostate tumors tend to be what scientists call "indolent" - so slow-growing and self-contained that many affected men die with prostate cancer, not of it. But for the percentage of men whose prostate tumors metastasize, ...

Pancreatic tumors may require a one-two-three punch

January 15, 2018
One of the many difficult things about pancreatic cancer is that tumors are resistant to most treatments because of their unique density and cell composition. However, in a new Wilmot Cancer Institute study, scientists discovered ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.