New analysis finds Medicare program underestimates heart attack mortality rates

Credit: Anne Lowe/public domain

New analysis of Medicare's Hospital Compare portal shows the statistical methodology used to rate and compare hospitals underestimates mortality rates of acute myocardial infarction (AMI) at small hospitals. The research, titled "Mortality Rate Estimation and Standardization for Public Reporting: Medicare's Hospital Compare," appears in the Journal of the American Statistical Association.

Hospital Compare collects data from Medicare claims and hospitals. That data is then processed through a statistical model known as a "random effects logit model," which unfortunately shrinks from small hospitals to resemble the national average.

"Hospital Compare's finding of average risk at small hospitals is a mistake because the current model is not properly calibrated," notes Edward I. George, professor of statistics at the University of Pennsylvania's Wharton School and one of the study's authors. "It's a mistake that has implications for patients."

Low volume hospitals, by definition, have little data regarding AMI mortality. In using the current model, a poorly performing small volume hospital will look average, and even a large hospital that performs far better than average will yield results that underestimate their quality. "The underestimation of AMI mortality rates at small hospitals, as seen in Hospital Compare, contradicts previously established research and consistent findings that mortality rates are typically higher at low volume hospitals," adds Jeffrey H. Silber, professor of pediatrics at the Children's Hospital of Philadelphia, professor of health care management at The Wharton School, and one of the study's authors.

George, Silber, and fellow researchers Paul R. Rosenbaum, also a professor of statistics at The Wharton School; Veronika Ročková, assistant professor of econometrics and statistics at The University of Chicago's Booth School of Business; and Ville Satopää, assistant professor of technology and operations management at INSEAD, propose an alternative model that yields more accurate and appropriately calibrated predictions.

This model would incorporate additional factors such as hospital volume, nursing staff, medical residents, and the hospital's ability to perform cardiovascular procedures into the calculation process. Combined with a different statistical method known as "direct standardization," the researchers' approach is seen to be more effective for adjusting rate comparisons between hospitals.

"Patients deserve to have the most accurate information available so they can make well-informed health care decisions. Our analysis demonstrates that further modeling with additional hospital attributes has the clear potential to shed more light on the rankings of such hospitals," the authors conclude.

Evaluating and assessing hospital performance, while challenging, is nothing new to the statistical community. In 2011, the Committee of Presidents of Statistical Societies (COPSS) published "Statistical Issues in Assessing Hospital Performance," which examined statistical issues identified by the Centers for Medicare and Medicaid Services (CMS) and other stakeholders about the government agency's approach to modeling hospital quality based on outcomes. Then chair of COPSS, Thomas Louis, a professor in the department of biostatistics at Johns Hopkins University's Bloomberg School of Public Health, knows this issue well.

"Adopting the modeling enhancements proposed by the authors in this new Hospital Compare research will improve predicting outcomes and choosing a ," said Louis. "I urge the CMS and all researchers working on profiling methods to seriously consider issues the authors have identified and their state-of-the-art approaches to many of them."

Explore further

Public reporting of hospital mortality rates for common conditions did not improve patient outcomes

More information: E.I. George et al. Mortality Rate Estimation and Standardization for Public Reporting: Medicare's Hospital Compare, Journal of the American Statistical Association (2017). DOI: 10.1080/01621459.2016.1276021
Provided by American Statistical Association
Citation: New analysis finds Medicare program underestimates heart attack mortality rates (2017, April 19) retrieved 7 May 2021 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments