Significant underreporting in safety data found on Nursing Home Compare website

The website Nursing Home Compare, sponsored by the Centers for Medicare & Medicaid Services, is a go-to resource for many families researching nursing home options for their loved ones. The number of falls that lead to injury are a critical category of concern for nursing home residents, however, a University of Chicago researcher has found that the data used by Nursing Home Compare to report patient safety related to falls may be highly inaccurate.

Prachi Sanghavi, Ph.D., an assistant professor in public health sciences at UChicago, uncovered significant discrepancies between the falls calculations used for Nursing Home Compare's ratings and actual Medicare claims for falls by nursing home residents from 2011-2015. She found that only 57.5% of falls were accounted for in the Nursing Home Compare's Minimum Data Set (MDS), which is self-reported by nursing homes. Reporting rates were higher for white residents (59%) than non-white residents (46%) and for long-term stays (62.9%) than short-term stays (47.1%). The findings were published December 29, 2019 in the journal Health Services Research.

"This is a substantial amount of underreporting and is deeply concerning because without good measurement, we cannot identify nursing homes that may be less safe and in need of improvement," Sanghavi said.

A significant, yet preventable, risk

Falls are a leading cause of death among the over-65 population, and they can lead to other serious injuries. Patients become fearful of walking again for fear of reinjury, yet falls are considered widely preventable. They are a discrete event that is easy to identify and record, compared to other clinical conditions on Nursing Home Compare such as pressure ulcers or infections, so there should be a wealth of reliable data.

"That's why falls are a patient safety measure on Nursing Home Compare," Sanghavi said. "They reflect how well a nursing home does at preventing these injuries."

Sanghavi started her research with a data set of nearly 88.7 million Medicare admissions claims from 2011 to 2015. She narrowed the sample set in stages, zeroing in on nursing home residents who met several criteria. First, their fall occurred during their time in the nursing home. Second, they were discharged from the nursing home to go to a hospital. Third, the patient's Medicare claim was filed under the code for a major injury fall. Finally, they returned to the same nursing home after treatment for the fall.

"We wanted to be as conservative as possible in our calculations so there would be little argument about whether a fall should have been reported," Sanghavi said. "Our primary outcome measure was based on whether a fall with the appropriate code was reported or not."

Using these criteria, Sanghavi identified 150,828 major injury falls in Medicare claims filed by nursing home residents. The data used by Nursing Home Compare accounted for only 57.5% of these falls, with far fewer falls being reported for black, Asian, and Hispanic patients relative to whites.

Accounting for discrepancies

Nursing Home Compare has faced prior scrutiny for using self-reported data. Sanghavi's own research was sparked by a 2014 New York Times investigation into serious deficiencies found in nursing homes rated five stars by Nursing Home Compare.

"I found it odd that Nursing Home Compare would use self-reported data," she said. "Having worked with Medicare claims data, I thought I could use it to study MDS reporting. The Medicare claims we used are hospital bills. They want to get paid and should not have an interest in nursing home public reporting. That's why they are a more objective source than the self-reported data from nursing homes."

Sanghavi has two theories on why the reporting rate is nearly 15 points higher for long-term stays than short-term stays. "It could be that the nursing homes are more familiar with long-stay patients," she said. "Plus, the falls measure on Nursing Home Compare is specifically for long stays. It could be that nursing homes are taking that measure more seriously."

Sanghavi was especially surprised by the 13-point difference in falls reporting between white and non-white residents. "I didn't expect it to be that different by race," she said, "but it is consistent with other modes of racial disparities in long-term care."

Based on her results, Sanghavi suggests that the Centers for Medicare & Medicaid Services change their evaluation criteria for falls on Nursing Home Compare.

"They should use an objective source, like claims data," she said. "It should be relatively easy for them to do, since they already have the data. There are other claims-based measures already used on Nursing Home Compare."