CMS postpones scheduled July hospital star-ratings release after hospitals say a sneak peek at their results showed a disconnect between performance measures and stars awarded by the agency.
June 15—The stars and the data didn’t align for the Centers for Medicare & Medicaid Services (CMS), which said this week that it was postponing its scheduled July update to the agency’s star ratings for some 4,000 hospitals nationwide.
In a June 12 notice posted on QualityNet, the CMS-created portal through which hospitals and other providers report their performance data to the agency, CMS said it “decided to postpone the July star ratings update to give time for additional analysis of the impact of changes to some of the measures on the star ratings and to address stakeholder concerns.”
Hospitals groups, which have been critical of the methodology CMS uses to assign stars to hospitals, said they welcomed the delay.
“CMS made the right call in postponing the July update of star ratings, and we appreciate the agency allowing more time for a fuller analysis of its methodology and measures and to hear from stakeholders, including hospitals and health systems, about concerns found in many preview reports,” Tom Nickels, executive vice president of the American Hospital Association (AHA), said in a prepared statement.
America’s Essential Hospitals (AEH) called CMS’s decision “prudent.”
“Reviews of July preview reports show large shifts in overall hospital star ratings from December 2017 to July 2018,” AEH said in its prepared statement. “These changes have created confusion and raised new questions about the reliability and validity of the methodology used to calculate these ratings.”
December Ratings Remain
CMS launched its overall star-rating program for hospitals in July 2016. The agency assigns up to five stars to a hospital based on how well it scores on 57 different performance measures. The measures fall into seven categories: mortality; safety; readmissions; patient experience; effectiveness of care; timeliness of care; and efficient use of medical imaging.
CMS publicly reports the number of stars each participating hospital gets on the government’s Hospital Compare website. CMS updates its hospital star ratings twice a year—in July and October. The most recent update—from December 2017, which itself was delayed from October 2017— assigned stars to 3,692 hospitals with:
- 337, or 9.1 percent, given five stars
- 1,155, or 31.3 percent, given four stars
- 1,187, or 32.2 percent, given three stars
- 753, or 20.4 percent, given two stars
- 260, or 7 percent given one star
Another 887 hospitals did not report all of the required data and were not eligible for a star rating from CMS last December.
With the most recent delay, the December 2017 ratings will remain on the Hospital Compare website for each hospital until the next update, CMS said in its notice.
Before CMS updates its star ratings on Hospital Compare, the agency makes “preview reports” available to hospitals through the QualityNet portal. Hospitals can download their preview reports, review them, identify any problems, and report those problems to CMS to correct before the final reports go public.
For the scheduled update this July, CMS made its preview reports available to hospitals on May 4. Hospitals had until June 2 to review their preview reports and contact CMS with any concerns.
Hospitals’ collective concerns apparently were so significant that CMS put the July update on hold.
Data-Stars Discrepancy
Nancy Foster, vice president of quality and patient safety at the AHA, said many hospitals saw little or no change in the their scores on the 57 performance measures over the past six months that CMS uses to determine how many stars a hospital receives. She described that as not unexpected over a six-month period. What was unexpected, she said, was the substantial change in star ratings for the same hospitals whose performance data didn’t move significantly.
“We were seeing the loss of two or more stars for some hospitals whose data didn’t change,” Foster said. “One couldn’t explain the other.”
The disconnect was so wide that the cause may be unrelated to the data itself but rather could be a math error, miscalculated data or a data-entry issue, according to Foster.
Foster said AHA and others also noticed that some performance measures like the complication rate for hip and knee replacement surgeries were weighted more heavily in CMS’s star-rating formula this time while others like the PSI 90 measure were weighted less. The PSI 90 measure is a composite of 10 patient safety indicators (PSIs) that includes a hospital’s postoperative sepsis rate and rate of patient falls that cause hip fractures.
The weighing of specific measures was so different that the data may have been accidently inverted in the formula, Foster said.
Methodological Impact
Changes in behind-the-scenes methodologies can have a significant impact—both good and bad—on the resulting overall provider performance scores released to the public.
Earlier this month, for example, the Agency for Healthcare Research and Quality (AHRQ) released its latest dataon hospital-acquired conditions (HACs). HACs are problems that experts say shouldn’t happen to patients after they’re admitted to a hospital. They include medication errors, falls, and a variety of infections.
In the 22-page report, the agency said it set a new baseline for the rate of HACs per 1,000 hospital discharges.
The new 2014 baseline includes all patients admitted to acute-care hospitals for most types of injuries and illnesses. The previous baseline included only patients who were admitted for four conditions: heart attacks, heart failure, pneumonia and surgery. By including more patients in the calculations, the HAC rate per 1,000 discharges dropped to 98 from 121—a 19 percent difference. Using the new baseline, the preliminary HAC rate in 2016 was 90 per 1,000 patient discharges.
After hospitals saw their preview reports for their July star ratings, the AHA staff “urged” CMS to take a step back and look at its numbers again, Foster said.
And CMS acted on those concerns.
“When changes are made to the underlying measures it is vital to take the time needed to understand the impact of those changes and ensure we are giving consumers the most useful information,” CMS said in its June 12 notice. “As part of this process, CMS will seek feedback from a multi-disciplinary Technical Expert Panel, a Provider Leadership Workgroup, and a public comment period.”
CMS did not indicate when it expected its hospital star ratings to be updated next. Foster said she didn’t have a firm timetable but said it could take three to six months or longer.
David Burda is a veteran healthcare business reporter. Follow him on Twitter: @DavidRBurda
