Sarah Jagger, JD, MPH
Former Vice President of Operations
On August 19, the Centers for Medicare and Medicaid Services (CMS) announced its plans to change the controversial methodology used for its hospital star ratings on Hospital Compare. However, changes won’t occur until 2021. At that time, CMS will “refresh” the star ratings on Hospital Compare using the current methodology in order to ensure that patients continue to have timely access to the most up-to-date hospital quality information. The press release about this change highlights the initiative as a major step forward in delivering on President Trump’s recent Executive Order on Improving Price and Quality Transparency in American Healthcare to Put Patients First, which seeks “to enhance the ability of patients to choose the healthcare that is best for them.”
To understand the issue, let’s take a step back: The debate here isn’t about whether we should give consumers ratings and information about the quality and experience of care at hospitals. The issue is what data is used and how it is used to determine the ratings in a way that is fair and equitable to the hospitals but also meaningful to consumers.
Imagine for example you are car shopping. A salesman shows you a car and raves that it is the number-one-rated SUV for three years running. You buy the car and get it home, only to realize that the ratings were primarily based on the opinions of young adults who don’t have children. It turns out the car really doesn’t work for the purposes of your family of four. In this example, certain measures were weighted more than others and skewed the results in a way that is misleading for car buyers with children
Hospitals are rightfully making the case to give consumers the most accurate and meaningful data possible so that those consumers can trust the information they are being provided and make informed choices about their health care.
The agency’s decision to update Hospital Compare under the current methodology is a disappointment to many hospitals and hospital organizations, including the two largest lobbying organizations for hospitals: the American Hospital Association (AHA) and America’s Essential Hospitals (AEH). These organizations believe that Hospital Compare should not be updated until such a time that there is a new methodology that has been vetted and is ready for implementation. AHA leadership argues that the current methodology has “substantial flaws” and “republishing the flawed ratings in 2020 will not advance the goal of providing the public with accurate, purposeful information about quality.” Furthermore, the AEH calls it “misleading to patients” to make ratings public while the methodology is still under review.
Hospital Compare Background
First available in 2005, Hospital Compare provides access to quality measures on more than 4,000 Medicare-certified hospitals as well as Veterans Health Administration and Military Health System hospitals. Users can compare hospitals based on patient experience, timeliness and effectiveness of care, and complication rates, among other factors. Information is also organized by medical condition such as heart attack, pneumonia, or type of surgery. Facilities are scored from 1-5 on a variety of metrics from patient satisfaction to mortality rates by a variety of ranking groups including U.S. News, Leapfrog, and CMS.
Providers have long argued that the ranking system oversimplifies quality and is overly complicated for consumers to interpret. Additionally, the program has been accused of penalizing safety net hospitals by not accounting for differences in patient population, as well as penalizing teaching hospitals by disregarding the complex conditions treated there. A March 2018 consultancy analysis published in Modern Healthcare confirmed that the previous star ratings system was biased in favor of specialty hospitals over major teaching hospitals.
February 2019 Update and Request for Comments
Hospital leaders blamed the model for drastic changes they saw in their star ratings from December 2017 to June 2018 that led to the 15-month hiatus of the star ratings update. Ratings were intended to be updated quarterly. CMS last updated the star ratings late February 2019, the first update since 2017.
Along with the update, CMS asked for public input on how the ratings could be improved. Specifically, the request sought feedback on nine potential changes to the star ratings. The agency received more than 800 comments within 145 comment letters from stakeholders including hospitals, health systems, hospital associations, and medical universities. Below we summarize the comments CMS received on some of the key proposed changes.
LATENT VARIABLE MODEL
Most commenters expressed issues with the latent variable model (LVM) and supported replacing it. The LVM is a statistical approach, designed by CMS in partnership with Yale New Haven Health, that gives greater emphasis to certain measures over others in the star ratings. As a result of this approach, some measures have a far greater influence than others in a hospital’s overall rating. Commenters argued that the LVM makes it difficult for hospitals to predict the impact of quality improvement activities and called for CMS to make the methodology more “transparent and predictable” or more “explicit.”
PEER GROUPING
CMS proposed comparing hospitals by type to determine star ratings. This proposal received mixed responses. Many welcomed the concept of peer grouping hospitals, but had varied suggestions on how they should be grouped. Suggested groupings included patient socioeconomic status or teaching, specialty, and community. Several organizations, however, took issue with grouping hospitals of different sizes or with populations facing different levels of social determinants challenges. These organizations suggested that it was flawed to have a small community hospital with minimal services available ranked as the highest in the region with the academic medical centers and safety nets looking poor in comparison.
MEASURE GROUPING
Most respondents supported CMS’ proposed three-step approach to grouping measures based on clinical criteria as well as several statistical guides, as most commenters felt balanced and consistent measure loadings were important. However, there was not a consensus on a specific regrouping option that was presented.
CONSUMER CUSTOMIZATION
CMS proposed creating a tool that would allow consumers to set their own weights for each of the measure groups used in the star ratings. Most respondents argued against this option on the grounds that it would be too complicated for consumers who were not experts in healthcare.
Next Steps
CMS provided no details on what the changes to the star rating could encompass for 2021. The release states that CMS plans to use the feedback from the public input request to guide its proposed changes to the star ratings. Along with these comments, CMS intends to form a technical expert panel (TEP) made up of approximately 15-20 individuals with differing perspectives and areas of expertise, such as:
- Hospital quality topic knowledge;
- Statistical modeling and expertise;
- Other topic knowledge (measure development, consumer testing, star ratings systems);
- Consumer/patient/family (caregiver) perspective;
- Healthcare disparities;
- Performance measurement;
- Quality improvement; and
- Purchaser perspective.
Proceedings of the TEP will be summarized in a report that is disclosed to the general public.
CMS is also planning more public outreach to shape potential changes to the Overall Star Ratings methodology. This includes a public listening session on the star ratings for a range of stakeholders, that will include a call-in option, on September 19, 2019, in Baltimore.
Atrómitos can help your organization navigate regulatory changes such as this one through its informed, expert analysis and strategic planning services. Contact us to talk more about this rule change and next steps.