Star razing: Providers face quality rating dips after system recalibration
Note: For more recent coverage of the Nursing Home Compare star rating system, see the November 2014 and January 2015 issues of PPS Alert for Long-Term Care.
On February 12, CMS announced the dawn of Nursing Home Compare 3.0, a version update marked by a major recalibration of the popular consumer search site’s five-star quality rating system. When the agency debuted the retooled feature just one week later, almost a third of nursing homes had shed at least one star from their overall rating?a change that some provider advocates say baffled industry business partners and consumers alike.
"What we’ve heard is confusion," says David Gifford, MD, senior vice president of quality and regulatory affairs at the American Health Care Association (AHCA), a national trade association for long-term care providers. "The public’s asking, ‘Why did you lose a star?’ "
Recalibration rundown
The five-star rating system scores a given nursing home in three domains, the general characteristics of which were preserved during the recalibration:
1.Quality measures (QM) pulled from MDS documentation completed by the facility.
2.Staffing levels identified in a form completed by the facility during its annual state inspection. The rating in this domain is calculated based on scores in two subcategories: 1) the number of hours RNs have worked per resident day in the two-week period leading up to the survey, and 2) the total number of hours worked by RNs, licensed practical nurses, CNAs, and physical therapists within the same time frame.
3.Health inspection reports that detail the facility’s performance on the latest three annual surveys, as well as the outcomes of any complaint investigations conducted within that time frame.
The methodology CMS uses to translate these individual component scores into a facility’s overall rating was also spared in the February calculation updates: The health inspection component?the only rating domain that doesn’t hinge on data submitted directly by a facility?still serves as the bedrock of the score. From there, one star can be added or subtracted based on the staffing component. The same goes for the quality measure score.
The recalibration instead targeted the two so-called self-reported categories, weeding the field of top performers in these arenas by:
1.Increasing the difficulty of achieving a staffing rating of four stars, which now requires nursing homes to earn four stars in either the RN or total staffing subcategory.
2.Ratcheting up the number of points necessary to earn a QM rating of two or more stars?a move that significantly redistributed providers along the rating scale. CMS reports that since the recalibration, the number of nursing homes with four- or five-star QM ratings has plunged from about 80% to 49%, while the number of facilities with one star in this arena has climbed from 8.5% to 13%. In total, Gifford says a staggering two-thirds of providers experienced a change in their QM score.
3.Incorporating two antipsychotic measures that have long been reported on Nursing Home Compare (one targeting rates of use among short-stay residents, the other aimed at their long-stay counterparts) into QM scoring methodology.
In addition to kindling major rating changes in the quality and staffing domains, the February shake-up also tided a shift of up to two stars in facilities’ overall ratings.
According to AHCA, overall score changes that can be wholly attributed to the quality and staffing calculation updates are as follows:
- About 29% of providers lost one star
- 2% lost two stars
- Less than 1% gained a star
Note: These statistics don’t account for the additional star shifts experienced in February by some providers whose health inspection data was updated at the time of the recalibration.
Curbing the ratings creep
The recent recalibration builds on the controversy that has embroiled the star system since the New York Times published its August 2014 exposé claiming the ratings’ reliance on self-reported measures enabled facilities to game the system. This position, the author and some consumer advocacy groups say, is backed by the steep climb of top-ranked facilities over the years (a climb that the new calibration has since stemmed).
But providers and their advocates bristled at these charges, instead attributing the influx of four- and five-star ratings seen prior to the recalibration to the industry’s fierce commitment to improving care delivery.
Although stakeholder factions are still contesting the primary drivers of the historic ratings creep, it’s undeniable that, with calculation methodology left largely unchecked in the six-year expanse between the advent of the star system and its February recalibration, ratings had trended upward.
When the five-star system was first inaugurated in December 2008, the number of top and bottom ratings awarded in a given category hovered within a few percentage points of their counterparts in the other two domains. However, between 2009 and 2013, the volume of nursing homes with an overall rating of five stars jumped more than 10%, while the number of facilities with one star fell more than 11%.
Because the health inspection rating is calculated using a state-specific fixed distribution (with five stars awarded to the top 10% of facilities, one star granted to the bottom 20% of performers, and the remaining star counts doled out in equal shares among the middle 70%), the skyward trajectory of the overall ratings can be traced exclusively to hikes in staffing and quality scores over the years.
Between 2009 and 2013, the proportion of four- and five-star ratings in the staffing domain increased about 13%, while these ratings soared almost 33% in the quality arena, according to data released by CMS contractor Abt Associates.
"We’ve seen the number of high five-star rated buildings increase, increase, increase," says Thomas Martin, senior research analyst at PointRight Inc., a healthcare analytics consulting firm in Cambridge, Massachusetts. "We were moving toward almost half of the industry being a five star in the quality domain."
These disproportionate leaps in QM scores shook many stakeholders’ trust in the system, according to Richard J. Mollot, executive director at the Long Term Care Community Coalition (LTCCC), a New York City?based advocacy group for elders and people with disabilities.
"It got to the point where it was hard to pay any attention at all to the quality measures," Mollot explains, recalling the site profiles of facilities with one-star ratings in the inspection and staffing categories and a five-star QM score. "It was just kind of bizarre."
Although most stakeholders seem to acknowledge that the spiraling ratings needed to be reined in, provider advocates worry that the steep drop-offs triggered by CMS’ drastic response will sully the reputations of well-performing facilities and ignite unintended operational consequences.
Nursing Home Compare was launched in 1998?and augmented by the five-star system a decade later?to facilitate consumers’ searches for facilities that best fit their specific care needs. But today, the reach of the site extends well beyond these original bounds. Far-flung entities (e.g., accountable care organizations, managed care plans, liability insurance carriers, and the U.S. Department of Housing and Urban Development) have appropriated the star ratings to determine facilities’ business worth, and consequently, Gifford says some providers have fielded calls from these stakeholders inquiring about sudden rating drops.
"It looks like . . . industrywide, nursing home practices have suddenly dipped and quality has decreased, but that’s not really the case," adds Martin.
Stakeholders support antipsychotics crackdown
Although Gifford decries the aggressiveness and abruptness of CMS’ score rebasing, he says AHCA backs the agency’s inclusion of antipsychotic measures in QM calculations?a decision that speaks to the growing precedence of this focus in the healthcare realm.
As Americans are living longer with more complex conditions, and nursing homes have consequently experienced an influx of residents with dementia, regulatory efforts to quell the use of antipsychotics are gaining momentum.
"Antipsychotic drugging . . . is, I think, one of the most important issues that we’re looking at today," Mollot says. "It’s emblematic of care in general for a lot of facilities in terms of . . . how staff are trained to meet the needs of their residents."
Although providers have already made significant strides in this domain, CMS and patient-centered groups like LTCCC say there’s a lot of work left to do.
For example, though the industry managed to whittle usage of antipsychotics by 15% between calendar years (CY) 2011 and 2013, CMS hopes to double this statistic by the end of CY 2016. And despite current progress, LTCCC reports that roughly 20% of residents nationwide were still on antipsychotic drugs in December 2014.
Because many providers focus on boosting ratings to attract prospective clients and business partners, Mollot thinks CMS’ decision to integrate antipsychotic use into rating calculations will fuel more necessary improvements in this arena.
Despite garnering support from provider and consumer advocates, the new QM measures left some participants in a February open door forum wary about the then-impending rating system updates. A couple callers voiced concerns that the incorporation of the short stay antipsychotic usage measure would penalize nursing homes for drugs prescribed to residents during prior stays at the hospital. One participant worried the new spotlight on antipsychotics would misrepresent her facility’s usage of the drugs among its unique population of residents with severe mental health issues.
Star striking justified, but too abrupt
Despite generally favorable receptions of the antipsychotic measure additions to QM ratings, some provider advocates had championed a much more graduated approach to getting the star system up to speed. For example, Gifford recounts AHCA’s suggestion to bump up the cutoff score needed to qualify for a given star count (called a cut point by CMS) on a quarterly basis, adding that he hopes CMS will still adopt such a method for the subsequent rounds of rating updates the agency already has in the pipeline.
"Each time they add new measures, if they’re going to rebase [ratings], that’s just going to add more confusion," Gifford explains. "I think there’s a better way that hopefully they’ll consider next time going forward."
He adds that employing an incremental method to adjust star cut points and calculations would also benefit the intended users of Nursing Home Compare, who would be able to make more meaningful judgments about a facility’s succession of star ratings?an ability Gifford says would be hampered by future iterations of this year’s recalibration, which rendered providers’ ratings between January and February incongruous.
"Rebasing this every couple years [would mean] that all prior ratings have no meaning compared to the new ratings," Gifford explains.
But while providers and their backers believe February’s jarring rating updates could malign nursing homes’ performance, consumer advocates say the rampant score inflation prior to the recalibration also misled consumers. "[The rating system] really wasn’t fair before for consumers," says Mollot. "You’re seeing a much more accurate rating now."
For this reason, Mollot finds many of the attempts he’s seen throughout the industry to characterize lowered ratings as mere byproducts of CMS’ more stringent calculation methodology too reductive. Instead, he urges providers to consider the full context of the agency’s decision to carry out the rating overhauls and reflect on whether star losses reveal any gaps in current care practices.
"If providers are thinking about [the recalibration] in a constructive way, that will be really useful to them in terms of the quality of care overall in the facility and . . . in terms of customer satisfaction, too," Mollot says. "The more accurate the information is, the more useful it will be for the nursing home[s] . . . to improve their care, and that’s really what we all want."
Despite the growth opportunities he thinks the recalibration presents, Mollot acknowledges the importance of explaining to alarmed clients that it is also the direct impetus of some sudden rating reductions.
But Martin thinks this feat may be easier said than done, given the minutiae of the rating calculation methodology. He recommends underscoring that the changes reflect higher expectations from CMS, rather than newfound performance lapses on the facility’s end.
"The five-star methodology has raised the bar," he says. "[CMS’] grading criteria has gotten more difficult, even though the facility didn’t necessarily take a dip in quality."
In addition, Gifford suggests trying out the language CMS posted on Nursing Home Compare post-recalibration to highlight the lack of comparability between QM scores introduced in January and their immediate predecessors.
"Many nursing homes will see a lower quality measure rating as a result of these changes, even though the underlying QM data may not have changed," CMS recently wrote on the "How we calculate ratings" page of the site. "Because of these changes, it is not appropriate to compare a facility’s QM ratings that appear in February with those that appeared in earlier months."
Mollot says anticipating tough questions from clients is a best practice that should extend beyond the scope of recalibration fallouts. For example, if a surveyor identifies a deficiency during an annual inspection, a provider should brace for queries about what its staff are doing to address the issue?and have a corrective plan in place to ensure the prepared answer is more than lip service.
Catch falling stars
In addition to strategically educating concerned clients on rating changes carried by the recalibration, Mollot and Gifford urge providers to use the revised scoring methodology as a tool to identify their facility’s specific hot spots and guide targeted quality improvement efforts.
Gifford recommends consulting the latest version of the rating system user’s guide, which CMS updated in February to reflect the new quality and staffing star cut points ushered in by the recalibration.
"One of the nice things in the changes is CMS actually provided the rates on each of the quality measures that you need to achieve a better score," says Gifford. "We’ve been encouraging members to look at those rates related to the points and figure out how much better they have to get on which measures to improve their score."
Martin echoes this recommendation, stressing the importance of determining a given facility’s proximity to the next star count in each domain.
"If you just understood where those cut points were and how close you were, you could get a feel for what you need to do to improve," he explains. "You might be one hour of nursing time away from the next star."
Because of the heightened emphasis on reducing antipsychotic drug usage, Mollot also recommends strengthening dementia care practices?a move he says doesn’t necessarily demand major resource expenditures or new hires. Sometimes, he says, a simple change in approach?such as delaying a distressed resident’s scheduled bath?is all it takes to overcome the use of chemical restraints and recoup time traditionally spent subduing difficult behaviors. This refocusing can allow staff to dedicate additional efforts to other facility needs and, in turn, boost star ratings.
"Providers who are moving to provide better dementia care will see benefits, and hopefully that will incentivize others as well," Mollot says.
Once nursing homes pinpoint their performance gaps and growth opportunities, Gifford suggests turning to national initiatives (e.g., the one developed by AHCA) or established quality improvement programs (e.g., the Baldrige Performance Excellence Program) for frameworks to transform performance goals into operational realities.
More changes on the horizon
Although providers may still be seeing stars after the February recalibration, CMS is far from done with its overhauls to the rating system.
In particular, experts are keenly anticipating the agency’s deployment of an electronic reporting system that will both cull staffing numbers from a nursing home’s payroll records and report this data on a quarterly basis, ultimately transforming it into star ratings, Mollot says. CMS recently announced that it intends to collect data through the system on a voluntary basis beginning October 1, 2015, and to mandate the industry’s nationwide reporting in this manner beginning July 1, 2016.
Gifford says CMS is also readying a handful of new staffing and quality measures for integration into star calculations in the not-so-distant future, including rates of turnover, retention, rehospitalization, and discharge to the community?prospects about which he is cautiously optimistic.
For his part, Mollot would like to see CMS regularly update criteria and refine calculation methodologies across all rating categories to undercut what he calls the tendency of some industry players to "teach to the test" by applying quick-fix strategies to bump up ratings rather than launching genuine quality improvement initiatives.
As CMS continues to flesh out the criteria within each rating domain, Gifford hopes the agency will find a way to continue representing the entire spectrum of insurers weighed in today’s ratings. Although current scores reflect all residents, regardless of their payer type, the proposed rehospitalization and discharge to community measures could be based on claims data that isn’t collected for all coverage plans.
"If this information is going to be used for [consumer] choice, it should reflect the full panel of individuals who are cared for by a provider," Gifford explains.
But despite stakeholders’ misgivings, Mollot stresses that the revamped rating system is an important tool for the nursing home community. "Nursing Home Compare, even with its imperfections, is by far the most valuable resource for people in terms of choosing a nursing home," he explains. "Capturing quality in an accurate way . . . is an important thing, and I think that benefits the good providers, as well as consumers."