Solutions

Solutions powered by
Learn More.
abaqis for Skilled Nursing

abaqis®

Mitigate risk and elevate your quality of care. Improving both clinical and business outcomes starts with a smarter, more integrated approach to regulatory training, continuing education and quality management.

Learn More

NEW TO THE STORE

hStream Content Marketplace

Connecting you to the best content in our industry-leading marketplace. Streamline content purchasing, updates, hosting and connectivity for your organizational needs.

NurseGrid Learn

Online store for Clinical Learners. Individual CE Credit and industry products for Clinical Learners are now available through NurseGrid Learn (Beta). Explore our robust collection of courses that provides an unparalleled level of flexibility and choice.

LATEST & POPULAR

BLOG

Nurse Strikes are Scary, but License Verification Doesn’t Have to be
BLOG

Ten Trends for 2023: Rising Temperatures and Generational Changes
BLOG

Ten Trends for 2023: CMS Cuts and Other Disruptors
BLOG

Ten Trends for 2023: Hospital at Home and Integration of Behavioral Health and Primary Care
BLOG

Ten Trends for 2023: Emerging Technology and Cybersecurity
BLOG

Ten Trends for 2023: Financial Woes and Staffing

< HealthStream Resources

Correlation Between CMS Star Rating and HealthStream Jane®

May 24, 2022
May 24, 2022

The Centers for Medicare & Medicaid Services (CMS) tracks and reports on several quality measures to provide transparency intesults on different measures which are distributed over five categories – mortality, safety of care, readmissions, patient experience, and timely & effective care (Centers for Medicare & Medicaid Services [CMS], 2021, p. 13). These categories hold different weights which roll up to an overall star rating between 1 – 5 stars (CMS, 2021, p. 17). This rating affects hospital reputation as well as reimbursement, so a higher star rating is desired. For several of these measures, a well-trained nursing staff is imperative. As an example, one of the measures tracks incidences of central line-associated bloodstream infections (CLABSI). In order to do well, staff need to be educated on infection prevention protocols and be motivated to follow them. HealthStream Jane® is a professional development tool used to uncover knowledge and clinical judgment gaps and provides recommended remediation for identified deficiencies. This study seeks to determine if acute care hospitals that utilize HealthStream Jane® had higher overall star ratings compared to those that did not.

Background and Significance

CMS Star Rating
As part of an effort to improve the quality of care and clinical outcomes, while also lowering costs, CMS implemented the Inpatient Quality Reporting (IQR) program, with over 3000 acute care hospitals participating annually. To motivate quality improvement, CMS tied reimbursement to these results (Shetty et al., 2021, p. 544). Several studies have shown strong, positive correlations between measure improvement and different interventions. For example, Shetty et al. (2021) point out that the automation that comes from using an electronic health record (EHR) system led to a decrease in readmissions, improved patient satisfaction, and a decrease in mortality (p. 544). With several measures linked to nursing education, clinical judgment, and early recognition of patient deterioration, a system to assess and develop these should lead to better patient outcomes and a higher star rating.

HealthStream Jane®
In their article on nursing professional development (NPD), Canfield (2021) shares that the focus of NPD practitioners should be “on developing education to address identified gaps in knowledge, skill and/or practice, maintain content integrity, and analyzing measurable outcomes to demonstrate closure or narrowing of the gaps that created the need for the education.” (p. 11). Implementingthis on a large scale is time-consuming, complex, and challenging. Tools like HealthStream Jane® help automate this process by quickly identifying gaps, providing development recommendations that can be auto assigned, and digital tracking capabilities to demonstrate the narrowing of the education gap. Although nurses can earn continuing education units (CEUs) for the completion of the recommended content, Graebe and Dickerson (2021) note that, “contact hours are the currency—quality nursing practice is the outcome” (p. 56).

Purpose Statement

The purpose of this study was to determine if a correlation exists between overall CMS star rating and the overall average score on HealthStream Jane® knowledge assessments.

Methodology

Research Design
A quantitative, correlational study design was chosen using historical assessment completion data along with publicly available CMS quality measure data. The null hypothesis is that there is no correlation between hospitals’ average HealthStream Jane® knowledge assessment score and CMS overall quality star rating.

Sampling
All acute care hospitals that had at least 20 HealthStream Jane® assessment completions during the timeframe of November 1, 2019, through October 31, 2020, and also had a valid CMS star rating between 1-5 were included in the study. This provided a total of 22 acute care hospitals that met the inclusion criteria. For instances where a user completed the same assessment more than once during the study timeframe, only the most recent assessment score was used in the calculation of the overall average score.

Data Measurement
Hospitals with access to HealthStream Jane® had the ability to assign knowledge assessments to staff based on clinical role. For example, a registered nurse (RN) that worked in the emergency department (ED) would complete three knowledge assessments – RN: Emergency Assessment, RN: Pharmacology Assessment, and a Cardiac Rhythm Identification Assessment. These knowledge assessments were graded automatically upon submission with a score between 0-100, and the results were stored to a database. CMS overall star ratings were acquired directly from CMS.

Data Collection
The knowledge assessment scores for the hospitals that met the inclusion criteria were queried from the HealthStream database. The CMS quality measure results were downloaded directly from the CMS website. To map the CMS overall quality star rating and the Jane® knowledge assessments score data by hospital, matching was made based on hospital name and address. Matching was carried out using the “token sort ratio” and “token set ratio” methods from the python library and verified with manual review.

Data Analysis
A linear regression analysis was conducted using the CMS overall star rating as the dependent variable and the average hospital knowledge assessment score as the independent variable. See Figure 1 for the joint plot between CMS overall quality star rating and hospitals’ average Jane® knowledge assessment score. When the average Jane® knowledge assessment score increases, so does the CMS star rating. The coefficient for the average Jane® knowledge assessment score was equal to 0.1202 (p=0.04). This indicates strong evidence against the null hypothesis, which as stated above, is that there is no correlation between hospitals’ average Jane® knowledge assessments score and CMS overall quality star rating, as there is only 4% probability (p=0.04) the null hypothesis is correct. Therefore, the null hypothesis was rejected, and the alternative hypothesis was accepted.

Graph image from HCA CMS Star Rating to Jane Article
Figure 1. Joint plot of CMS overall star rating versus overall HealthStream Jane® average score.

Limitations of Study

This study only included 22 acute care hospitals that were utilizing HealthStream Jane® during the period of November 1, 2019, through October 31, 2020. Facilities not using HealthStream Jane® during this period, or those that did not have an overall star rating provided by CMS were excluded.

Conclusion

A positive correlation exists between overall knowledge assessment average scores and overall CMS star rating. For every one-point increase in the average knowledge score, there is a corresponding 0.1202 increase in star rating. As pointed out by Shetty et al. (2021) several items correlate to an increase in star rating, and as this study demonstrated, a knowledgeable staff as evidenced by assessment scores is one of them.

Author

Ying Ding, Richard Tran, Joseph Caracci
HealthStream, Inc., AI & Data Outcomes Team
HealthStream

Download the Article 

References
1. Canfield, S. J. (2021). Nursing Professional Development: Lifelong Learning. New Mexico Nurse, 66(2), 10–15
2. Center for Medicare & Medicaid Services. (2021). Overall Hospital Quality Star Rating on Care Compare Methodology Report (v. 4.0). Retrieved from https://qualitynet.cms.gov/files/603966dda413b400224ddf50?filename=Star_Rtngs_CompMthdlgy_v4.1.pdf
3. Graebe, J. & Dickerson, P. S. (2021). It is not about the contact hours! The Journal of Continuing Education in Nursing, 52(2), 55-56. http://dx.doi.org/10.3928/00220124-20210114-01
4. Shetty, K. D., Robbins, M. W., Tolpadi, A. A., Campbell, K. N., Clancy, A. M., Bodkin, N., Durham, M., & Damberg, C. L. (2021). Actions to Improve Quality: Results from a National Hospital Survey. American Journal of Managed Care, 27(12), 544–551. https://doi-org.wilkes.idm.oclc.org/10.37765/ajmc.2021.88793