One health system found that artificial intelligence (AI) can provide an effective means to ensure the right nurses are hired for the right roles.
Healthcare labor markets have never been tighter. The U.S. Bureau of Labor Statistics (BLS) in February 2018 reported that healthcare unemployment remained at 2.5 percent, which is below what often is considered full employment. A survey by Compdata found the average turnover in healthcare jobs in 2017 was 20.6 percent. a
The impact of such turnover rates is most problematic among nurses, whose work directly affects safety, quality, and patient experience. The most recent data from NSI Nursing Solutions indicate that the annual turnover rate for bedside registered nurses (RNs) is nearly 17 percent, and first-year turnover for new nurses is roughly twice that rate. b
The outlook for the next several years is more ominous: The National Council of State Boards of Nursing says half of all nurses are aged 50 or older, and more than 1 million of them are expected to retire in the next decade. c The BLS projects 649,100 replacement nurses will be needed by 2024, which, combined with the needed growth in the nursing ranks, would create 1.09 million job openings.
MultiCare Health System’s Experience
Leaders at MultiCare Health System, Tacoma, Wash., were aware of this challenge, having faced nursing turnover surpassing 10 percent in 2015, with first-year turnover nearly twice that amount. MultiCare competes for talent with other large health systems across the entire region, including the Seattle metro area and in Spokane. In King County, which includes Seattle, the competition for experienced and newer RNs is especially fierce. Accordingly, MultiCare has built a pipeline of new talent by working closely with local colleges and establishing a nurse residency program.
One reason the organization strives to attract and successfully retain highly skilled nurses is that filling the vacancy created when a nurse leaves is an expensive proposition. MultiCare estimates that the average cost of filling a vacancy—including overtime, lost productivity, pre-hire recruitment, and training—amounts to $80,000. Because nurses make up a large percentage of the system’s clinical staff, the cost of replacing nurses annually exceeds $1 million.
Beyond the expense, operating with numerous vacancies or continually relying on fill-ins can be noticeable to patients, which can have an impact on their satisfaction with their care.
Improving this picture is central to MultiCare’s strategic priorities. The organization’s plan is founded on three pillars:
- Performance excellence
- Population-based care and market expansion
- Access to care and service
In the first pillar, performance excellence, MultiCare leaders look to four measures, each of which is affected by its staffing success. For each measure, MultiCare established a goal of being in the top decile among the nation’s health systems.
The first measure is employee and provider engagement, which drives MultiCare to make its system the best place to find fulfillment at work as well as being a safe workplace. The second and third measures reflect the goals of being in the top decile in quality and safety as well as in patient experience.
For its fourth measure, MultiCare’s goal is to be in the lowest quartile in total cost of care and the top quartile in operating margin. As a not-for-profit organization, this measure correlates with the necessary net revenue to allow the organization to continue making investments in the community.
The Opportunity for Applying AI
In 2015, although MultiCare had worked hard to be an employer of choice and engage its current staff, more needed to be done. MultiCare had a new president and CEO, Bill Robertson, who had just started in May 2014. In his prior role, Robertson had overseen the application of predictive analytics and machine learning—components of the broader discipline known as AI—to reduce turnover and improve employee engagement. He saw the opportunity to use this solution again in MultiCare’s large, geographically diverse health system.
Many healthcare providers and insurers have begun to use AI in clinical care and process redesign to assist in personalized medicine and in making decisions such as where to place new clinics and what those clinics should offer. Some organizations have also begun to explore ways to use these new analytics tools to determine which candidates are most likely to make the best hires and stay on the job long enough to justify the investment a health system will make in them.
Although people charged with hiring in organizations tend to believe they can be objective in evaluating potential hires, evidence suggests that it is difficult for any of them to avoid bringing subconscious preconception to the task. A meta-analysis of 17 studies of applicant evaluations, first published in 2013, found that a simple equation outperformed human decisions by at least 25 percent. d The research indicates this effect may be widely applicable, provided there is a large number of candidates, regardless of whether the job is on the front lines, in middle management, or in the C-suite.
Data Requirements for Applying AI Successfully
Applying analytics to data meaningfully constitutes the chief preliminary challenge of an AI approach. A report by the Advisory Board (see the exhibit above) illustrates some of the challenges of moving across the spectrum of analytics, from accumulating data that describe what is occurring with hires to determining who is likely to leave to ultimately determining who to hire and when. Studies of AI utilization in the healthcare workforce to support the hiring process suggest that the most common impediments to organizations’ success are a failure to capture the right data, and an incomplete understanding of which talent dimensions drive outcomes in the organization.
Moreover, unlike many earlier business software solutions that could be developed and implemented by an organization itself, using AI requires a more complex implementation and customization. However, once an AI solution is in place, machine learning can take place on a continuous basis, enabling the solution to keep pace with an ever-changing organization with little to no manual intervention through automated data capture and model training. The concept is that by leveraging large amounts of data an organization can objectively evaluate a huge variety of data points relating to thousands of people, thus gaining valuable signals from patterns in the data instead of emotional, subjective, and subconscious perceptions. Finding or developing a specific solution is a part of an organization’s due diligence in undertaking such an approach, and this effort is outside of the scope of this discussion, which is focused on the application and potential results of the technology.
MultiCare chose a cloud-based tool that integrates with its applicant tracking and human resources information systems. The solution uses data from the job candidate’s application, interactions with the platform using a mobile app provided, and publicly available data from third parties.
Custom predictive models for each location, department, and role were created. For example, the predictors of retention for a ER nurse candidate in one hospital in the system could be very different from the predictors of retention for a ER nurse candidate in a different hospital in the same system. This could be a result of the leadership or culture within those two different facilities, the patient mix that each facility serves or unique characteristics of the different departments themselves. As outcome data are fed into the tool, machine learning comes into play over time, and predictions of applicant success grow more accurate. One example of outcome data would be a data feed of all hires and terminations. Those data allow the tool to observe who is hired, who is retained, and who leaves the organization. Those data, coupled with the data captured about the employee prior to hire, allow the tool to learn how best to predict the likelihood of retention for a new candidate.
The predictive analytics solution was deployed at MultiCare in nursing across the system in May 2015. The deployment took approximately eight weeks and involved:
- Determining which roles to target
- Collecting applicant, interaction, and outcome data
- Building customized models for each location, department, and role
- Integrating the applicant tracking system into the tool
- Designing and implementing applicant, recruiter, and hiring manager workflows and experiences
- Automating the continuous refresh of outcomes data
The engagement focused on two groups. The first was the human resources (HR) team, because this new process needed to be incorporated into the hiring process workflow. In general, not much changed. HR simply received the candidate’s application along with a recommendation generated by the algorithms. Because MultiCare is concerned about flow of applications, the recruiter forwarded all the applications to the other concerned group—nursing leaders. Chief nurse executives still make the final decisions.
Just as with every other process change in health care, nurse leaders raised initial concerns from nurse leaders, who wondered whether it would reduce the number of applicants for every position. At the beginning it was mandated that the algorithm-based prediction be used as the basis for the hiring decision. Later it was changed from a mandate to a recommendation, accommodated situations where there were not enough recommended candidates.
Hiring managers were shown their unit’s turnover data, and how the data compared across the MultiCare system as well as against similar units in other healthcare organizations. As a result, more nurse managers were encouraged to use the data to make the hiring choices.
Following these training sessions, MultiCare observed a shift in departments across the system to hiring larger percentages of recommended candidates, with some departments hiring exclusively recommended candidates. And just as with every other change, a critical aspect of the approach was stakeholders’ confidence that senior leadership was fully committed to achieving the goal and willing to engage in ensuring success.
A Focus on Engagement
In just the first four months after rollout, 90-day turnover was reduced by 36.5 percent. Through the third quarter of 2017, predictive-modeling-based recommended hires turned over, on average 29 percent less than those not recommended. More than 20,000 applications have been processed, and the applicant completion rate on the app is more than 90 percent. This level of performance has generated an estimated $1.9 million annual net cash savings.
A Focus on Retention
Making good hiring decisions is one thing; keeping new hires is another. Retention is a huge challenge, so MultiCare is now very much focusing on how it is engaging its employees. Through regular leadership rounding, leaders can have an ongoing dialogue with nurses on the care promises they are creating and fulfilling together. The team is working on numerous other engagement initiatives in conjunction with predictive modeling to help MultiCare get to the top 10th percentile.
In August 2017, MultiCare added to its tool algorithms for predicting the likelihood that a prospective employee would be retained and the additional likelihood that the candidate would demonstrate engagement in the given role and location. With this new capability, HR has been able to optimize for these two critical outcomes at the point of hire, which has helped mitigate risk associated with candidates identified as being unlikely to achieve either outcome.
For example, if a candidate is predicted to be an engaged employee, but exhibits risk of leaving, the manager can decide whether that level of risk is acceptable. In addition, if a manager decides to hire such a candidate, the manager can institute best practices such as 30-, 60-, and 90-day checkpoints or assign a mentor to the new hire to help ensure the person stays on the job.
As people who have been in leadership know, evaluations of engagement place employees along a continuum from disengaged in their jobs to ambivalent to content to engaged. Clearly, it is most desirable to have engaged staff, which means they view themselves as part of the team and are committed to working to further common organizational goals. From 2016 to 2017, the percentage of MultiCare’s nursing workforce who met these criteria for being engaged improved from the 34th percentile on Advisory Board’s benchmark to the 37th percentile. Even more notably, at least for MultiCare, was that the disengaged portion fell from 7.2 to 6.2 percent. Although these may seem like small changes, the organization’s leaders saw it as a move in the right direction.
Other Applications of AI
MultiCare also has applied AI tools to its nurse residency program, which led to another key finding. When newly minted nursing school grads apply for a residency, they list multiple departments they might want to work in. Using AI, these candidates can be evaluated based on organizational fit, much like a potential new hire, though comparisons with existing nurse residents on likelihood of retention and engagement. A recruiter then can see how an applicant might perform in a certain department and can evaluate the person’s likelihood of success as a nurse in the emergency department at Tacoma General, for example, versus retention in the OR at Good Samaritan.
This is now an evidence-based process, and the data show where to focus improvement efforts.
A key lesson that MultiCare has learned from adoption of AI in these applications is that it is critically important to be forthright and informative from the start, to ensure hiring managers and HR understand exactly what is about to happen and why. Team leaders should fully articulate the value of the predictions and reinforce the message that high turnover is not a reflection on anyone because it is a truly national problem.
For MultiCare, the future is in expanding the use of AI beyond staff nurses to other roles, including clinical nurse assistants and medical technologists. The organization expects to explore how the platform may someday be used to predict outcomes even for physicians and senior leadership.
Meanwhile, MultiCare will be working to identify additional outcomes to refine how employees fit into its performance goals and transformation agenda. These considerations include a new hire’s likely impact on patient experience, readmission rates, and other quality metrics.
Footnotes
a. Fanning, K., “Pay, Turnover and Recruitment, Oh My! 2017 Healthcare Results Now Available,” blog post, July 13, 2017.
b. NSI Nursing Solutions, Inc., 2018 National Health Care Retention & RN Staffing Report , 2018.
c. National Council of State Boards of Nursing, 2017 National Nursing Workforce Study, 2018.
d. Kuncel, N.R., Klieger, D.M., Connelly, B.S., and Ones, D.S., “Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions: A Meta-Analysis,” Journal of Applied Psychology, Sept. 16, 2013.
