Analytics tools to support operations beyond COVID-19
The rural health system is focused on the relationship between patient flow and staffing levels amid the nationwide nursing shortage. Nowak said the analytics service deployed a model to determine labor allocations based on bed usage.
“The idea is to have the right person in the right place at the right time,” he said.
Nowak added that the health system monitors metrics such as staff turnover and overtime, as well as responses to employee surveys, to measure the success of their operating models.
“If you can keep an employee who is much more fulfilled, who loves coming to work, who is having fun at work, they will be happier and our patients will have better outcomes,” he said.
Avoid the “black box”
The technology would be obsolete without constant monitoring for efficiency, reliability and bias. Predictive models may be less accurate for vulnerable populations and may exacerbate existing health disparities. That’s why it’s important for developers to understand each model in its entirety and understand that the analysis tools are only aids, with no autonomous decision-making capabilities, Nowak said.
The technology “gives the vendor one more tool in their toolbox,” he said.
“When you step into these algorithms under development, you hear about the infamous ‘black box,’” he added. “He’ll spit out some predictions and you’ll say, ‘Well, that’s what the computer said.’ ”
To prevent this from happening, hospitals subject their models to a variety of stress tests.
Metsker said hospitals should also use sample sizes that reflect the populations to which they apply predictive models. Otherwise, the models will carry a bias.
“If you’re talking about a clinical model, these variables need to be validated with a reasonable population. A hospital in downtown Chicago has a different population than a hospital in Gig Harbor, Washington,” he said. “On the operations side, you have your different set of variables [depending on location]— volumes, services, geography and finances.
Once the models are deployed, analysts say they are checked in real time for accuracy and compared to historical data to track longitudinal effectiveness.
“I always remind people how we do that,” Marroquin said. Any decision to use analytics tools, he said, “stems only from the need of an organization.”
The COVID-19 pandemic has become a testing ground for health systems and big tech companies to launch cutting-edge technology in unprecedented time.
According to Dr. David Rhew, Microsoft’s global medical director and vice president of healthcare, the focus should be on expanding the tools for other use cases.
“We’re seeing efficiencies, we’re seeing provider experiences being learned, and we’re seeing patient experiences improving. So it became an interesting element, because we’re not talking about trying to solve just one thing anymore,” he said. “These technologies can help us address many other aspects.”
As an example, Rhew said Microsoft is working in the natural language processing space to develop tools that track conversations in the exam room and aggregate them into a patient’s EHR. It can reduce the time doctors spend taking notes, Rhew said, as well as the exhaustion they can feel juggling multiple tasks at once.
Health systems are also sowing the seeds of future innovation by partnering with colleges, in hopes that students will settle into data-driven healthcare careers.
For example, Virginia Mason and Sanford have joined forces with their local universities – the University of Washington Tacoma and Dakota State University, respectively – to recruit the next generation of clinical analysts, data scientists and information technology specialists directly from the classroom.
At Virginia Mason, four students from UW Tacoma’s Master of Science in Business Analytics program work at the system’s mission control command center, Metsker said. They are responsible for creating models in R or Python programming languages to identify ancillary service backlogs and make staffing recommendations.
And at Dakota State University, Nowak said business and technology students can participate in different health e-technology programs through Sanford’s pipeline partnership with the school. The program can attract students to Sanford before another employer catches their eye, he said.
“The quality of applicants I get today, compared to even 10 years ago, is day and night. These are young people who want data and they want access to it,” Nowak said. It’s amazing, when given that access and a bit of freedom, what they can do.”