Embi discussed the challenges around algorithm development and monitoring and the goals of VAMOS. 

“We often develop these algorithms based on the data we have and in the environments we have, and we validate them, and then we deploy them, but we tend to not have the capability to be able to do ongoing systematic monitoring and updating, and using more of a learning health system approach. That’s our goal,” he said. “Years ago I wrote a paper analogizing what we need to do with the increasing frequency of AI solutions in healthcare to be able to monitor them effectively to pharmacovigilance, and there's a lot about that analogy that holds up, and certainly some things that don’t. But an important idea is that if we don't keep an eye on what's actually happening, and look at not just the performance metrics technically, but the outcomes of interest, then we're not really going to know whether or not we're moving the needle on effectiveness, equity, and other important considerations.”

The concept behind VAMOS, he added, is the creation of a socio-technical solution to enable organizational governance and oversight, team-based monitoring, the ability to respond to any issues that are being seen in near real-time, and the capture and reporting of AI performance. 

“The idea was a dashboard where we had defined metrics where we could set bounds for performance and safety bounds. We would look at metrics, including those such as accuracy and precision and drift…but also responsiveness to alerts, as well as measures of fairness and equity,” Embi said. “We wanted to build into this feedback mechanisms — getting more from the field, but also being able to respond to things we see, and take actions as a result, rather than waiting for a catastrophe or a serendipitous study, but actually doing this in a more real-time operational fashion that might lead to us investigating the cause, correcting and updating the model, notifying teams, and even pausing the algorithm if necessary.”

Embi showed a screenshot of the current version of the VAMOS dashboard, including Cornelius, which is what VUMC calls its readmission prediction model. VUMC teams can drill down to look at real-time information and have the ability to select other metrics beyond the default metrics. 

“We’re clearly at a 1.0 version,” Embi said, “but this has been a really interesting exercise. We’ve learned a lot, and we've launched a collaborative. We’ve now licensed this out of the university to establish an open source collaborative where we have early adopters that are going to be taking on the use of the platform, so that we can create a network with the idea that we will be able to inform what are the standards we need to be using here, how do we share learnings about doing this in the real world, and eventually, in terms of the future network capabilities, be able to have continuous model performance across settings, as well as the ability to learn from others.”

Health systems listed on his slide about the collaborative include UCSF, OHSU and Mass General Brigham. 

Embi gave an example of how the network effect might prove valuable. “If one of our colleagues from another site is using the platform, and perhaps they're running the same algorithm we are, and they're starting to see an issue, because maybe they have a larger Hispanic population than we do, for instance, we might learn from that and be able to take action because of an adjustment that needs to be made. Or if we're looking to adopt something that has been implemented somewhere else, we now have the potential to have an evidence-generating network that can help inform that, and many other use cases that flow from that.”

The VAMOS Collaborative has already been meeting and members are in the process of disseminating a version of the platform and creating an open source community. “We've also engaged with standards organization HL7, who's at the table to help us with a lot of the standards components, and with the Trustworthy and Responsible AI Netwok,” Embi said. “Work streams that are already under way include creating that open source ecosystem, enabling convergence on platform functions, co-designing capabilities, standardizing our metrics and ultimately building the foundation for trustworthy AI at scale, which is our goal.”

In 2025, the Vanderbilt team published a preprint that provided greater detail about their work on VAMOS.