In June the National Committee for Quality Assurance (NCQA) appointed Ari Robicsek, M.D., as its first chief medical officer. In July, he spoke with Healthcare Innovation about his goals for accelerating NCQA's quality measure modernization agenda.

Robicsek joined NCQA from Providence, where he served as executive vice president and chief analytics and research officer. Before that, he was vice president of clinical analytics and associate chief medical information officer at NorthShore University HealthSystem.

Healthcare Innovation: First, could you talk about why NCQA wanted to create a chief medical officer position now and some of the activities your new job will entail?

Robicsek: I think NCQA was interested in in creating a role to help lead the work around ensuring that the measures that we build are measures that matter and are going to work in the future. I think the organization recognizes that there's a lot of opportunity to collect and use more information than we do today, information that does a better job of capturing clinical practice and clinical decision-making — the meaningful quality of care and the outcomes that patients are experiencing, which is something that we couldn't really get at before, especially from the patient's perspective.

There is huge potential to expand the way that measurement is being done, so they looked for somebody who had experience in the provider space, both working as a clinician and doing analytics in provider organizations using lots of electronic health record data, and somebody who understands workflows, and also somebody who's done work in the space of artificial intelligence, recognizing that AI will probably be a valuable enabler for measurement in the future.

HCI: In the press release announcing your appointment, NCQA President and CEO Dr. Vivek Garg said that the growth of more interoperable data and AI modernization makes this an important time to define the next generation of digital quality infrastructure. What do you see AI enabling that wouldn't have been possible five years ago? 

Robicsek: Here is one example: a huge amount of what is important about the way a patient is being cared for is buried inside unstructured information in records. It’s buried in the notes. There have been huge advances in using AI to actually extract valuable information from notes. Imagine a world where we're able to use information coming from the notes to get at whether the clinician is making a good decision in this particular case and where we're not relying on incredibly simple heuristics that don't reflect the complexity of real-life decision-making.

Many clinicians are using ambient intelligent tools like digital scribes in their practices. Imagine if we could use tools like that to get at questions that we've never been able to answer before about the experience of care. For example, did the doctor listen to the patient? Did the doctor answer the patient's questions? Did the doctor explain things well? Was there a shared decision-making process that happened in the interaction? Those are things that we've been completely blind to until now. 

I feel like we are very lacking in information about what happens to patients after we care for them, and that lack creates at least two huge holes in our system. One is if we don't know whether people got better after we treated them. Then our mechanism for paying clinicians can't be based on whether the doctor helped the patient get better. It's based on what the doctor documented they did. So that's really a few degrees of separation from what we would like to pay the doctors on. Did they help the patient get better and stay healthy? We can't pay on that because we don't measure it, but imagine if we were systematically collecting information about whether patients got better in the context of the care that they received, and we got that information directly from patients. That could be directly through patients answering questions using patient-reported outcomes instruments. It could be through AI operating inside wearables, collecting information about what a patient is now functionally able to to do during their day. If we could measure that, we could use it to inform payment.

Also, we’re not systematically learning from what we do. So if we do $4.5 trillion of healthcare every year — what’s the value of the learning that we get from providing that care downstream of that care?

Imagine if we collected outcomes data. Then we could use that to figure out which interventions work for which people and then feed that back into the system. Imagine if we made better use of AI tools to interact with all of us as consumers of healthcare to get at whether we are actually getting better and actually living the lives that we want to live to help inform both payment and learning. For me, that's the most exciting frontier possibility of AI.

HCI: Are the reasons the industry hasn’t yet done more with patient-reported outcome measures technological in nature — or are they cultural or financial?

Robicsek: I think yes to everything. There's a little bit of chicken and egg, where we pay based on just the things that the doctor documents that they did, and we're not paying based on how the patient did, so the doctor doesn't have an incentive to collect how the patient did.

If you ask doctors if they want to know if their patient got better, they absolutely do. But if you ask if they want to build and pay for an infrastructure for systematically collecting that information, they would say nobody pays me to do that. So that's the chicken part of it. The egg part of it is that we don't pay doctors on whether or not people are getting better because we're not collecting that information, right? I believe that it would be culturally possible to get to a place where we break that chicken-and-egg problem by more systematically collecting information from patients on outcomes if we can make it inexpensive and really easy to do. And that's where I think the technology comes into it.

HCI: Let me ask you about your time at Providence. You spearheaded something called the Value-Oriented Architecture initiative. Could you talk about that framework and some of the impact you saw it having within Providence?

Robicsek: It came out of the insight that there was an incredibly high degree of variability in how clinicians manage the same problem. If the same patient had surgery done by the 200 different orthopedic surgeons inside of Providence, there would be a whole set of differences across those clinicians — and some of those differences led to very large differences in how much the procedure costs. 

We wanted to know whether all this variability is warranted, and the answer is almost certainly not. But unless I come to orthopedic surgeons  with outcomes, that's not a conversation that's useful. That's why it wasn't a cost-oriented architecture, it was a value-oriented architecture. We built a data system that allowed us to simultaneously look at the inputs: how much does it cost to care for patients with a given condition, and it could be unilateral primary knee replacement or it could be coronary artery bypass surgery.

We created scoring based on a very distilled group of measures that clinicians found motivating. We created a a data platform that allowed us to say for a particular clinical scenario, here's a scatter plot that shows, for example, all the orthopedic surgeons in our system doing this particular procedure, and everybody is shown on two axes. Axis number one is the total cost of doing the procedure, and the other axis is the quality of care that they're providing. And where you want to be is in the quadrant where you're getting great outcomes at relatively low cost. We hade lots of surgeons getting great outcomes at low cost, and other surgeons who are getting great outcomes at high cost. Now we can go to those high-cost surgeons and say, "Hey, your colleague down the hall is getting just as good outcomes as you, but they're not using this particular cement or this very expensive intra-operative medication, or they're not using as many staff in the operating room.” Then we were able to identify many ways to reduce the cost to the system.

We were able to drill down and say here are the specific things that you do differently than your colleagues that are driving costs. And then because we made it actionable, they were able to change. We brought it to the point of clear visibility and actionability, so we were able to bring about change. 

After the first few years we achieved at least a $50 million-a-year savings for the system by getting in front of clinicians and actually talk about value, not just talking about cost.

HCI: Does that work inform your thinking about measurement at NCQA? 

Robicsek: It strengthened my feeling that doctors want to do the right thing, but you need to give them useful, actionable information. Part of my philosophy of measurement is that measurement is intervention. So in the same way that giving patients a medication is intervention, introducing a measure into a population of doctors, especially if it's an incentive-backed measure, that is an intervention because there's a good chance it's going to change clinician behavior. If it changes clinician behavior, we hope that it's going to change clinician behavior in the right way. I think we can do a lot more to bring available data and real-world evidence sciences to bear on figuring out whether or not once we implement an intervention that is an incentive-backed measure, the right thing is happening, and we’re bringing about the change in behavior that we want. 

Oftentimes, when we measure something, including incentive-backed measures, it's something that the doctor herself doesn't have control over. So, for example, if we say we're implementing a measure that a particular guideline needs to be followed, and patients in a particular scenario should be on a particular medication, what happens if that medication is unaffordable to patients or requires a prior authorization or there are other major barriers to patients getting that medication? That's a measure that is not intervenable for that clinician. So you have what economists call a principal-agent problem.

I think that we don't do enough work when we design measures as an industry in thinking about principal-agent problems and then helping design our measures so that they can help with those things. 

HCI: Could you talk about the shift to electronic clinical quality measures? There has been some pushback from ACOs and others to CMS that the pace of change has been too fast.

Robicsek: In a world where the chassis of our measures is administrative data, we're pretty far removed from what's happening at the bedside, what's happening in the exam room. Without using the massive amount of expanded digital information available to us, we’re losing out on really important information — for example, risk adjustment information that might allow us to make our measures more fair or information that might allow us to make our measures more respectful of the clinical process. 

Now we have measures that are extremely simplistic. They say: Doctor, if you see a patient with X, you must do Y. Well, there are 100 scenarios where actually, if you see a patient with X, you shouldn't do Y, but it depends on something subtle that's oftentimes buried inside the electronic record and not available in claims data or other very standard data sources. What we end up doing is penalizing doctors for doing the right thing.

We can make our measures a lot more useful if we're not tying our arms behind our backs with the very limited sort of data sets that have traditionally been available. Part of that is actionability. Claims always have a lag in them. I was sitting in a meeting the other day with a chief medical officer from a health plan who said that your data comes to us and it feels like light from a distant star. You can't manage a business on signals that are light from a distance star, right? So there would be great power in having data that's way more timely than it is today.

We have already talked about patient-reported outcome measures that you're not going to get out of traditional data sources…So I totally appreciate that it is a painful evolution. I started my career with paper charts and I spent years working in the trenches. I was an associate CMIO at a healthcare system and very involved in the work of standing up electronic medical record systems. I recognize how challenging and more importantly how expensive these transitions are. I think that what this change will bring us is what I call measures that matter, that detect and catalyze good care. 

HCI: What about the importance of multi-payer alignment on harmonizing quality measures? One of the things we hear a lot from people participating in value-based care programs is that each payer has its own set of measures and it adds this huge layer of complexity.

Robicsek: Having just come from the provider side, the cacophony is real. Actually, I published a paper with a health economist friend of mine named Claire Boone in JAMA Health Forum, where at Providence we looked at how many different quality measures the average primary care physician was incentivized on in their contract, and there were 57 on average per primary care physician. It's impossible to run anything like that. One of my goals, and I know Vivek's goals as well, is to the extent it is in our power, we want to work on measure harmonization across the industry. That is a very important goal.