Transcription
Moving. Hello. Yeah, they're moving. Excuse me. We request you to please exit quickly so that we can start our next seats. Welcome back to JPAL's AI for Social Good Impact That Works seminar. Our next session is AI in Health: Saving Lives at Scale. Please join me in welcoming Zad Overme. Zead is an associate professor of health policy and management at the UC Berkeley School of Public Health, where he works at the intersection of machine learning, medicine, and health policy. He helped set up the computational precision health program, and his research focuses on how machine learning can help clinicians make better decisions. A fun fact about Zead is that he was named one of the 100 most influential people in AI by Time magazine. Zad, over to you.
Thank you, uh, so much. So I thought, um, uh, I would tell you a story about when I first came to my job at Berkeley. So this was in 2018. I was a junior, untenured faculty member, and I had just signed up to give the first seminar of the year, uh, in the economics department. Now, the economics department at Berkeley is, uh, friendlier than the economics department at other institutions affiliated with JPEL. Uh, but they are still economists, and still very hardcore. So I was nervous. And about, um, three or four days before the seminar, I started feeling a little unwell, like my my stomach was hurting. I just didn't feel right. And I made the mistake of mentioning this to my wife, um, who said something like, "Oh, is your tummy hurting? Are you a little nervous about your?" Anyway, uh, she was not very sympathetic.
So I, uh, I prepared for my talk. I prepared for my first class that I was teaching. Um, and I gave the seminar. And during the seminar, instead of feeling just globally not well, I started feeling, um, unwell in a particular place in my stomach, which is the bottom right side of my stomach. Um, some of you might know what I have already, but I want to first make an observation that this made me realize, which is that, um, when people make really important health decisions, like, "Do I come into the hospital? Do I ask a doctor about this?" very, very important with very little data. So at the end of the seminar, I actually drove myself to the emergency room, uh, near Berkeley, uh, and I got a CT scan. And this is my CT scan. Um, and some of you might be able to see what I have or guess what I have, but I had acute appendicitis, and it had perforated, and I needed to have my appendix removed. Um, I think what makes this story surprising and a little bit embarrassing for me is that I trained as an emergency room doctor. So, there are a few things that you're not supposed to miss as an emergency room doctor, and this is one of them. And yet, I missed it in myself for four or five days.
So I think what this highlighted to me is how critically important it is to be able to have access to this kind of data, the kind of data that normally live only inside of the health system, and that you can only have access to if you're lucky enough, um, to be close to an emergency room with a CT scanner, to have access, both financial and linguistic and socioeconomic, to these kinds of technologies. Because without data like this, there's no diagnosis of critical illnesses, and without diagnosis, there's no treatment. So this data is really the key to a lot of things that happen in the health system. And yet, in order to get access to this, you have to be part of the health system. You have to have access to begin with. So there's a circularity to this problem, and it locks out many, many people, uh, of course, not just in the US, but but even more around the world.
So, um, this kind of underdiagnosis is of course not just limited to acute surgical things like appendicitis, um, but to many, many different conditions. And a particularly salient example, um, to many people, happens even in high-income countries, which is heart attack. So there are these studies that follow big cohorts of people across time in the US, and it, um, every so often, they do an MRI of the heart to detect scarring in the wall of the heart that indicates that this person had a heart attack. And in these studies, when they look at all of the people who have scar in their heart consistent with a heart attack, most studies find that the majority of those heart attacks are silent. They are not known to the patient, and not known to the doctor, and not diagnosed, either because the symptoms were subtle, um, or because the person never sought care to get those symptoms diagnosed. So, as you can imagine, this is, um, a huge problem even in high-income countries. It's an even bigger problem elsewhere in the world.
And so, um, there's, uh, data, uh, from South India, from Tamil Nadu, um, that asks people about things like high blood pressure and diabetes, and asks them, "Do you have these things?" and then does the objective measure of those things. And even for these basic risk factors for heart attacks, we, we can't do cardiac MRIs in a lot of places, but even for these risk factors, there's dramatic underdiagnosis and underawareness of the things that cause heart attack. So, this is a problem everywhere. Um, and it's a problem that really urgently needs solutions. Um, so I thought I could tell you, I'm going to tell you about half of the solution. Um, and instead of, uh, just telling you, I'm going to show you. And you'll see more of this, uh, from my colleague Shredda, um, in, in a few moments.
This device is a mobile handheld electrocardiogram. So, the two silver sides are electrodes. You put your fingers on it, and it records the electrical field generated by your heart as it beats. And I think these technologies are really miraculous because they cost so little. That device costs $60. In the US, you can order it on Amazon and get it delivered in the same day. Uh, it connects to your phone or tablet via Bluetooth with no wires, and it requires absolutely no training to do an electrocardiogram. So this is really an incredible technology, um, that that's become commoditized over the past 5 to 10 years.
But the reason that these kinds of devices have not had the impact, um, that that any of us would have hoped, is because there's a key constraint, which is that what do you do once you generate this waveform? Well, you have to send it to a doctor for interpretation. This is too complex for patients, and even for some doctors, like myself, have trouble interpreting electrocardiogram waveforms. And so, if you need to, it's great that you can generate this data wherever you are in the world. But if you still need to send it to a doctor to get interpreted and to translate those data into decision-making, you haven't solved the key bottleneck, um, which is access to the healthcare system. So even though these devices are wonderful and miraculous, they are only part of the solution to democratizing access to diagnosis around the world.
So this is an AI summit. Um, could AI help? Yes. Uh, but it's important to remember that there's nothing magical about artificial intelligence. And in fact, artificial intelligence is just data, in a very literal sense. So in an ideal world, what you'd have as far as a data set to train your AI algorithm is a data set that links data from these cheap devices to some ground truth outcome about heart attack, appendicitis, etc., etc. Unfortunately, those data on the ground truth are quite hard to find, um, even in high-income countries, and certainly in low and middle-income countries. And so, what we often do when we're training AI tools is we make a key and very pernicious substitution, which is we say, "Okay, well, we can't get the ground truth linked to these electrocardiogram waveforms. So instead, we're going to ask doctors to look at the waveforms and tell us what the doctor thinks about that waveform." So we substitute human judgment and human interpretation for some sense of the ground truth of, "Does this person really have a heart attack or not?" Let's just ask the human. And the problem with humans, many of my good friends are humans, so this is nothing about humans in general. But humans have biases, and humans make mistakes. And human knowledge about the human body, and many other things, is incomplete. And so when we automate human judgment, we also run the risk of automating all of the problems with human judgment along with it. And I think that's what makes it so critically important to train artificial intelligence not on doctors, but on patients, on what happens to patients and their outcomes.
So, um, let me tell you about some work that I, um, am doing in collaboration with JPAL South Asia, uh, in a few, uh, villages in Tamil Nadu. So, uh, this is a line of people queuing up, um, for a district health center. And what we did over the summer of 2023 is set up, uh, health camps, a long-standing tradition, um, where we just had a, um, a facility outside of this district health center where, uh, thanks to the tireless work of, um, of our colleagues at JPAL South Asia, we rented out a bunch of very fancy equipment that you can normally only get inside of hospitals. So, for example, we rented out a cardiac ultrasound machine and a technologist to acquire the images of cardiac ultrasounds. We hired cardiologists to interpret those images and look, uh, among other things, for signs that this person had had a heart attack, signs of scarring in their heart. And at the same time, we did data collection from these cheap $50 devices. Um, so here you see two patients that were, uh, participants in this health camp. Um, uh, this young lady is having an electrocardiogram done and sent to the tablet in front of her. Um, and the gentleman over here is getting a cardiac ultrasound, which serves as our way of understanding if this person has had a heart attack by looking at, um, wall motion abnormalities in the walls of his heart. So every patient got both the fancy, expensive tests from the hospital and the cheap, low-cost tests. Um, and of course, if we found any abnormalities, we communicated those back to the patient and their doctors, um, as, as part of this study.
So, um, so the eventual goal of this is to create a way to screen people for a number of serious health conditions, including prior, potentially silent heart attacks, in the comfort and safety of their own home, either by doing the electrocardiogram themselves or with the assistance of a community health worker. And just to give you a sense of how well, even this early version of the algorithm is working, when we flag the highest risk, two, three, four, five percent of the population, just general population of people, um, visiting an ambulatory health center, 10% have actually had signs of a prior heart attack. And that's compared to about 2% in the general population, um, of people that we were looking at. So, we're finding very high-risk people, and people who, even though the algorithm is saying that they're high risk, and they do indeed have silent heart attack, um, on their, their echo, their, their cardiac ultrasound, they lack the traditional risk factors like blood pressure and diabetes that are traditional, because they were defined in Western populations. And as we know, South Asian populations have different risk profiles and different risk factors that this electrocardiographic approach is able to find, despite the fact that they lack those traditional risk factors. So I think this highlights the fact that, um, these algorithms can be trained and tuned to populations all over the world that might be different from the populations in which most of our medical knowledge is developed.
When we put this through the cost-effectiveness of screening, in other words, sending high-risk people for confirmatory tests at the health center and treating them with the medications that you need to be on for people who have had prior heart attacks, this comes out to be cost-effective, um, even using Indian guidelines. So it's around $2,000 per disability-adjusted life year, even for this very early version of the algorithm, which is continuously improving as we collect more data. So this is very exciting. And the next step is actually, uh, this wouldn't be JPAL unless the next step were a randomized evaluation. Uh, and so we're doing a randomized evaluation where we're going to start screening people either with traditional risk factors like blood pressure, um, etc., or this AI-enhanced electrocardiogram screening approach, and referring the high-risk people for confirmatory testing in both groups, and seeing which approach finds more people that we want to find, people with prior heart attacks.
So, um, one of the exciting things about doing this project is that, unlike a lot of the research that, um, that I've done in the past, this is a real thing. It's a device, and it's already interacting with real people. And so, to give you a sense of that, um, if you don't mind, I'm going to introduce, uh, Miss Shesh, who is an instrumental part of our team with JPAL South Asia. And we're going to violate every principle of giving a talk by doing a live demo and hoping that it works. Um, so if you'll bear with us for a second, we're going to get this set up, and then you will see my own electrocardiogram being put through, uh, our, um, our web application and screening me in real time, um, for my own risk of prior heart attack. Um, luckily, since I'm a physician, uh, nobody needs to take responsibility for the outcome of the test besides myself. Um, all right. So, we're getting it hooked up, and I'm hoping it works because we tried it before the talk. But of course, live demos have a way of frustrating your expectations. Uh, we're going to switch to the other podium.
Um, one thing I'll, um, I'll just mention as we're getting set up is that this approach, um, is not limited to mobile electrocardiograms. So, um, those of you who, um, uh, who have Apple Watches, of course, know that there's increasing amounts of mobile technology built into your watch that can be fed in. But these electrocardiograms are not the only low-cost data collection devices that we have available to us. So, in the course of this study in Tamil Nadu, we also collected a number of other low-cost devices. So, um, those of you who sadly had to live through, um, uh, COVID here, no doubt had mobile, portable pulse oximeters. Those pulse oximeters, in addition to generating oxygen percentage numbers, generate a waveform that actually represents the blood flow through your finger that the pulse oximeter is, is measuring to get that fraction of oxygen. But that waveform has an enormous amount of information about how your heart is working, how your capillaries are working, how your blood vessels are allowing the flow of blood, um, through all of these channels. So that also can, can plug into algorithms. Uh, the gentleman in the front row is wearing an Oura ring or, or a related ring that can also measure these kinds of things. Continuous temperature monitoring is another feature of these rings. There are smartphone-based attachments that can take a picture of your retina. A fun fact about the retina: you got a lot of fun facts about the speakers from Audrey. A fun fact about the retina is that it is part of your central nervous system. It's, uh, the only exposed part of your central nervous system. And with a $200, um, smartphone attachment, we can actually get a view into the central nervous system that can be useful for diagnosing stroke, intracranial hemorrhage from trauma, uh, retinal disease, um, other kinds of diseases of the eye.
Okay. Um, thank you so much for getting this working, Sha. Um, would you mind, uh, telling us a little bit about what we're going to do?
Of course. Thanks so much, Zad. Uh, sorry this took a bit of time. Uh, good morning, everyone. We'd now like to demonstrate what this looks like in practice. So all a participant has to do is basically place their fingers on this very small device, uh, that's connected to my phone, as you see, um, and that's it. No wires, no hospitals, and just 30 seconds. Um, Zad, I hope you don't mind. Do you want to go there?
Sounds good. So what this device does is it records a single ECG, which is the electrical signal of your heart. And once we capture that, I enter the participant's ID onto, um, our app that we developed in-house. And behind the scenes, our AI model, it analyzes this waveform, the pattern of the heart, almost instantly, um, which it has learned to do from the thousands of real-life cases that we've collected from the field. And within seconds, you get a risk score that we'll show you just in a Um, so what this risk score tells us is, based on just, again, 30 seconds with this single ECG alone, how likely it is that this participant has had a silent heart attack in the past.
The, uh, the stress of the live demo is elevating my heart rate.
I hope it hasn't provoked a heart attack. That would be unfortunate.
Are there any other doctors in the audience?
All right, I think we're ready to go. So what you're seeing now is, um, the ECG was automatically uploaded into a web application and generating a, uh, low-risk score. Thank goodness for that. Um, thank you so much for. And thank you so much to our, uh, tech team here. Uh, live demos are an unpleasant, uh, ask for, um, for the tech team, and I, I really appreciate your help. Thank you.
Okay. Um, so as I mentioned, this is not just about ECGs. You could imagine a setup where a very small amount of hardware, totaling under $20,000 US dollars, comprised of an ECG, um, a retinal photo, a mobile chest X-ray unit, a handheld ultrasound device, can actually be put together in a district health center or even in a little kiosk in a, uh, shopping center, um, somewhere else, and could generate an enormous amount of raw device data that can feed into a number of diagnoses that are of critical importance to lots of causes of burden of disease in low and middle-income countries, but in high-income countries as well. So I think the, the scope and the scale of these kinds of diagnostic devices and approaches is really enormous. And I'll just mention one very personal application of this, uh, for me, to give you a sense of, of, of what this can do, um, in, in life. So, my, my wife and I have, uh, twin daughters who are almost two years old. And the way that we found out that we were having twin daughters is because I, um, as part of my practice in emergency medicine, I own a handheld ultrasound device. So, this costs $2,000, and it can generate, uh, pictures like this of our, uh, of our two future daughters. So, we knew that we were having twins, uh, well before most people know, um, because we are lucky enough to have access to this technology. And I think democratizing and broadening access to both the hardware and the software components is really incredibly important as we seek to diagnose illnesses across the world.
The last thing I'll say is that I think one of the really exciting parts of this work is that it represents a fundamentally new way that we can develop AI algorithms, um, in a, in a, in a sense that many people, I think, in, um, well-resourced settings find counterintuitive. Because this is an innovation that starts here in India. Why does it start here? Because there's very high volume of data at very low cost, but also because there is an enormous amount of human capital here, um, engineering and AI talent, people who know how to build hardware and and train AI products. There's the ability to move quickly and at the same time, and especially thanks to, um, the, the team at JPAL, there's scope for very rigorous testing. This kind of innovation can scale seamlessly to the rest of the world. One interesting fact about our algorithm is that we pre-trained our AI model on electrocardiogram data from Sweden and the US, and that actually helped a lot for, um, improving the model performance in India. And that interoperability, if you will, of the ECG waveform is a really exciting part of building algorithms that run on this kind of physiological data. Because as we've tested algorithms that are trained in one place, for example, in Sweden, they generalize incredibly well to the US, to India, to Taiwan, to wherever we've tested them, because we do electrocardiograms the same way in all of those places, and because the heart is fundamentally wired in the same physiological way in all of those places. And so those algorithms can generalize, and they can solve barriers to access that are, um, widespread, not just here in India, but all over the world, where people struggle to get access to the formal health system because of distance, because of poverty, because of, um, discrimination, because of cultural or linguistic barriers. So I'm very excited about that. And and I would urge anyone who's working on this kind of work here with data is to protect and steward this secret sauce that you have here in India, because it's really, uh, such a valuable resource to the, to, not just to India, but to the rest of the world, to be able to develop AI products in a way that is safe, ethical, but also rigorous and fast. Thank you very much.
Thank you so much, Zad, for that insightful presentation, and may your risk of heart attack remain low for many years. Please rejoin the stage for a group photo with the rest of the session speakers. It's my pleasure to invite to the stage the panelists for the AI and Health: Saving Lives at Scale session to join Zad on stage. The panel will be moderated by Amy Barnes. Amy, please join us on stage. Amy is a policy manager at JPAL Global, where she leads JPAL's health sector. With all the promising applications of AI in health, Amy has increasingly been pushing to increase the available evidence on the real-world effects of AI solutions on both providers and their patients. A fun fact is that Amy started her time at JPAL focused on policies to strengthen governance and public service delivery and to reduce crime and conflict. Now, please join me in welcoming our panelists, starting with Shahed Alam. Shahed is the co-founder and co-CEO of Nura Health. Since its founding in 2014, Nura Health has been transferring health skills to family caregivers and patients. So far, NURA has trained over 43 million caregivers and patients across India, Bangladesh, Indonesia, and Nepal. And now, Shahed is using AI to take NURA's mission to the next level. A fun fact about Shahed is that he earned his MD from Stanford. He's an associate faculty member at Ariadne Labs at Harvard Chan School of Public Health, and he's also been a TED speaker. Also on stage, Zamir Bray. Zamir is the deputy director of technology diffusion at the Gates Foundation. In this capacity, Zamir leads the foundation's strategic coordination on the equitable and safe use of AI, including work like the internal AI task force, AI-related grand challenges, and external AI advisory efforts. A fun fact about Zamir is that he trained as a medical doctor, and he also earned an MBA. And on top of that, he has a PhD in health systems and innovation and a Master of Law from the University of Cape Town, um, and he serves on the board of Groot Schuur Hospital in Cape Town. Finally, Rob Sherman is the vice president of policy and deputy chief privacy officer at Meta, where he works to set the governance and safeguards for how AI-powered products are designed and deployed responsibly. A fun fact about Rob is that before joining Meta, he was a lawyer advising major tech and digital tech companies on privacy and data security, and he was recognized by Chambers USA as one of the nation's leading media regulatory law lawyers. We have very great people on stage with us today, Amy. Over to you.
Thank you so much, Audrey. I'll just, the audience can send in questions by scanning this Menti code. And thank you, Amy.
Awesome. Um, first, just so excited to be here with all of you today, hear your thoughts around, you know, the rapid evolution of AI and health systems, how it can be integrated, your thoughts on evaluation, of course. Um, audience members, please do submit your questions. But I'll just, uh, take the moderator privilege and start diving in with one. Um, I want to start locally with Shahed. I know we're so far, but going to start with you and your organization, Nora Health, which does a lot of work, including in India, to train family members, caregivers, to advocate more for their loved ones in hospitals and to provide better care when they're leaving the hospitals to return home together.
So, Shahed, Nora is already working across multiple countries and already recognized for its impact. At this stage in Nora Health's growth, what problem are you trying to solve, and what's the role of AI in your solution moving forward?
Thanks so much for, for having us and and also for that question. Um, at Nora Health, again, we support the family caregiver of the patient with the knowledge, skills, and confidence they need to care for their loved ones. We work with, uh, governments and health systems in order to incorporate these programs that really support caregivers. The anchor of, of our work really starts in healthcare facilities when families are anxiously waiting, uh, by their loved one, and we provide them a skill-based training by a trusted health worker in person, um, to engage them in their, in their loved one's care. Um, now, once they go home, families, uh, may, you know, they'll face some sort of a challenge. They need, they may need a reminder or a nudge to remember what to do, uh, in order to care for their loved ones. And so that's where, um, for us, technology and our mobile messaging service steps in. So after an in-person training, to continue to receive nudges and reminders for families once they go home. Um, from the very beginning, as we've developed the service, we've, uh, not developed this as a chatbot because we wanted to always keep humans, that human connection between patients and families at home and a health worker at the back end, as, as a critical and core part of what we do. But as we've scaled it, we have seen that that whole process can become very constrained. We went from, um, uh, response times that were pretty quick to, uh, to response times that needed to take more than 10 hours.
So that's where, uh, AI really stepped in for us, um, to be able to address this challenge of keeping high-quality support to families once they're at home, but also ensuring that response times don't balloon to something like 10 hours. We had families who, who loved the service. They would reach out to us and they would, um, ask our nurses to, to name their newborn, but they were getting increasingly, uh, frustrated with how long it took. So we, uh, sat down with our health workers, and, and that's really where it begins, is to define what are the challenges that, that people are facing. And for us, those are health workers, and, uh, and started to look at, uh, a lot of manual, uh, tasks that they were doing, looking up things in an FAQ database, annotating things manually, um, and then with them alongside us, we started to co-develop an AI co-pilot that supports health workers in answering questions, providing them context on, on the patient and family that they're supporting, and importantly, also tracking operational and other data so that we understand the, the quality of the, of the responses and the system overall. Um, and with that in place, we've been able to reduce, uh, those 10 hours down to 30 minutes. Um, we've improved retention, so families, more than 30% of families keep coming back to us for, for more support. And, uh, importantly, the health workers are happier. They're getting to focus on what motivates them, which is to provide care, versus to fill out a bunch of forms on an Excel sheet.
And to follow up on that, I've heard rumors at least that you're planning a randomized evaluation later this year on this. Could you say more about why you're exploring an RCT now, given that you already have some evidence of Nora's impact?
You are correct. Um, and, uh, you know, for us, impact and evaluations is not a one-time thing. As we evolve, as, as the world changes, we need to continually update, you know, the evidence that we have and the knowledge that we have. Um, the RCT is really critical for us because while we've done evaluations on the model, some of the product metrics, how it's affecting users, ultimately, what we hold ourselves accountable to is having an impact on health outcomes, things like newborn morbidity and mortality. And the RCT is a way for us to actually, uh, in a more definitive way, answer that. So, in a simple, in a simple way, it is to understand the impact on this incredibly important outcome, which is newborn mortality, and to understand also incrementally what does the AI-supported tool provide, um, versus our, our standard program, which is the, the in-person initiative. Um, and hopefully, with, with that data, ultimately, what we want to do is to have family caregiver support be the standard of care across health systems globally. And what RCTs and, and this evidence that we're generating will allow us to do is to provide information on what works, what doesn't, to governments, so that they can use their scarce resources in supporting caregivers most effectively.
Thank you so much. Zamir, I want to bring you in next and hear from you from the perspective of a funder who's thinking about technology diffusion a lot in health systems around the world, because AI tools are evolving so rapidly. How should funders and implementers like Nora Health design evaluations that can keep pace with that reality while still producing evidence that's credible enough to drive adoption and policy?
Yeah, thanks, Amy. Um, I, I, I must just depart for a second and, um, I wonder if Zad is still around because when Zad made his admission about being a clinician that missed the diagnosis, I realized that's why I was invited to this panel. I too missed the diagnosis, and I was just remembering that it was actually my first trip to India 15 years ago when I came here, had a great time, um, and went back home and had a classic swinging fever for six days and completely missed that it was typhoid fever and two ended up in the surgery room with cholecystitis. Is, um, so to, to Zad, this is a bit of a clinician anonymous, uh, admission session, so thanks for creating a safe space. Um, but Amy, to, to get to your question and also, you know, just to pick up on where Shahed left off, I think the, the role of evaluations for decision-makers, for governments where resources are scarce, is absolutely critical. And I can already tell from the proceedings this week that this will be one of the key messages we take, take back, which is we need to double down on evaluation, um, particularly giving decision-makers the right kind of evidence at the right time. Um, I'm a big fan of RCTs. They have fundamentally changed the global health landscape, but there are two kind of issues we need to grapple with. The one is the models change very fast. And I see some colleagues smiling with me because this is a very real issue. Um, you know, and I was doing a little bit of homework on like, how long does this take? Takes us six months to design the RCT. Let's do the RCT for maybe 24 months, and then take 18 months to publish the data. That means that if we all left here today and we started designing, Shahed, maybe 2029, we have some good results that you can put out there, right? I'm not saying that that's what we should do. The other reality is that policymakers, like in India, like in many countries, uh, we work in as the Gates Foundation, are really hoping that AI brings new promise, brings new tools to frontline workers that it's never been able to do before. And so the policy windows, the policy cycles don't track to publication cycles. Policy decisions are often made because of resource constraints, or there's an outbreak, or there's some political economy, uh, issue that drives a decision. And so being receptive to that means that we're going to have to challenge ourselves on conventional models of doing RCTs or, uh, you know, long trial designs that shouldn't mean that we compromise the evidence generation. So, just to be clear, we have to have good evidence. But in the early phases, we can start thinking, does this give us a safety signal? Can we do something in short steps, in three to six months, that says this looks like this could be safe, could be interesting? Um, and very much to what the previous speaker said, maybe those are short, inexpensive, quick experiments that we can run. Then we can move into kind of pragmatic trials, adaptive trial designs that don't take us two or three years, but give us the kind of evidence around adoption. And this is really key. We've seen lots of models perform really well in labs and desktops. Um, they beat every single benchmark. And there's no doubt that most models today are better than clinicians. Those results don't hold up when you hit the field. Why? There are some fundamental issues. Uh, you know, one trial we saw was that the clinicians actually didn't trust the AI. So adoptions were stuck at about four, five, six percent until someone actually sat with the clinicians, went through their own clinical cases and their mistakes, and showed them that somebody in the room, there were 100 clinicians, uh, in a trial in Kenya, and one of them was making the mistakes, and those mistakes were significant and catastrophic, that they then sat around the table and said, "Hang on, there's something we need to embrace about the technology." And then we saw adoption surge to upwards of 60%. Still not at 100%. Um, so, so I think that how we think about evidence is, what's the right methodology for the right time? Um, and again, want to just applaud, you know, the, the Indian government for, and the India AI Mission for proactively recognizing that evaluation and evidence will need to be a key part of the AI strategy for the country, setting up these centers of excellence on health, education, agriculture, etc., and then bringing really strong partners, you know, like JPAL, who has the experience, but also locally, like ICMR, um, and, and many other folks who can kind of put our heads together, and I'm sure folks in the room, about what are the best ways to evaluate AI in a period where if we were stuck on one method that took us that long, we'd likely work ourselves into irrelevance and, and increase the, the inequality of how AI plays out.
Thank you so much for sharing that perspective, Zamir. Rob, I want to bring you in next, uh, from a global governance vantage point. As we're thinking about what's needed to really drive adoption and policy and impact, what are the biggest privacy and governance mistakes that you see when health AI projects are scaling, and how can programs avoid them without stalling innovation?
So, thanks for the question. I mean, I think it's really an exciting opportunity and an an exciting moment where we are sitting here where the technology that is being built is capable of solving lots of different problems. We've heard about a bunch of them from this, this panel, from the speakers already, from preventing heart attacks to diagnosing anemia to giving, um, personalized health guidance to people who otherwise wouldn't be served by the traditional healthcare system. So, these are really exciting opportunities. And I, I, before I answer the question about privacy and, and, and security, I want to say, um, you know, I, I was just sitting here listening to this conversation and thinking back to the past couple of global AI summits that we've had. Four years ago, um, the first one was in the UK, and I think I wouldn't have been able to say the same thing four years ago that we build general-purpose technology, um, like the kind of technology that my company builds, and then people use it for all of these compelling use cases that actually enable health delivery to scale and to enable the kind of health delivery that would previously have been limited to more wealthy nations, to, to more limited populations, um, is something that everyone can have. And I, I just think that's a really exciting opportunity right now. So, what are the big mistakes? Um, the first one is just treating governance as a gate rather than a guide. Um, I think very often you get into this mode where people say, "Hey, we're really worried about X issue or worried about privacy." I, I spent my career thinking about privacy. I worry about privacy a lot too. Um, but I think it's important to realize that the fact that we are worried about privacy doesn't mean that we shouldn't be able to use data in novel ways in order to deliver really positive health outcomes. We actually did a study, um, with Stanford University, um, on something called deliberative democracy. Basically looking at at broadly representative groups of people in the US and in India and asking them for their attitudes about about frontier AI. And one of the things that we found is here in India, 90 plus percent of people said that they want AI to be used for cutting-edge cures and that they're willing to contribute their data in privacy-preserving ways to have that happen. And so I think as a starting point, it's important to know that, um, as a starting point, it's important to know that if you're not, if you, if you're stopping the development or stopping the deployment because you're worried about those issues rather than trying to build in a way that is privacy-safe and that is secure, you're making a mistake. The other big thing, which I think we also heard, and I think you, you talked about earlier, is the idea of human in the loop. And, you know, the traditional understanding here is, is always humans are the gold standard. We should make sure that we have humans in the loop and making decisions at all times. And that has two consequences. You talked about mistakes, and you talked about people not getting access to care because there are bottlenecks. And so I've come to think about this problem, especially as we're getting to a point where the AI is potentially more capable than humans, faster than humans, and more reliable than humans, thinking about humans as the architect. We should be driving the way that the technology should be used and thinking about AI as a tool that can help us scale our impact so that we don't run into this case where, um, people who could be helped by the technology aren't receiving it.
And Shahed or Zamir, do you have any reactions to Rob's comments here?
Yeah, I, you know, it's, Rob, you, you remind me of a real encounter I had, um, speaking to a skeptic head of neurosurgery at a very big hospital. I won't mention the name. He looked at me and he said, "This AI thing is a complete waste of time and it's not going to change anything in my practice." And, you know, it was in South Africa. And he said, "The only reason I'll give you five minutes is 'cause you were my student at one point in time." And put me on the spot and said, "There's this, there's the CT scan. Now show me." And I was like, I didn't come to do a demo. I didn't have anything special in my pocket. I didn't have a trained model. You know, and I took out the general-purpose LLM. I won't say the name. Him. Took an image. It's multimodal. You know, which one, maybe. And I turned the phone around, and he looked me dead in the eye and he said, "Hang on. Actually, this is, this is crazy." Because he then told me the clinical case. This was a case, a very real case of a 12-year-old child who was admitted to a district hospital, who the clinician on duty was late in the morning. I, I don't know, looked at the child and said, "Child's got raised intracranial pressure of unknown cause." And for whatever reason, decided to do a lumbar puncture. Clinicians in the audience are gasping. You know what happened next? Unfortunately, the child coned. By the morning, the child did not survive. And when he looked at the LLM result, it said to him, very clearly, it was the first time I saw bold and capital letters, "Do not lumbar puncture whatever you do." That clinician turned around and said to me, "This is something I've never seen before." And just last week, he emailed me to say he submitted his first publication to the New England Journal of Medicine on the power of AI for African communities. And that's where I think we're going to get the change is when clinicians start to see the evidence and trust that this will make a change for their patients, uh, on, in a, kind of real-life situation.
Shahed, anything that you would briefly add in 30 seconds there?
Okay. Um, before we get to one last question, I just want to say that this discussion has been really exciting for me personally because while I can't say too much yet, JPAL is going to very soon be really massively expanding our work through a new initiative to support research on AI and health, specifically in low and middle-income countries. And that's going to be formally announced on the 20th, uh, at the summit. So stay tuned for additional details there. But I just want to flag that with that new initiative in mind, we would be really eager to reconnect with the researchers and implementers and policymakers in the room here today, um, after the announcement on the 20th to speak more about how we can, you know, build the evidence base on health and AI together, um, and make those new evaluations possible. So, with that in mind, as JPAL is looking to support even more research on AI and how to integrate promising tools into the health system, I want to hear from each of you one more time. So, in one to two minutes each, based on your experience, what differentiates AI applications that succeed from those that fail? That could be data availability, regulation, evaluation, other factors. And what advice would you give to implementers who want to build AI into programs iteratively while also staying grounded in learning what works? So I'll start with Rob first, maybe, and then Shahed and Zamir, feel free to bounce off of that.
Sure. So I think that, you know, we, I, I talked earlier about the, the downside risk of not deploying. And I think that, you know, by thinking about things in binary terms, you often run into this case where you leave opportunity on the table. And I think the projects that are successful are ones that have a dual focus. They are thinking, they're thinking about the way to the, the upside. Who are they, who are they helping? What is the specific mechanism by which they're going to deliver that? They're thinking about governance upfront. They're making sure that they're identifying the downside risks and they're building them in. So, as an example, when we do our data modeling, we look at anonymization technology. So, we take data and we build, we build the data sets in ways that protect the anonymity of the people that are in it. Um, so we don't have problems later on. Um, and then thinking about access to data. We, this was something that came up earlier too. Making sure that the data that's being used is broadly representative of the people that are, that we're trying to serve. I think if you build a project and you build a model based on data from people in the US, you're going to end up with something that serves people in the US and doesn't serve people in the rest of the world. And so I think thinking about both a very clear, targeted upside, a very clear mechanism of action, very clear governance for how to build it responsibly, and broadly representative data so that the technology and the models that you're building can serve as many people as possible. I think those are the key, key attributes.
Shahed, maybe over to you.
Sure. Yeah. And thank you. Building, building on that, um, maybe I'll share like three things, uh, for implementers who are thinking about incorporating AI. Um, first would be starting with people, not the technology. Um, for us, that meant health workers, understanding their needs, their challenges, their motivations as professionals, deeply, and then seeing how this tool can unlock potential. Um, again, for us, that was about unlocking moments that they provide care. Um, second, it was really important for us to get very precise with our definitions. What do we even mean by good care? Um, how, what, what's in the bounds of, of this service? What's out of the bounds of the service? Both of those things helped, uh, essentially provide a north star for things like when we developed out the knowledge base, built-in guardrails for our prompting, um, and again, do that in a way where we iteratively come back to it. Um, and finally, especially because we had humans in the loop, um, putting that, that product or that tool out there to, um, in a, in a small, uh, geography for us to learn and understand and, and then iterate on. Um, we learned so much more, uh, when we actually started to communicate with families and see what were the challenges that our health workers were facing, uh, when we put it out there. So again, um, of course, do that in a safe way. For us, it was keeping people in the loop, health workers specifically, and, and monitoring all the conversations, but what we learned was, um, was tremendous in those initial days.
Thanks, uh, Amy. You know, I think a couple of things. I mean, I, I think we're going to hear this week, you know, lots about models and the cool tricks they're going to do and, and so on. I think it's a 10% problem. I think models, their accuracy will be 10% of what we need going forward. I think there's a 20% on how does this actually integrate, like what sort of infrastructure do you need? And then I think 70% of this is going to come down to change management and how we engage people and take people along on the journey. Um, and I think if we, if we neglect that, I think this is going to be an uphill battle for, for some time. I also think that as we think about evidence and adoption, there's a very real cost issue. You know, one of the studies we did in partnership with Bath over here showed that the clinician assist consult actually only costs 4 US cents. And the accuracy was up to 97%. That may be expensive when you expand it over thousands, you know, of users, but actually the cost of compute will continue to come down. It's 240 times less than what it was two and a half years ago. So you can easily start to see that this becomes scalable and sustainable. I think that one of the biggest game changers, if I, if I was forced to bet, I don't bet, but if I was forced to bet on what I think could be the biggest game changer for development on AI, I think it's personalized health coaches, >> in the pocket of every single individual that can speak in their language, that has context and memory over time, that eventually doesn't only serve health needs, but is your coach for education, >> is your coach for financial, um, but just a coach that you can trust with decision-making, >> over time.
Thank you so much, Samir. That's a really exciting vision to lay out and I think a good way to close. It was so great hearing from each of you. Um, I think this was just a great way to highlight the need for more evidence, the excitement around AI and health, and just really appreciate you taking the time to be here and all of you for attending.
Thank you. Thank you so much to all of our AI and Health speakers. Um, it's really just amazing to hear these perspectives from every type of organization from all around the world. I hope you're all learning a lot. We'll take a short break and we'll rejoin at 11:40. Please try to exit from the right side of the stage. Thank you very much.