How AI is Shaping the Future of Education | Askwith Education Forum

To the first Asquith Education Forum of the 2024-25 academic year. The Asquith Education Forums is our premier event series at HGSE. It's an opportunity for us to come together in-person, as a community, to discuss, debate, and learn about the most pressing questions, and challenges, and opportunities facing the field of education.

And there's little doubt that the rise of generative artificial intelligence is a worthy topic for our first session. And it appears you all agree. Few, if any, technological innovations have captured popular attention and imagination in the same way as these tools have over the past two years. The pace of adoption has been nothing short of remarkable, and the field of education has been no exception. In fact, the first and only time that monthly visits to the ChatGPT website fell was in June 2023, just at the end of the '22-'23 school year.

As of May 2024, 82% of US undergraduates and 72% of K-12 students reported having used AI in their school work. Half of each group reported using it several times or more a week. And, of course, some of us react to those numbers with a sense of alarm. We worry about students cheating, being exposed to biased or hallucinated content, or simply using AI tools as a substitute for the kind of deep thought that's so essential for learning.

Others of us acknowledged those risks, but look past them to the opportunities that AI presents to personalize instruction and to support both students and educators in new ways. What's clear to all, though, is that AI is already shaping the future of education in ways that everyone involved in the sector-- from policymakers and leaders to teachers, students, and parents-- need to understand.

And, of course, technologies don't shape the future of education on their own. Rather, their impact is mediated by how humans in various roles respond. So tonight's event is an opportunity to hear the perspectives of individuals who are not only observers, but active participants in that process of shaping AI's impact in their roles as policymakers, as tool developers, and as researchers. So let me introduce you to my conversation partners for today.

First, we'll hear from Roberto Rodriguez, who's Assistant Secretary for Planning, Evaluation, and Policy Development at the US Department of Education. Roberto advises the Secretary of Education on all matters related to policy development, implementation and review. His portfolio includes the Office of Education Technology, which, this summer, released new guidance for edtech developers on designing for education with AI. I'm also very happy to say that Roberto is a proud and an exceptionally loyal alum of the GSE.

Next up will be Marta McAllister, who describes herself in her LinkedIn profile as "a former teacher turned Googler who drives the voice of educators in product and program development." Marta began her career as a teacher in New York City and now, in addition to other enterprise-wide responsibilities, manages teams building Google's teaching and learning products and bringing them to market.

And finally, we'll hear from our own Ying Xu, Assistant Professor of Education and one of our newest faculty members here at the Harvard Graduate School of Education. Ying's research focuses on designing technologies that promote language and literacy development and STEM learning. Please join me in welcoming all three of them to HGSE. [APPLAUSE]

Each of them will spend a few minutes talking from the podium and then join me on stage for a conversation. And we want you all to be part of that conversation, as well. Some of you may already have index cards, and we have volunteers who will be handing out more index cards as our conversation proceeds. If you have a question you'd like to pose, please write it on one of those cards. We'll be gathering them, compiling them, and getting just as many of them into the conversation as time allows. So look forward to that. And for now, please join me in welcoming Assistant Secretary Rodriguez to the stage. [APPLAUSE]

ROBERTO RODRIGUEZ: Hi. Good afternoon. It's wonderful to be here. I want to thank Dean West for the opportunity to lend my perspective and our voice, from a federal policymaking perspective, to this really important conversation around gen AI and the future of our education system, and teaching, and learning. And I'm just going to begin here, hopefully. Let me see. Ooh. Here we go.

Let me just begin by grounding us a little bit in some of the work that we've done here at the federal level to examine AI. And you already heard Dean West note just around the rise and the propensity in use of AI in our classrooms and in our schools. Our department really began this work in 2021, informed by a landscape analysis, and with conversations with over 700 educators and stakeholders around the country to really begin to think more about how we support and experience AI in teaching and learning.

AI is becoming more commonplace, more advanced, more available in the lives and learning of our students. And at the Department of Education, we want to be sure that we can help provide the guidance, the support, and foster a conversation with our field, with our stakeholders, in our elementary and secondary schools, in our higher education community, in our early childhood programs around how they think about AI. I have the privilege and opportunity to work on a whole host of policy issues for the Biden-Harris administration and education. This is one that I think I am so excited about because I really think it has the potential to address so many of the opportunity-challenges, so many of the equity gaps that we have in our system, and really help us to go further, faster as a country in AI.

So I'm going to touch on a few recommendations just to ground us from our report in May of 2023. And first, note that this fits into the broader work that President Biden launched in the fall of last year through an executive order on the safe, secure, and trustworthy development and use of artificial intelligence. That really introduced an all-of-government approach, where every agency across our federal government is working to address AI, to think about its potential in its use, both within the government and within those agencies, as well as with the programs and the constituencies that they serve. That certainly is the case for us at the US Department of Education, as well.

And the President really made a proclamation that-- was it to the order of-- in order to harness AI for good and realize its myriad benefits, we also have to mitigate its substantial risks. The report that we released, AI in the future of teaching and learning, also recognizes the tremendous potential of AI to really help to foster new, and dynamic, and innovative settings for teaching and learning to be a tool for and in the hands of our educators, to help support student engagement, a stronger feedback loop, the opportunity for formative and adaptive uses for instruction and for advancement. But it also recognizes the substantial risks.

The first notion here that I just want to really emphasize is that we really reject the notion of AI as a replacement for teachers and as a replacement for educators. And so this notion of humans in the loop is really a notion of making sure that we're equipping our educators with the capacity that they need to make important judgments, to lead pedagogically, to make sure that they're helping to support the determinations and the cases when AI is used, that they know when AI is present, and that they're able to really help inform those decisions around AI. And humans in the loop has to be, first and foremost, to that end.

We want to align AI models to a shared vision of education. What that means is that too often we start with a question of what can I do, rather than beginning with a question of what is our vision for education, what is the problem that we're trying to solve in our classroom, in our school, or for our students with respect to their learning, and how can AI help to-- help us to solve that problem? How can AI be a tool to that end?

We want to make sure that AI is designed using modern learning principles. We know more than ever before about how our students learn, how they engage. We know more about the science of learning and the social and emotional aspects that are so importantly connected to their engagement in academic content. We need to make sure that we're applying those modern learning principles, as well as our principles around data science and data use, as we think about AI.

We want to prioritize strengthening trust. And I'll return to this point, but it's one of the most important fundamental points as we think about AI, which is, how do we engender and support, trust, and make sure that our educators, and those that are leading our-- leading education trust the tools that they're using?

We want to inform and involve educators at every turn. We know that if AI is going to reach its potential in our education system, our educators have to be in the driver's seat to that end. I'm really excited about the recommendation on focusing R&D to improving AI and in addressing context, because we know that we have to be asking questions about what's working for whom and when, and making sure that we're focusing those R&D questions in the development of new tools, which I know we'll talk about shortly as we think about using AI.

And then, finally, we want to develop the guidelines and guardrails that are needed to make sure that AI is being used in a safe and an effective way, and in a way that maximizes student privacy. I'm going to just briefly show you this framework that is behind the developer guide that Dean West referenced and that we recently released, and that's part of our charge to address emerging technologies in the field and our Office of Educational Technology. We really look to make sure that we can bring our community, our edtech community, and our developer community together with educators to better think about the design principles around using, and testing, and developing, and, ultimately, scaling AI.

And there are three core components here-- providing evidence for rationale and impact, ensuring safety and security, and advancing equity and protecting civil rights, which really gets to how we think about bias and how bias plays out within AI and AI tools. All three of those components, we believe, are really critical for engendering trust and promoting transparency. Again, we want to make sure that our educators know about the tools that they're using, that they have-- that they can be assured of their safety, their security, the student privacy questions, their impact on equity, their implications for bias, and their efficacy. Do they work, and how do we know they work? How are they aligned? To what evidence principles?

And then, finally, I'll just cover here the dual stack, which we believe is a powerful graph to really think about how we coordinate the work around innovation. On one side of that stack, you have more of an innovative framework to think about product, and service development, and how we think about delivering new products and new services to market. And we encourage, on the other side of that equation, organizations to define a parallel stack that has a coordinated set of responsibilities. So again, how do we think about safety and security? How do we think about transparency, bias mitigation, and representation of all involved-- our students, our educators, our parents-- while we also think about the more innovative development and deployment of AI tools? I really look forward to our continued conversation this afternoon. And thank you again. [APPLAUSE]

MARTA MCALLISTER: OK, cool. Hello, everybody. My name is Marta McAllister. By the volume of messages I've gotten on LinkedIn, enough of you probably stalked me to know who I am. But I am a former educator, as was mentioned earlier, and I'm really honored to be here. I'm so grateful that you extended this invitation.

I'm going to start with a quick story. So this is actually a photo of my colleague, Carly's, son. He's about eight years old. And I wish I could show a picture of my own kids, but I don't do nearly anything this cool with my own kids. He's really obsessed with cars, and his dream, one day, is to build the fastest car ever. But as you might notice from this photo, there's a bit of a problem. This thing doesn't go very fast.

And so we actually-- his parents, being cool, built this with him. But then they also asked him, if you could build your dream car, what would it look like? And he said, a supercar with a drag coefficient of 0.22, a spoiler, gull-wing doors, and Minecraft creeper livery. [LAUGHTER] If you are a car person, you probably know what some of that means. I am not, which is why I put it on a slide. And usually the conversation would have stopped there, because how do Carly's parents actually build anything that meets that criteria?

But at the time, thankfully, Gemini had just recently come out. We actually put that prompt into Google Slides. And not only did we get a visualization of the car with the criteria, but a whole Power-- like a whole presentation about that car, which is kind of amazing. But as you might tell, the story is really not about the car. What I like about this is it shows a pretty cool potential of what I can do, which is to help visualize our dreams. And as some of you might really appreciate, often, visualizing your dreams is the first step to being able to make that a reality. And having observed tons of classrooms, it's amazing to see a lot of the various ways that people are using gen AI. Of course, there are concerns around that as well. But I've seen so many amazing a-ha moments for kids that usually had a conversation stop. And I know there's a lot of paranoia about, are we going to lose creative thought? But I thought this was a great story that sort of emphasizes the opposite.

OK, I can't resist a bit of interaction-- former teacher, never goes away. So I'm going to quiz the audience a little bit. Don't worry if you get it wrong. How many years do you think it took to get 50 million users to ride on an airplane? I heard 30. Anybody else? I promise I'm not judging you. I know it feels like a lecture hall, but there's no grade here. OK. 50. 80. 68 years to get 50 million users-- people to ride an airplane. What about to get 50 million on the internet? 12? 12 what? Years? 12 years? Anyone else? AUDIENCE: Two. MARTA MCALLISTER: Two years? Seven years. How many-- How long did it take to get 50 million users on Facebook? AUDIENCE: [INAUDIBLE] MARTA MCALLISTER: An hour and a half. [LAUGHTER] Somebody said five years. I heard one year. Three years to get to 50 million. How long did it take to get 50 million users to use generative AI tools? Two months? Two weeks? Five weeks. Yeah.

So this is obviously a change happening. The adoption curve is very fast on this, which is part of the reason why I think people are freaking out, because the adoption is really outpacing our ability to make sense of it. Obviously, a lot of people see this as a really strong tech enabler, particularly at technology leaders, but also education leaders. It was 73% of the people surveyed in this survey that's linked here-- I can show you it afterwards, if people are interested-- said that they see gen AI as the number one tech enabler for this year.

That's all happening in the backdrop of teacher burnout and educator shortages, right? Post-COVID burnout is very real. There's an average-- teachers work an average of 53 hours a week, and they're feeling burnt out. Schools are having a really hard time attracting and retaining educators. And at the same time, we're seeing a lot of people report that when they use generative AI for the administrative burden layer of their job, they save anywhere from 5 to 10 hours a week. In fact, a lot of the people piloting solutions that we've been working with have cited up to 13 hours a week of saving time. So obviously, a lot of people are hopeful that some of this can actually help with the teacher burnout or hours worked per week, and focusing more on that versus replacing teachers.

The other thing that is obviously very common, too, is this emphasis on AI, digital literacy, where we want to make sure that this next generation-- I'm sure many of you in this room want to be equipped with how to use gen AI productively, and to be prepared for this next wave of workforce. And so just a couple more stats here, where 75% of faculty are using gen AI tools believe that graduates will need to know how to use these tools effectively in order to succeed in a professional setting. And 65% of University students want training in AI tools. They feel like their schools should actually be promoting more of these tools so that they can learn how to use them responsibly and productively.

Obviously, it's not all roses. We really acknowledge that with every side of the coin, there's an opposite tension. For all this democratization of information, you've got the flip side of concerns around privacy. For all of this creative opportunity, you have the fears that we're going to turn into the blobs from Wall-E. This is definitely not a cut-and-dry space, and there are tons of concerns around content safety, cheating and plagiarism, information accuracy, privacy, bias, and, most importantly, this idea that we're going to lose critical thought. The effort in learning is the point. And if we lose the effort part, what does that mean for how we learn?

I will share more, probably, in the Q&A on this. But one of the things that-- Google actually was one of the first companies back in 2017 and 2018 where we established AI principles to guide the company. We have a learning layer on top of principles that guide our work. I really appreciated how Rodrigo mentioned humans in the loop. Teachers in the loop is one of our guiding principles, just as an example. And so these are available for you to read externally, if you'd like. But we do have principles to guide us.

And I also wanted to share that on the education team-- because, obviously, Google is a big company. There are a lot of teams that build things not for education and learning. We specifically have worked with learning science experts and pedagogical experts to develop LearnLM, which is a family of models actually fine-tuned to learning science principles. And these are the learning science principles that we ground in. So this is both to inform product development and our approach to how we build and make product decisions. So when we're making a feature about student visibility in something like Google Classroom, we are intentionally making that choice because we know that it supports metacognition, which learning science suggests that increases students' ability to retain and learn something, just as an example. So we are really proud of the fact that we've been able to take generic large language models and actually have a fine-tuned set of models that are directly trained on learning.

And I share this because, obviously, there are a lot of tools at Google, like Google Search, whose main goal is to get you an answer. But obviously, if you're trying to learn something, that's the antithesis of learning, right? And so how do we actually have active learning as part of that, and what are the thoughtful guardrails and speed bumps we can put in the product experience that doesn't just get you the direct answer? So that's just another example of how we apply those learning science principles in product design. So I'm just going to emphasize again, we are not in the business of replacing people. Our hope for AI is that it actually elevates the educator and expands our capacity. Obviously, I think we're a factor-- one variable in a very large experiment in education. And there's-- we play a very small and humble role in that. But I want to make super clear that I don't believe that gen AI can replace the value and soul of a classroom, which, to me, is really the educator and the peers that learn from each other. Usually this is when I make a joke about hopping in my sweet ride and getting out of here, but I'm going to stick around for Q&A. And thank you all so much for your time. [APPLAUSE]

YING XU: Good afternoon, everyone. It's my first month at HGSE, and I'm already amazed by the forward-looking mindset the school adopts to approach AI. And I-- it's such an honor to be here to exchange ideas with all of you. So I do research on the educational impact of AI-- in particular, generative AI, which happens to be quite a hot topic at this moment. But the reason why I started this research actually comes from my belief in how much we could learn through conversations and dialogue, just what we're doing here right now-- listening, talking, asking questions, and exchanging ideas-- and probably like many of you have been doing with ChatGPT.

So the importance of conversation is what actually led me to focus my research on using AI to promote the kind of meaningful dialogue that could support learning. So my research revolves around two core areas. The first one is how AI could impact student learning, and the second one is how AI could impact the broader learning ecosystem. So we could start from the student learning part.

Most of my research has been conducted through partnership with public media, and I collaborated with PBS Kids to develop interactive television shows that allow kids to interact with a TV show's main character when they watch STEM-related programs. So this kind of dialogue could-- is designed to prime children to engage in observations, predictions, and problem solving. They're also designed to be a fun experience for the kids. And here, you can see on the screen, are two examples of the PBS Kids show we collaborated on, and you can see the kind of questions the characters ask the kids. So the characters will listen to children's responses, and either give direct feedback or ask follow-up questions based on what the child answers.

So just think about the average child spends almost two hours every day watching TV, and also think about how public media could be accessible learning resources for children, particularly those from less privileged households. And what if we could use AI to turn some of the screen time into active STEM learning experiences? That could have huge impact on their growth and development. So we have actually been carrying out studies to test if this kind of interactive television shows could indeed help children learn. One consistent finding is that engaging in dialogue with the media character lead kids to better comprehend science concepts and be more motivated to think about the science problems compared to the kids who watch the regular broadcast version of the television shows without this AI-assisted dialogue.

So beyond the young audience targeted by PBS Kids, I also partnered with other organizations, like WGBH, that serves older students like high schoolers. And through my studies with students from various age groups-- So we've consistently see the additional benefits AI-assisted dialogue offers to the students. And this kind of reaffirms how important conversation is, and also showcase one of the many possibilities how AI could enhance education through supporting students' individual learning.

Well, of course, there are always important caveats when being so optimistic. So the question of who has access, who benefits, and who doesn't benefit from it always exists. Let me just give you one example. In our studies, since the younger students have not fully developed their reading and writing proficiency yet-- so we leveraged automatic speech recognition so that students could talk to the main character instead of type, like how you interact with ChatGPT. But what we found was the speech recognition accuracy for bilingual Latino students are actually much lower than the typical results for monolingual students from the other studies. So if the AI cannot accurately interpret the child's speech, they cannot provide a targeted scaffolding children need. And those are the missed opportunities to use AI that could have truly benefited those students.

At this point, it seems that I've been shifting between being optimistic and being cautious about the challenges. So how do we reconcile this? So let's take a broader look at where and how learning happens. So students are part of the learning ecosystems that involve educators, parents, different resources, and environments, and policies, and culture. And all of these ecosystem elements mediate, either for good or for bad, the impact AI has on students' learning. So if you think about how AI might impact the future of education, it is actually very important for us to consider how AI interacts with each of those key elements.

So I want to highlight the stakeholders in this presentation, especially our educators. So if AI is, indeed, an effective educational tool, how will it influence the role educators play in the future of education? So from my own studies, there are two perspectives. I think they are very consistent with what the two previous speakers shared. One is that AI cannot replace educators. And second, AI could actually be used to support educators.

On the first point, in some of my studies I compare students learning from an AI tutor as compared to a human tutor. Although we found that the learning benefits in some domains are quite comparable across the two conditions, but there are also many other aspects of learning AI cannot fully support yet. If we look in deeper, to look at how students actually interact and engage with the AI tutor, we actually found that when students interacted with a human tutor, there are more student-driven and inquiry-based conversations that are happening as compared to the interactions with an AI tutor. And in my studies, along with many other studies, experienced educators are much better at identifying students' misconceptions and provide tailored feedback than AI. So all of those empowerment can have a significant impact on students' long term learning trajectory.

Second, AI is not just a tool for students, but it's also a tool for our educators. In the past few years, I have been conducting research looking at how AI can support teachers in lesson planning and classroom instruction. In all of these projects, we embrace the teacher-AI collaborative approach, which is consistent with the teacher in the loop model, where teachers could customize the questions and feedback that AI chatbot asks the students in order to reach their instructional goals. So we're carrying out studies to examine how teachers feel about their collaboration with AI, teachers' actual practices when they engage in this co-creation of content with AI, and how all of those factors impact student learning.

So now I wanted to take us back to the big picture again. Just think about the public introduction of large language models happened right after the pandemic. So this is a time when our education system had just lived through a grand experiment of rapid technology adoption. So there were actually many lessons learned, and a lot of those lessons are still relevant to our discussions with AI and also some of the evidence I just shared, which is neither AI nor other technologies could replace classrooms, educators, and the human connections that really cultivate students' learning and development. So the real question is not who or what AI can replace, but how we could leverage all the resources we have. So we need to find a way to use AI to amplify the benefits each element in this learning ecosystem brings to students. I look forward to more discussions on this. [APPLAUSE]

MARTIN WEST: All right. Thanks to each of you for fascinating presentations. I'm really looking forward to the opportunity to dive in. Roberto, starting with you, the US Department of Education doesn't issue multiple rounds of guidance on every new technology that emerges. What made AI different that you would choose to invest the level of scrutiny? I guess you got an executive order telling you to do it, but you did it with great enthusiasm. Was it the scale of the promise it offers, or maybe the potential threats that it poses, or both?

ROBERTO RODRIGUEZ: I think the motivation there really was both, Marty. And I'll say, we got going, as I noted on AI in the future of teaching and learning is a piece of guidance in 2021-- actually, long before the White House executive order. And in that-- even in that time, as we were putting the report together, ChatGPT showed up on the scene. And this technology is so powerful and is evolving so quickly. And we were really working to make sure we had something that we could really provide the field to help better understand, and distill, and think about some of the questions.

One of our aspirations behind the report-- and we'll be putting forth, I should note, a new School Leaders Toolkit, which will be a further iteration of this guidance specifically for principals and for school superintendents to think about how to apply some of those recommendations that I shared earlier. But we were so-- We really wanted to move that forward because AI was just so present in the lives and learning of our students so quickly, in schools and out of school. And one of the things that we love every district to take up is a policy around AI-- not a one-size-fits-all federal mandate or directive. You're not going to see that coming from our department, but you will see guidance and encouragement to really wrestle with some of the questions that we unpacked earlier-- the promise and the potential of AI, as we think about personalizing and individualizing instruction. We have a huge academic learning curve that we're trying to still climb up, in terms of academic recovery. We have a student engagement challenge still in our schools that we think AI has interesting applications to address. The load that our teachers are carrying, that has already been noted earlier-- potentially. And then, as we think about our neurodiverse learners and our English learners, these are populations also that have real potential for AI to reach.

MARTIN WEST: Great. Well, so let's keep talking about that potential. I'm obliged to ask a skeptical question. Marta, you're going to get the skeptical question as the representative of the [INAUDIBLE] In 1922, I believe it was, Thomas Edison declared in his confidence that "The motion picture is destined to revolutionize our educational system," and that, "In a few years, it will supplant largely, if not entirely, the use of textbooks." So we have this history of predictions from people working in the tech industry of their day-- that TV, the overhead projector-- what are some of the others-- oh, the internet-- that all of these would, in short order, revolutionize American classrooms, classrooms around the world. Given that history, is it possible that we're exaggerating the implications of generative AI?

MARTA MCALLISTER: Yeah. So in general, I feel that any time there's new tech, whether it's built-- purpose built for education, but especially when it's not, educators are going to grapple with, how do we integrate this into instruction, and teaching, and learning? There's actually an MIT event where Mitch Resnick and Justin Reich were having a really interesting conversation about this very topic, and it really resonated with me. Mitch's key point was, start-- you have to actually start first with what are you trying to achieve with the instruction, and not starting from what does the tech do. And that really resonated with me.

And the thing that Justin really emphasized, both in that talk, but also in his book-- how many of you guys have read Failure to Disrupt? Yeah, it's a good one. And I don't get commission from Justin. Although, Justin, I know you're on sabbatical. If you watch this, I'll happily take a cut. But one of the key points he makes in that book, and in a lot of his talks, is that learning is fundamentally social. No tech solution is going to be the silver bullet because systems only change when you are actually doing multiple things-- engaging with the different factors in that system, working with the various players in that system. And so I think we'd be fools to think that AI in its own is going to revolutionize teaching and learning because, like many of the other tools that have come before it, they're only as good and as impactful as they are being used by the people that have them in their hands.

That said, I do think that many of those tech innovations that you mentioned have had positive iterations and moved things forward. While I don't think they've totally revolutionized the system, I think they've made contributions. And so I think AI will make a significant contribution, especially when it's used thoughtfully and when tech providers are partnering with the people in the system and designing the tech that they're building.

MARTIN WEST: I hear echoes of Ying's presentation in that response. MARTA MCALLISTER: --said resonated.

MARTIN WEST: So, Ying, let me turn to you for a moment and see if you can help us understand exactly the moment that we're in. Your research on the implications of AI for education actually predated the launch of ChatGPT and the public understanding of large language models. So that means that AI and large language model must not be synonymous, right? There's different versions of AI. Can you help us understand how the new tools that are available fit into the broader landscape, and then the new questions that they provide for you as a researcher?

YING XU: Yeah, thank you. This is a great question. So the short answer is, yes, so AI actually existed way before large language models. And even now, AI is actually way more than large language models. And the longer answer is, so the scientific community has long been exploring how to build machines that could simulate human intelligence. So there were actually a lot of AI breakthroughs. For example, I'm not sure how many of you remember Alpha-- I think it was AlphaGo, and defeated a human player on a board game. So that made the huge news headline at that time.

But the difference of LLMs is that it has sustained public attention and interest. It has been two years, and we're still talking about it. I think the difference is probably because LLM could do a lot. It could help us with a lot of tasks. That is very relevant to our everyday life. And the second is accessibility. So many models have a chat interface that the end users could directly engage with AI without any technical expertise. So I think one very interesting research question that would emerge from this broad access is just asking, how does this broad access to AI tools would actually change the landscape of education? Just like how we ask students' access to cell phone and internet, whether those access would change education.

But LLMs are general technologies. They are not designed for education, at least from the very beginning. So that leads to another very interesting and important research direction, which is to think about how we could adapt those general purpose technology for education use, making them more specialized and pedagogically informed. So there are a lot of efforts-- research efforts investing in this direction. So for example, we could teach large language models evidence-based instructional practices. So there are researchers-- there are research projects teaching LLMs the Next Generation Science Standards so that it could engage students in science-oriented dialogues based on those benchmarks.

But the next issue is, no matter how much guardrails we put on AI, we can't perfectly control AI's behaviors. So there are times AI could still provide inaccurate information or biased information. So that comes to another very important research area, which is AI literacy. So on the one hand, we should improve LLM and improve AI. But on the other hand, we should also educate our students to be more critical consumers of this technology or critical players in this landscape, and so that they can maintain a healthy level of skepticism, and they could actively engage in detecting errors and then interrogating-- some of the biases and misinformation AI might distribute.

MARTIN WEST: And, Marta, I heard, I think, in your description of LearnLM, an example of what Ying was talking about. MARTA MCALLISTER: Right, exactly. Yeah, we worked with learning scientists to actually fine tune the general models that were not purpose-built for education, to ground those in learning science principles that we've already been using to ground our product design process because of that. We found it to be challenging to rely on general models.

MARTIN WEST: So let's go back to another issue that you just raised, Ying, which is the risk of bias. And, Roberto, one of the aspects of the guidance for developers that caught my eye is how you linked this issue of bias in the data sets on which large language models are trained to the department's role in protecting student civil rights. Can you help us understand that connection and its implications for how we regulate AI use in schools going forward?

ROBERTO RODRIGUEZ: Yeah, I mean, I think we-- the bigger principle here is that we do have to examine and interrogate closely the development and the presence of bias in the tools. And I think mitigating bias has to be a first-order commitment that actually begins at the product level, and begins at the development level, and carries forth both in terms of how we think about utilization of technologies and LLMs in this broader process, and then how those things play out, how they're used in the classroom. There certainly is, I think, potential for manifestation of bias or of inequities in the use of any tools, of any technologies. So AI is no different in that respect. But because of its power, and potential, and also because of the lack of AI literacy, which is a wonderful point that Ying raised, we need to be ever-vigilant on that point.

So this is-- our agency has joined several other federal agencies in our Office of Civil Rights in affirming and reaffirming our commitment to enforcing all civil rights laws and making sure that there's equal access and non-discrimination in how we carry forth education. We have heard from folks about the danger and the worry of bias in AI. And so I think we're trying to engage a conversation right now with the developer community on that front. You saw that precedent in our developer guide. And we also have-- we are not a direct evaluator of models, nor do we endorse or prescribe any one approach or curriculum at the US Department of Ed. But we want to make sure that our districts, our systems are thinking about this, that they're asking the right questions as they think about utilizing tech tools, including AI, and that they have those conversations directly with their leaders and with the developers, around how to make sure that those products are not perpetuating bias or perpetuating discriminatory actions.

MARTIN WEST: So could you imagine the department at the Office of Civil Rights taking enforcement action against a district that's adopted a tool that you view as harmful?

ROBERTO RODRIGUEZ: I cannot forecast or opine on what types of actions would come out of our Office of Civil Rights. From an enforcement perspective, I will say that there are-- there is a whole process, and we will continue to maintain-- be vigilant to that end. But I do think that districts need to be asking this question, and we need to be, the bigger point here is, more intentional from a design perspective early.

MARTIN WEST: To some extent, you're trying to get ahead of it by engaging with the developers now.

ROBERTO RODRIGUEZ: And asking them to also not just think about it in how they're developing their tools, but in also following those tools through and continuing to evaluate how they're being used so that they can return to correct that, if that's a problem.

MARTIN WEST: Marta, how does Google deal with the challenge of bias in its algorithms and training data?

MARTA MCALLISTER: yeah, so as I mentioned earlier in the preso, we do have company-wide principles. And then, on the education side, because we're building solutions specifically for schools, we have applied learning layer to that. I'd say a lot of those principles guide us in a certain direction, but of course it's not a perfect thing. We also have a lot of educators, learning scientists, experts that we work with for what we call our eval process. So as you might have experienced testing gen AI tools, sometimes they do very weird things. And it's helpful to have people play around with it and catch what those things are. We can't possibly find everything ourselves. And so a big part of our product rollout process is also piloting and working with educators as an extension of our product team. Our pilot program, which we lovingly call School Food, because Google has this program called Dog Food, where Googlers test products before they're released, we created School Food to basically work with schools who are interested in helping shaped the development of technology to partner with us and give us their honest and open feedback. And that's been pretty pivotal to how we've launched all of our tools, but particularly with gen AI because they've been giving us so much valuable feedback along the way. But obviously, even at that-- even with all of that pre-work, and all of the work we've done with LearnLM, you still can't totally control what goes out in the wild. And so that's why we also have a lot of the eval type process. You'll see, I'm sure, in many of the tools you may have tested, the thumbs up, thumbs down, give us feedback on the response that was given to you. That's all critical information for us to improve what's actually being served out to you, whether it's for accuracy or bias. And so those are guardrails that we put in place, both in the product design process, but also when the product is out in the wild on the feedback loop that we have.

MARTIN WEST: And, Ying is this something that you worried about in your collaborations with PBS Kids or WGBH?

YING XU: Yeah, I can't represent my media collaborators. I could talk about some of the considerations and measures we have taken when we engage in this kind of research. Yeah, so we are very cautious about the AI safety issue, and in particular for products and research targeted at younger students, especially preschool and kindergarten students. And for my collaboration with PBS Kids, we actually did not use or expose any AI-generated content directly to children. So all the conversation and dialogue is pre-developed by the research team, and by the PBS, and by the content developer. So when the students engage in conversations in the TV show and in our product, they follow a specific dialogue tree. So what the AI does is to identify which branch the student's response falls into and then advances the conversation accordingly. So, of course, there are trade offs. The trade offs are you spend a lot of time to develop this big dialogue tree because we want to capture as many student responses as possible. And there will be some responses we miss because we can't predict everything. But this is the only way, so far, we could maintain full oversight of the content students are exposed to. And also, we have time to actually really craft the dialogue very carefully so that-- to make sure it's educational value.

MARTIN WEST: I really like that example because sometimes I think we think the alternative is not using AI at all or let ChatGPT rip, right? And you're showing that there's sort of a middle ground where you're taking advantage of some of the potential of the technology, but also creating a safe, and secure, and pedagogically-sound environment. MARTA MCALLISTER: And what your answer actually reminded me of, too, is one of our principles is this teacher in the loop. I forgot to mention that all of the features that we've released for education purpose-built tools, we don't surface things directly to students. We have a speed bump, so to speak, where the teacher can evaluate if that's what they want to be shared. Obviously, I can't speak for a lot of the consumer products that Google also has. But for the ones that are built for education, that's another way we mitigate that.

MARTIN WEST: So I wanted to-- Marta, to follow up on that a little bit. We're just now starting to see some of the first wave of research on how access to generative AI is affecting students who are using it. There was an experimental study that came out from the University of Pennsylvania a couple of weeks ago that showed-- it was in a high school in Turkey. But it showed that students who were permitted to use ChatGPT for their homework, they earned higher grades on those assignments, but performed worse on the final exam at the end of the term than students who did not have access to AI. And so that suggests to me that there may be some tensions between usage of generative AI for at least some forms of schoolwork and how much students learn. So as you all are thinking about designing products, how do you try to steer students towards usage that enhances their learning, rather than hinders it? And, perhaps more pointedly, why should we, as the education community, trust Google, a firm that I think we all think is trying to maximize usage, to really put learning over usage?

MARTA MCALLISTER: Totally. I think it's a fair question. I promise I will answer it. But one quick thing I want to point out about that study, because I happened to read that-- Ethan Mollick recently shared an article. Many of you are nodding. You probably read it. MARTIN WEST: Are you going to talk about "the third condition?" MARTA MCALLISTER: No, no, no. I liked his concept about illusory learning and all of that. But that same study, the one in Turkey, I think it's important, asterisk, that the people that actually-- they actually showed that when it was unprompted and open-ended use of ChatGPT, that was the result. But when they had a guided tutor prompt, they actually saw the same, if not higher, assessment scores. MARTIN WEST: Yeah, that was the third condition in the study. I conveniently left that one out. MARTA MCALLISTER: Sure. [LAUGHTER] He also-- in that same article, he pointed out a Stanford-- there was a massive programming course that actually saw an increase in final

grades where they use structured gen AI. So my point in saying that is not, oh, gen AI is great, or there's no problems with it. It is to say, I think structure and how you use this tech is particularly important.

The other caveat, before I get to the meat of my answer, I'll call out is you can remember, for 10 years, Google for Education specifically was free. And so we do have a lot of this philanthropic approach. But that said, we have a lot of different aspects of what we care about and how we design these things for learning outcomes. One is this teacher in the loop, as I mentioned. Another is our focus on learning science. So a long time ago, we used to be very productivity focused and learning agnostic. We were just there to save teachers time. Post-COVID, given the role we played during that and just the scale, after the fact, we felt pretty much a responsibility to get more into the learning opinionated space. And when that shift happened, we made a pivot internally, as well, to invest more in a pedagogical layer to our product design. We are not an educational expert, and so we had to solicit the expertise of learning scientists and people in the community. And we partner with other companies, right? We play one particular role in this equation, but that's why we partner with so many other people that are also playing a role in education.

MARTIN WEST: Ying, over the past decade, as you know, there's been this growing emphasis on making decisions about educational resource allocations based on rigorous evidence. So the kinds of experimental or quasi-experimental studies that the department compiles in the What Works Clearinghouse. With AI being adopted so rapidly and it's seemingly constantly evolving, it seems inevitable that research will always be lagging behind the technological frontier. So how do we tackle the challenge of ensuring that decisions are evidence based without falling behind of where and how students and teachers are actually using the technology?

YING XU: Yeah, so I don't think there is a perfect solution for this. Actually, for technologies that have been here for a long time, we still don't have conclusive answers pointing to either directions, just like how we actually went back and forth between the cell phones on whether we should allow cell phones at school, and things like that. I think one solution might be we could actually take a broader look. So as we actually talked about just now, AI actually did not emerge out of the blue. It didn't emerge from nowhere. So there has been technologies before AI that share a lot of similarities. Those are the things that we could look back, and maybe it could be used as a useful reference for us. If we think about intelligent tutoring systems, people have been studying the efficacy of those systems for decades. And they might not be as customized as AI, but they share a lot of similarities, like engaging students in conversations, and asking questions, and providing feedback. So if we look at some of the benefits and trade offs of those systems, that might offer some insights on how we might predict AI's future in education. And the other solution is look broader, and just look at the theories of how people learn and learning sciences. So there are some basic principles that should hold true. So, for example, students would benefit from immediate feedback and formative feedback in their education and learning process. So based on that, we might recommend school districts and educators to look for AI products that could offer this similar type of formative feedback, and just encourage them to focus on those features rather than the specific products. I think having this kind of high-level recommendation and evidence will also give the district, and schools, and educators more flexibility to choose the product and platform that will be most suitable for the content instead of making recommendations directly tied to a specific product or platform.

MARTIN WEST: So evidence based could be-- can mean being grounded in solid principles rather than necessarily backed by an efficacy study. Yeah. Roberto, this was a theme in the department's guidance, as well.

ROBERTO RODRIGUEZ: Yeah, I mean, we do need these efficacy studies. And we to-- we have a set of articulated evidence standards at the department that we've tried to distill for the edtech community, and some one-pagers on our website, which I encourage everybody to take a look at. That's certainly important, though, summative and gold standard studies around how edtech is designed and used. But I also think we do need more of these formative studies-- and more of the studies like the one you referenced, Marty, in Turkey-- to better understand, you know, what are the conditions around which this edtech is being used? How can it be informed by sound pedagogy, and in ways that actually help to support outcomes for students? We are in vehement agreement here, it sounds like, around the importance of humans in the loop, educators in the loop, and students in the loop rather than a vision of a machine-driven, educator-absent endeavor in education. That's good, but we have a lot of work to do to go from point A to point B in terms of enhancing the AI literacy of our teachers, of our education system, and of our students to be able to get there. I think exercising that good pedagogical judgment, that good instruction, using the right tools, that's going to be inherent for educators. But they need to better understand what AI is and isn't. They need to be equipped with more capacity to better understand how AI can be used, and our students need more AI literacy, both in terms of how they think about interrogating and examining AI in their learning and in their classrooms, but also in their lives. I mean, every young person has AI in some form or fashion at the tip of their fingers on their cell phone. And so the technology has evolved so rapidly. There are so many implications for learning and for the broader kind of social and emotional well being of our kids. And we need to really catch up, I think, as a country to grow our capacity to be able to ask the right questions and begin to answer those more quickly.

MARTIN WEST: Marta, do you want to add anything on how Google thinks about testing the efficacy of its programs as they evolve?

MARTA MCALLISTER: Yeah, so I should first mention there are obviously a ton of teams at Google that work on things they would consider what I call lowercase learning, right-- things like Search and YouTube that are ironically not built for learning, but probably to the most leveraged tools in the world to learned something new. Those are tools that are not purpose-built for education, whereas our team is more on the capital E education side, where we purpose build things for schools. And, as I mentioned previously, we, historically, were very learning agnostic. And so our efficacy was more on productivity metrics, and adoption metrics, and user satisfaction. When we took that pivot that I was mentioning to be more learning opinionated in our product, we also had to change a lot of aspects to our product design. So we implemented what a lot of edtech companies have already been implementing-- things like logic models where you can say x feature is designed to move y attitudinal, behavioral, or learning outcome, which is supported by z learning science hypothesis. And for some products that we have-- we have this consumer product called Readalong that goes direct to students. It works with literacy. We were able to do in RCT for that particular product because it made sense to. It's direct to students. It was an experiment with enough variables that we could get accurate data on. But for many of our products, like Google Classroom, that's an experiment with way too many variables for us to in any way claim direct learning impact. And so we've done a lot of work with external folks, like Leading Educators and WestEd, to develop-- they developed this framework called, VATT-- Value Added Technology and Teaching. Don't quote me on that-- something like that. You can Google it. But the VATT framework basically is intended to measure efficacy on instructional best practices. So what is the impact of those edtech tools on teaching practices that we know are connected to better learning outcomes? So rather than saying Google Classroom is helping to improve learning by x, y, z. Internally, we have certain features and metrics we'd like to lean on to know that things are going in a good direction. But rather than leaning into that, we really focus on, OK, what are the instructional behaviors and practices that we are reinforcing that we know are connected to good attitudinal, behavioral, and learning outcomes? And so that's a bit of the work we've done on that side. But again, for the products that are more direct to the learner, where we could do something like an RCT, then that's where we've done that.

MARTIN WEST: Great. I see my colleagues have collected some questions from the audience, so I'm going to ask them to share them and begin to get some of your voices into the conversation. But before I do, I have to ask one more question about this point of vehement agreement among the panelists, which is that we need humans in the loop and we shouldn't be thinking about replacing, right? This was a theme throughout each of your presentations. At the same time, our nation and others around the world face real severe challenges in terms of staffing schools and classrooms. They also face, in many cases, financial trends that threaten to make those challenges worse over time. So is it perhaps too restrictive in thinking about the potential of AI to limit ourselves to only supporting current teachers in their current role? Shouldn't we be thinking about maybe replacing some of the traditional ways we've used labor in education?

ROBERTO RODRIGUEZ: Well, I'll just say, I feel like this is-- we are just at the beginning of our understanding of how we think about AI and its potential to enhance and deepen teaching, learning, engagement outcomes. I think we need to better understand what that looks like and how that looks before we take the leap beyond. All those points are well taken, though, Marty, I would say, obviously. And I think as we even think about AI-enhanced and enabled tutors that work alongside educators to aggregate information, provide that formative data en masse from a classroom, to be able to then respond with targeted tier 1 and tier 2 interventions following that, that's a really exciting way where AI is bootstrapping good instruction in a positive way. But I don't think we're quite to the point where we're ready to go all machine, although it is a provocative discussion. And I know that-- I know others disagree there.

MARTA MCALLISTER: I don't disagree.

ROBERTO RODRIGUEZ: I don't know [INAUDIBLE] might think.

YING XU: I'm not sure whether this directly answers your question. But to a lot of the research in teacher AI collaboration-- So we're actually very cognizant we want to involve teachers, but we don't want to increase additional workload for those teachers. If you think about a spectrum, on the one side is the teachers manually curating all the instructional materials. And on the other side is AI taking over and doing everything automatically. So in a lot of the research projects, we're actually trying to find the sweet spot on this spectrum. So where is the good balance between we want to involve teachers, but not just adding a tons of work to them? And in this way, we might preserve some of the teacher workforce without.

MARTIN WEST: All right. So let me turn to some questions from the audience. This first one is really about how the rise of AI could change curricular choices. "In a future of machines and AI should K-12 students lean in to STEM, or rather into arts and humanities?" Who wants to take a crack at that for me?

MARTA MCALLISTER: I'll take an initial crack at that, which is just to say, as the disciplines change, or as different subjects change and tools evolve in those disciplines, it's important to think and reflect on what are the skills that are going to be relevant in that discipline. I think all of the skills you mentioned-- STEM, and arts, and humanities are still relevant. But it's interesting to reflect on what AI is bad at. And yeah, a lot of the personal human parts are what AI is bad at. And so you could argue that it is better to focus on those things. That's what I think it's really important, for people to have a baseline of knowledge so that they can interact with these tools in a thoughtful way, right? When the calculator was introduced, there were still many instances-- there are still many moments in the class where a teacher would say, please don't use the calculator for this test or this assessment, because they needed you to have that foundational knowledge to be able to understand the problems you were solving later when you use that calculator. So I think there's still a lot of value in many of the different subject areas. But in terms of what becomes employable later, I think there's a lot of arguments you can make around what will be useful. And I think what's interesting about AI, too, is-- I'm not going to say it's democratizing access to education or the AI tutors, as you mentioned, are only so valuable because humans are more important,. Many of you in this room, might be really self-motivated and do fine with a tutor bot, but most people don't learn that way. They're not self-motivated enough for that. And so I definitely think that there's a point in which humans will always still be important, and how we learn is more important than necessarily just the subjects we learn.

ROBERTO RODRIGUEZ: I agree with that.

MARTIN WEST: All right. So let me turn to-- this question is on the crisis in mental health facing our nation among youth. Data shows that the mental health, and depression, and even suicide rate among US students has gone up since the mid 2000s, the time when social media became ubiquitous. Some express concern it is, in part, due to addiction to social media. How do you think AI can positively help in the mental health of our students?

ROBERTO RODRIGUEZ: Well, that's a great question. The mental health and well being of our students is something that we are-- and we all need to necessarily be concerned and attentive to, right? And I think we are at a point as a country-- and we began to see these rates of mental health needs rise even during and through the pandemic. But when you look at the challenges that our students are facing in terms of their mental health in our schools and our school-aged students, our challenges on our college campuses with respect to mental health, it is top of mind for every educator that I-- every system leader that I visit with. And there are also-- are the dangers of social media, which we need to acknowledge. Our surgeon general has done a fantastic job, I think, of documenting and bringing into the national light some of the research and the data here around that, which I think necessitates us to all be vigilant and also promote and commit to thinking about digital literacy and how we help to support our young people, to teach them how to be and stay safe and practice good citizenship online. But I do think AI is an interesting application. There's an interesting test case and interesting question around whether I could help on the mental health perspective. I do know edtech more broadly. There are a number of tools that provide platforms not necessarily to substitute for mental health providers and mental health practitioners, but to help connect and force multiply some of the services there. And I've personally seen some promising examples of that. Again, I don't think that replaces the need for a grade school psychologist, school social workers, and counselors. And we are marching forward as an administration and helping to try to do more there. But it is-- I think there might be some interesting test cases there to look at.

YING XU: Yeah, I think that we don't have a lot of evidence so far to say that it's going to be positive or negative. I think it could go either direction. I could see AI could provide, actually, a safe space for students to express themselves because they won't feel judged by a person. But we could also see that students might form attachment with AI. And what if that attachment takes away their interactions with other people? So that would be probably having very negative consequences.

MARTIN WEST: Here's a question that's addressed to Marta in particular.

MARTA MCALLISTER: Oh, guys, go you easy on me.

MARTIN WEST: How do you expect to mitigate what they call the Matthew effect in education? That is, the phenomenon where those who are already advantaged tend to receive even more advantages with regard to gen AI use.

MARTA MCALLISTER: Yeah, I think it's a great question, and we've seen that a lot of edtech has exacerbated the gap. And I'm not going to pretend that a lot of the edtech doesn't have that unintended consequence. A lot of our work-- I mentioned, for a long time we've been free. Obviously, that's to democratize access to our products. We only recently started charging in order to be able to sustain what we give away for free. So I think, in one way, that's-- I think we supported 23 net new countries during COVID that had no access to edtech prior to pandemic, which is pretty remarkable because these are folks that were almost 100% on paper that now, overnight, had access to edtech tools. And while we've seen a lot of post-pandemic learning loss, that is precisely why we've made this pivot to being more learning opinionated and to try to have very intentional feature product choices that can reinforce the very things we're afraid to lose. So, for example, we're very paranoid-- everyone is-- that we're going to turn into these blobs from Wall-E, as I mentioned, where you're going to have kids just staring at a screen and having this chat bot be their only interaction. Well, there are very interesting startups out there making very intentional product choices that actually force peer-to-peer collaboration, that actually insert that speed bump that reinforces that-- to mitigate that fear. There are other things like directly getting the answer, and speed bumps where you can actually slow down the process to getting the answer. We do this with Google Search today. We don't build Google Search, but we know that if somebody is searching something with learning intent, we actually insert an explainer and a step-by-step process before you get the answer. And so I think there's a lot of-- from a product and edtech point of view, there's a lot of mitigations we put into the tools that many people have access to for free that aren't just reinforcing this speed to answer versus getting the learning. That's a small role we can play. But I would be lying if I said that we could overcome this massive gap that exists. I think we play a very small and humble role in that, and we just try to be as responsible as we can in what we do offer, and to make it as widely as accessible as we can.

MARTIN WEST: Well, it seems to me that part of the answer there also has to be, if you just make the tools available and let nature run its course, that you're going to reinforce disparities, right? I mean, in the sense that when Khan Academy put out free SAT tutoring, it was more used by advantaged families than by less advantaged families. And so it seems like this is why we need partnerships with educators, why we need the government playing a role in-- there's only so much a tech company can do on its own.

MARTA MCALLISTER: Totally, which reinforces the earlier point I was saying, which is that systems only change when you really interact with a lot of elements of those systems. And I should also mention we do-- I talked a lot about products, but we also do a lot on the programming side. So when-- we do a lot with professional development courses. We launched a course on gen AI, not just on our own solutions, but also on the AI digital literacy point. But yeah, we play one row in a very big ecosystem. And I think we should remember the role that we play.

ROBERTO RODRIGUEZ: Marta, just your point is so important here, too. I think it speaks to the bigger point here, that the choices we make now around AI, and how it's used, and for whom, and in what context matter, right? And we have the agency to make those decisions, to make those choices. So I just want to, again, remind us all that, we have a whole host of products, powerful technology. We have that present in the lives and learning right now of our students, whether their schools have policies articulated around AI or not. And we have choices to make as parents, as educators, as leaders around how, when, for whom, under what conditions. And we have to really seize that mantle.

MARTIN WEST: So, Roberto, here's a question directed to you. When creating the guidelines-- and I'll interpret that as either the guidelines for developers or the prior set for educators more broadly-- which topics generated the most debate or conflicting views?

ROBERTO RODRIGUEZ: Yeah, I would say I think there is still-- what we do here, in both the report and in the broader developer guide, is to say, we want to make sure we're creating an ecosystem and a set of norms and practices that maximizes the potential-- the powerful potential that AI can play in being a learning tool and to minimize and mitigate the risks. I don't think there's a whole lot of controversy around that orientation. But I think the big challenge-- and you hit on it, Marty-- is the bigger question around bias, and discrimination, and inequity, and how we think about those dynamics in this still-evolving and not yet fully tested and tried environment of AI in schools and in education. So that's the issue that I think is the one that attracts the most attention and the one that keeps me up the most at night. And it really is about how we bring alongside the rapid evolution and adoption of AI-enabled tools-- a really strong commitment and set of actions and capacities around AI literacy. Because we need to-- our educators need to know when AI is present or not present in a tool. They need to know how to examine that tool, how to ask the right questions of developers, of curriculum providers. And AI-enabled tools are being embedded so quickly into the market, and into products, and into the ecosystem that I worry our educators and our system leaders don't have the capacity to be able to ask those questions that they need to. And so that is why we're really trying to do all we can to provide the guidance to the field as quickly as we can to support that work.

MARTIN WEST: So several of the remaining questions are asking questions about the implications of AI for curriculum. We saw a version of that earlier, "What should I be studying?" This is obviously something that we at HGSE have been wrestling with as an institution. We're always trying to anticipate what professionals in the field of education need to in order to be able to do their jobs well and have impact at scale-- and therefore, the knowledge and skills we need to give them while they're here with us. And we also have a bunch of students in the audience who are thinking about how they're going to invest their time and energies in learning over the next year or more while they're with us. What advice do you have for HGSE as an institution? I'll ask the version of the question that will help us. You can sort of-- what advice do you have for us as an institution? Ying, you can get us started.

YING XU: I think it's too new for me to offer any advice [INAUDIBLE] HGSE. I think that one very important thing we need to keep in mind is the development of AI is actually not dominated by the tech sectors, and educators and people from the social sciences actually would play a very important role, especially right now-- so when the technologies are already moving so fast. And we really need to think about the ethical considerations and how that is really impacting the human society. And this is a job that we should do, and we should be taking responsibility for asking those questions. And I think that's for the students at HGSE. And those are the big questions they should be interrogating and thinking about-- how to make AI more responsible, and ethical, and safe for our society.

ROBERTO RODRIGUEZ: Yeah, I certainly agree wholeheartedly with Ying's point there. And I am optimistic and excited about HGSE's ability to tackle some of those big pieces and really to bring alongside the different disciplines that impact and affect teaching and learning. I will just offer, one of the things that we try to do at the federal level-- and we've been trying to do this even more in the context of AI-- is to bring our developer community and those that might speak a language a little bit different than the day-to-day when it comes to our educators, our teachers, our school principals and our superintendents-- but to bring them alongside our education community more actively, and to create the type of network and conditions for those two communities to connect from the get-go. And this is just a question of language, right? And our developer community sometimes speaks a bit of a different language than we speak in education. And yet, the potential for our tools to solve those challenges is great, but only if. It is informed by the practical realities, and of those that know our students the most and know them the best. And those working with them every day. So HGSE has the opportunity to, I think, make that match, if you will, in an effective and powerful way.

MARTIN WEST: Marta?

MARTA MCALLISTER: Yeah, I feel like an underlying part of that question is, what are the skills you should be reinforcing and focusing on as an institution that's trying to prepare the students that are your constituents for the future workforce? And I'll speak just as a hiring manager, from this perspective. It's interesting to me that, before the emergence of gen AI and after, the skills I look for are still pretty much the same. And they are critical thinking, creative problem solving, the ability to learn and adapt, and critical thought. And so I think, as long as you are creating a learning environment that still reinforces those types of skills, the content and subject matter expertise might change, and tools might change. But if you have people that are coming out that are able to adapt and learn, the ability to navigate this ambiguity, that's a critical skill that we look for. So I would say, keep doing that and your kids will be just fine.

MARTIN WEST: All right. We have three minutes left. So in 60 seconds or less, what is your vision for how AI will shape the future of education in the next 10 years? [LAUGHTER] What's most important? What excites you most?

MARTA MCALLISTER: You go first.

ROBERTO RODRIGUEZ: I will say, I think the ability for us to disrupt some of the norms that we've seen in our system that have held us back from adapting, personalizing, individualizing focusing on meeting every learner where they are and helping them grow. I think alongside great pedagogy and a great educator, AI is a really powerful tool for that.

YING XU: I'm excited about the possibility of how I could offer a different-- unique, but complementary to all the other resources students already have. Yeah, I think that it's always important to remember AI is not replacing anything that we're already having and benefiting from, but it's offering something just additional to everything we're already using.

MARTA MCALLISTER: Yeah, it's not exactly a vision in 60 seconds, but some of the things I'm excited about, since you offered that in your question. Last year, when a lot of generative AI tools were really exploding, I spent a lot of time interviewing learning scientists to get their perspective on how do you think this technology will change how we teach and how we learn. And one of the key points that was interesting out of those conversations was that we might actually see the adoption of better instructional best practices, whether teachers are intending to or not, because of the things they're trying to avoid. For example, they might actually gravitate more towards flipped classroom model because they are going to want to do more in-class assessment of mastery and verbal demonstration that the student actually understood something, because they're paranoid about take home homework assignments and cheating. And so I thought that was really interesting about how AI might shape how we teach because of the incentives and the fears around the technology. So that was one point. And the other thing that I found really interesting, I think this was actually from a talk from Ethan Mollick back at ASU-GSV in April, where he was saying, now you have democratized-- while you haven't democratized, necessarily, tutoring, you have democratized access to everybody doing things that used to be limited to those who could code, which includes teachers. And so oftentimes, educators would have to rely on edtech providers to provide a solution that then they would have to hack around to make work for their use case. But now they have a technology where they can actually build custom solutions for what they want to do in the class. And then the last thing I'll say-- this is actually cheating. This is a bit of advice to your other question.

MARTIN WEST: That's all right.

MARTA MCALLISTER: But also, what excites me about this is how you bring along students in that journey is particularly interesting to me. I promise this isn't an ode to Justin Reich, but I did observe one of his classes. And what I found fascinating about it was the assignment was for the students to use any generative AI tool to make a point about generative AI. That was the assignment. And they had half the class present, the other half observe, and then switch. And the range was amazing. One student who was really into politics and government, had this whole thing about deepfakes. Another student, who's really into art, looked into watermarking tools, and models scraping, and copyright infringement. And my point in sharing that is I'm really excited about how we might see this partnership between the teachers and students, and having them learn together evolve as a result of this tech, and all of us being confused about it.

MARTIN WESt: See, that wasn't so hard.

MARTA MCALLISTER: I was looking at the timer. I might have gone like 70 seconds, but--

MARTIN WEST: Well, thanks to each of you. In just a moment, I'm going to offer a special thank you to each of our panelists. But I want to make an announcement or two first, if we could get to the next slide. There we go. HGSE students, if you're in attendance or if you're watching upstairs, you're invited to a post-Askwith Fireside Chat. So a new innovation, but we have a lot of students who are intensely interested in this topic and wanted to provide an opportunity for them to continue the conversation. And so Luis Gaitan, Bharath Sriraam have organize-- no, I'm sorry, Luis has organized this., I know. And I'm going to give him credit for it. And so there will be pizza available, and come if you can. The other thing I want to say is that we'll hope to see you at the next Askwith Education Forum, which will be Tuesday, October 22, when we will discuss the upcoming Massachusetts ballot question number 2 on the MCAS exam as a high school graduation requirement. And then I want to say, Roberto, thank you for continuing to be a loyal alum. Marta, thank you for subjecting yourself, potentially, to a barrage of inquiries on LinkedIn. And Ying, thank you for squeezing this into your first month on the HGSE faculty. So please join me in giving them all a hand. [APPLAUSE]