AI vs Academic Integrity: From Bans to Best Practices (AI x Education Webinar Series)

Welcome everyone. Welcome again. My name is Sophie. I will be your host for today. Today's webinar will focus on sharing best practices for creating AI course policies. These insights come from workshopping ideas with students at Harvard and collaborating with educators both in the US and internationally.

I'm thrilled to introduce today's speaker, Sarah Newman. Newman is the director of Art and Education at Harvard University's metaLab. Her work explores the ethical and pedagogical dimensions of AI through both research and teaching. She leads the AI Pedagogy Project, which provides educators with a lot of great guidance for responsibly engaging with AI in their classroom.

And without further ado, let's welcome Newman.

Hi everybody. Um, thank you, Sophie, and uh, thanks thanks to all of you for being here. I'm thrilled to be here with all of you. Um, it's really an honor to be speaking to so many educators. And this stuff is—and I see some familiar faces, so it's always nice to see some familiar faces. I won't put anybody on the spot right now, uh, but this is a really hard topic, and let's—I hope this is a chance to learn from each other. This is not a situation where I have all the answers for you, and I'm going to be transmitting the answers. I have some thoughts, I have opinions, I have ideas, I have some, you know, research and data, but you really—this is an emergent space, and the conversation should keep changing. So I'm really, really honored and humbled to be here, and it's—it's really coming from that place that I'm going to share with you what I have today. Okay. So without further ado, um, I'm going to talk for a while, uh, and then we will have 15 minutes, let's say, at the end for uh, Q&A, and uh, let's get rolling here. And uh, to my—um, let's see, can you see my screen, everybody? Good. Cool. And to my assistants and co-hosts and everybody, please keep an eye on the waiting room. I see people are still coming in, and I'm not going to be looking there. I'm—now that we're—we're getting going. Okay, so this is the talk you're at. Hopefully you're in the right place, uh, and um, really excited to get into it. What we're going to be doing today is—um, I'm going to give you a little introduction to myself, so you know who you're hearing from, and I'm also curious to hear who's here and a few of your opinions about things. Then we're going to talk a little bit about the problem. I think we know what the problems—some of the problems are; there are many. We're going to talk a little bit about that, and then I'm going to offer you some suggestions, uh, some policy recommendations and some resources, and then we're going to go to Q&A. So no—no—no big surprises. Hopefully this is what you—this is what you came for. All right. So a little bit about me. I do a combination of AI research, Art and Design, and education. So I'm just going to give you little snapshots from my portfolio so you have a sense of my background. I—um, I'm really coming out of the liberal arts, arts and Humanities. My degrees are in philosophy and Fine Art. Uh, I came to technology through the ethics door about 10 years ago, and really also—same with AI—really coming at it as a—um, as a humanist more than as a technologist, which I think is really valuable. And for those of you here who have Humanities backgrounds, which maybe many of you—it's really essential that we're at the table right now in making decisions and having conversations. It's really, really important. So one of my goals in doing talks like this is to bring more people to the table and make sure that everybody feels empowered to have an opinion, to have a voice here. And it will actually make our education systems better and our technologies better if folks with diverse backgrounds are coming to the table. So uh, in terms of AI research, I also co-founded something called the Data Nutrition Project. Uh, we make nutrition labels for data sets. It's—it's a—it's—it's fairly technical, but it's also meant to be really legible. Essentially what's—what's in the data set. So now we're talking a lot about generative AI, large language models—what's in the data set? How do you know? Well, how about if there's a nutrition label for it, and it says what's on—it's in the data set. So I co-founded this nonprofit in 2018. We're still going. You can read more about us on our website. Uh, Maddie's going to drop some links in the chat throughout, so you have references for later, um, but that's my—some of my AI research. Art and Design—I'm an installation artist. This is—um, a piece I really liked that's about AI, and it's reflecting on AI. It's called *The Moral Labyrinth*. This was an installation um, from 2018, where it's basically a walking Labyrinth, which most of you, I'm sure, are familiar with the shape of a labyrinth, and the idea of a walking Labyrinth or a meditation Labyrinth. And in this case, the whole Labyrinth is made up of questions, and all of the quest—the questions are sort of poetic and whimsical, kind of strange, but each of them has a corollary in technology, and specifically this whole project was inspired by something that's become more popular and more known now called the value alignment problem in AI, which is essentially—well, there's a lot of ways to define it at this point because it's—um, sort of been co-opted by a bunch of—bunch of different sort of camps, but essentially the value alignment problem is how do we design technologies that are aligned with the values of the people these technologies are meant to serve? And it's really hard to do that because people have really different values—across cultures, we have different values just from one individual to the next. We have different values—I mean, look what's going on in the political landscape in the US right now—and then even individually we have conflicting values all the time, like we value many things that are in conflict with each other. So this idea of aligning values is—is like an impossibility, truly, um, and the idea that we could possibly align AI, I think, is—um, I don't think we'll get there. That's why it's called the value alignment problem. But the idea of this piece was to sort of walk through and meditate on your own values and recognize that to try to program a technology that's powerful, that's going to affect a lot of people, um, with one set of values is not always—it's not always going to serve everybody; in fact, it never will. But the other cool thing about this piece is the entire thing was made out of baking soda. So it's—it's not painted to the ground; it's not fixed in any way. It was—in that sense, it was like a sand mandala. So if you stepped on it, if you kicked it—it was short-term; it was just up for a weekend—um, but if you kicked it, if you stepped on it, so throughout the weekend it sort of disintegrated, um, which made it very difficult to install, um, but really cool to experience. Uh, and I'm also uh, authoring some children's books right now. Um, this is illustrated by my friend Michael Sherman, and this story, which hopefully will be out next year, um, is called *I'm Not a Tomato*, and it's about—it's about bias and data sets, basically, um, but specifically in this—in the case, because it's a whimsical children's story, it's—um, about a red round thing that rolls down a mountain, and in each village it rolls through, it gets mistaken for the food that grows in that village. When it rolls through Tomato Town, and the only thing they've ever seen is tomatoes, everybody thinks it's a tomato, but it's not a tomato, and then it rolls through AppleVille, and its journey continues, and it keeps getting mistaken for what the people in that town know, and it has to do with overfitting and having skews in our training data, and we'll come back to that a little bit later. And then lastly, that's what we're here to talk about today, is the work I'm doing in AI in education. Um, with the launch of the large language models—the public—public launch of the large language models in 2022—um, me and my team at metaLab at Harvard recognized that—and and many of you, of course, recognize that—there's a lot of change underway, and we started something called the AI Pedagogy Project, which I'll be giving you some—a little tour of later, and uh, Maddie will also put this link in the chat, but basically it's a resource for educators to help them contend with this moment, um, and so we wanted to put together some guidance, some materials, some assignments, um, an LLM use tutorial, and so I founded the AI Pedagogy Project, um, and we have a bunch of new stuff coming, and it's—it's been a really great resource. It's totally public; it's totally free, and it's curating from basically educators that are doing really great work. We're spotlighting their work, so we—I'm sure that some of you—we have your work already on our site, and um, happy to be in touch about this later. Okay. So what actually—well, sorry, before we get to that, I am going to go—go back and launch polls because I gave you my short intro. Now I want to know a little bit more about what is—um, where you all are coming from. So let me see if I can do this here. Okay, here we go. Thanks—thanks in advance for participating in the poll. This is gonna give everybody a sense of other people in the room. It's not just helpful for me; it's helpful for all of you. So the first question is, what do you do? And let's see—share results—um, can you see the poll results? Yeah, yes. Okay. So thanks. So all right. So it looks like we have—um, I'm not sure—this one, but anyway—um, sorry about any glitches in the polls—um, so it looks like we have mostly higher ed educators, and we have some higher ed administrators, we have learning technology librarians and other education roles—um, other—yeah, sorry, I didn't—I wasn't able to capture—um, all professions, but it looks like we're mostly folks from higher ed, which is great. Um, that's where I'm coming from, too. All right, here's the next one. There's six question—questions, just warning you. Um, General attitude towards AI, let's say this is specific to education, not just in—in the world. So General attitude towards AI in education. All right, looks like we got—okay, well, we have only 3% that are getting—me out of here. Oh, sorry, let me—in a poll and share the results with you. Cool. Okay, this is—this is really helpful to see. Um, 20% of you are hooray; 30% are tentatively hopeful. So that's 50% of the room is swinging—or, you know, on that sort of positive side of the continuum—or even split, actually. It's not true, because 37%—so more than—more than 50% of the room—even split between hope and concern; 37% somewhat pessimistic; 11%, and get me out of here, 3%. Um, great. Okay, that's—that's also really helpful, and you know, this has been changing. It's really interesting to see like a year ago to now how attitudes are changing, and maybe we'll—we'll come back to that, too. This question is, do you agree with your school or institution's AI policy? All right, we got most of you have voted. I'm going to end the poll here, and let me see if I can share the results. Okay, this is really helpful to see. Can you see the results? Awesome. 50% of you—my school—institution doesn't have a policy. Wow. Okay, well, um, hopefully this will be informative, and maybe some content to share with them. Policies are really important. I think that's—hopefully one big takeaway for today—policies are really important. Um, oh yes, it is good. That's great. It could use improvement. Okay, not surprising. It's terrible. That's also not surprising. Okay, um, let me see if it'll let me go again. All right, here we go. Now we're getting into it. Have you ever used AI for something you're not proud of or wouldn't share openly? Now remember this is anonymous. All right, here we go. 19% of you—that responded—have used AI for something you're not proud of or wouldn't share openly, and 70% of you have not; 4% of you would rather not say. Um, so thanks so much for—for—um, doing that one. That is a hard question, but I think it's important in the context as we're starting to talk about how our students use AI. Um, and this is the last one. Thanks again for participating. Have you used AI detection tools? Do you use them regularly? And share the results. All right. 56% of you have never used. Bravo. I'm—uh, for—I'm—uh, for foregrounding—on to come—have tried but don't use regularly; I use regularly. Well, thanks for that, too. This is really—um, helpful. Of course, some of you may have schools' policies that are requiring or requesting that you use AI detection tools, but we're going to talk about—um, some of the many problems with them. All right, let me exit out of here. All right, thanks again for doing that. I know it was a lot of clicking and some patience, too. Now I'm gonna—I'm realizing what time it is, and I'm realizing how much more time we have, so I'm going to go pretty quickly through some of this stuff to get the stuff that I think you're really here for and what's hopefully most important to you. What actually is AI? Well, this is one of the first things that's really confusing about it. It's a broad, non-specific term. What it means has changed since the term was coined 70 years ago, and it keeps changing. Things that used to be considered AI are no longer considered AI. Things that we're calling AI now we might not call AI in the future. It doesn't really mean anything specific. So it's very good to use specifics when you're talking about this technology. I compare it to transportation. Depending on where you are and what year it is, if you say transportation, you might be referring to a very different type of vehicle. So keep that in mind. If you're talking about air travel, you should say air travel, and same with AI—you should talk about the specific AI application, because otherwise we could often be talking past each other. When we're debating about AI, we might be thinking about completely different technologies. So if you're having a conversation with somebody about transportation, and one of you is thinking about a bicycle and one of you is thinking about a train, and you're debating on the ethics of it, you're not even talking about the same thing. There's a lot of uses of AI. This is kind of off the top of my head list of some of the AI applications or some of the applications in which AI technologies are embedded. We're not talking about all of these today. I'm just showing you that there's many things that you use every day, whether maybe you know or not, that are using AI or using some form of machine learning, which is often basically taking a big data set and making predictions based on what's in the data set about what's going to happen. That's slightly reduced, but that's essentially what most AI is. And the reason that we have more AI now—or AI tools now—than we did a while back has to do with the speed of our computers and how much data we're collecting. The more data that we collect, the better predictions we can make, and the faster our computers, um, the more we can optimize all of this. Today we're focused on generative AI, though. Generative AI is a subset of AI that involves—um, creating something that is novel or appears novel, even though it's based on these patterns found in training data, and we'll come back to that as we start to talk about large language models. Now, AI in education—and again, we've been having conversations like this for a couple years, so this isn't going to be particularly new to you, but I just want to cover some basic opportunities and challenges. We're basically going from broad to narrow. So we're talking about like, okay, this is the whole field of AI; this is generative AI; this is generative AI and education; these are opportunities and challeng—and like, here we are—are—um, talking about academic integrity and what it means for academic integrity to have AI in our classrooms or in our students' hands, even if it's not in our classrooms—um, but starting with some opportunities—um, and again, I think this is really review—there's opportunities around efficiency and automation; there's opportunities around personalized and potentially—personalized and potentially inclusive learning; there's also risks on the opposite side about access and who gets included and who gets left behind. Most of the things that are opportunities also have risks, and that's not limited to AI. That's true in most technologies. The same thing—like the opportunity of a high-speed train to go extremely fast also has risks. Most of our technologies—it's—it's the things that make it an opportunity can also be a risk, and it's good to talk about them together. So anyway, in the opportunities camp—again, different kinds of innovation and expanded learning, like maybe driving educators—like you all and me—to be more innovative; opportunities for students' creativity; opportunities specified AI for specific academic areas or academic research; and then critical thinking and social impact. We need to teach critical thinking like we never have before because suddenly there's tools available that are designed to seem true, but they're not true; they just are designed to seem true. And critical thinking skills are essential now; also critical thinking about the ethics of these tools, and we'll get more into that shortly. Um, and there's also new forms of collaboration that can emerge, potentials to impact economic mobility. Of course, these are the opportunities—no surprises here. And then on the flip side, um, there's challenges. This is not a comprehensive list, but this is some of the challenges, and I think we know this, and I think again this is why—this is why you're here. So what's a problem? Um, specifically in education, there's—we—we know that there's a lot of issues around academic integrity, plagiarism, which is—I made a separate bullet because there's—academic integrity is a bigger bucket that isn't limited to just plagiarism. There's a lot of other ways that you—you know, even if you're just not permitted to use something and you use it, it doesn't always count as plagiarism, but it's still compromising your academic integrity. But of course, also plagiarism—plagiarism is not new to AI, and I think it's important to remember that there's histories for all of these things. Um, copyright, um, which includes—if you're using the tool to, for example, evaluate your students' work, and you don't hold the copyright to your students' work, and you're giving that data to an AI tool, you're violating the students' copyrighted work because the company that you're putting the content into then has a copy of it without your students' permission, unless you get your students' permission. And then likewise, who owns the content that the models were trained on? As you may know, most of the popular large language models were trained on basically a scraping of the internet, which is essentially stolen data. So there's also ethical questions around that. But it's not just like, okay, ban these tools; it's all bad, or just let them—you know, this is the—this is the reality; let's use them. It's not—even though the title of this talk is AI versus academic integrity, it doesn't have to be versus. You can use AI, you know, with integrity if we are clear about our policies, if we're transparent, if it makes sense, if students are learning critical thinking, etc. So it's—it's framed as a—as a versus, but it really can be an and. Teachers—um, have a lot going on—like educators are overworked, overwhelmed. This is not necessarily their area of expertise. I think most educators are not getting extra time off to uh, just learn about all this stuff and try to keep up with these conversations. So it's a really—a big challenge—put on educators right now. And our students are really diverse and have different needs and have different opinions. So um, this is hard, as I said at the beginning. This is hard. And one—one other thing I'll say is that this is a historic time, and it's an exciting time, and even though I think a lot of people are in the—the weeds with like frustration about just how complicated it is and how much more work it is to try to get your students—to try to figure out how to get your students to do things in a certain way—if you take a step back or a few steps back, this is such an interesting time to be an educator. Like this moment—like 2024, 2023, 2022—this is such an interesting time. Like the fact that we get to be having these conversations, the fact that we get to watch our University administrations—for those of us who are not administrators—try to churn through these policies and these policies—maybe they're not that good—then revise them. And likewise for the students—for students to be students right now, this is like something they will remember for the rest of their lives. So as overwhelmed and frustrated as we might feel, try to keep in mind how historic this is and how exciting this is and give yourself some grace. Like we're not—none of us are getting it perfectly right, and that's why we iterate. So how do we move forward? Um, talked about this already. All right. So I think I alluded to this, but it's really not about teaching students how to use AI. I mean, many of the students already know how to use AI, and you can use it in a lot of different ways that will keep changing, um, but really helping students—um, understand like which skills you're trying to teach them, which skills you want them to learn, and how to apply them. And something I'd recommend is that while our focus here is on academic integrity, to have—for—for you all as educators, as leaders—to have a grasp of the many ethical dimensions of generative AI is important. It's not limited to whether your students cheat on an essay. There's a lot of ethical dimensions, including things like, you know, as I have here—labor, climate, misinformation, skills—learning skills or—or unlearning skills or deskilling, um, inequality, and more. So having—that's not the focus of this talk, but having a—a working command of—of some of these ethical dimensions will be really helpful for—for you, but especially helpful for your students. So since Halloween's coming up next week, and this is a new metaphor, so I'm sort of trying it on for size, but since Halloween's coming up next week, I've been thinking about how AI tools—I mean, maybe I'm just speaking about myself. Okay, so this maybe has more to do with my relationship with candy than anything else, but I think AI tools are a little bit like Halloween candy—packaged in a way that is colorful, shiny, and tempting. The ingredients are mostly garbage; it tastes good in the moment, but it doesn't nourish. If you consume too much, you'll get sick. This is more about candy, but it's sort of metaphorically about AI. We want what we can have, right? I think this is true for most of us. Forbidding it dogmatically will just make kids sneak it—or adults sneak it. So it's better to have some reasonable policy about candy or AI use unless there's a real reason for total avoidance—like an allergy, in the case of candy, not in the case of AI—but unless there's a really compelling reason for total blank—total ban—total avoidance, a reasonable policy will make kids—or depends which—which you're talking about—but in the case of candy, a reasonable policy will make kids sort of learn about, you know, how much they like and how much they don't like, and if they eat too much, they get a stomach ache. In the case of AI, helping our students understand what these technologies do is helping prepare them for the world that they're graduating into. And the allergy thing is like—in some cases—like if your kid is allergic to peanuts, then they definitely should not have a Snickers. That would be really dangerous. Okay, yeah, I see these peanut allergies out there. And uh, likewise, if you're teaching a skill that the AI replaces, then the students should not use AI for that assignment. Of course, it's kind of like the peanut allergy. So when there's a specific thing that you're trying to—just like with—if you're teaching spelling—if you're an elementary school teacher and you're teaching spelling—spell check is not a perfect analogy to AI by a long shot, neither is candy—but if you're teaching spelling, then of course you—they can't use spell check because you're teaching them how to spell.

You're teaching them an intuition about spelling and about the roots of words, etc. But if you're not teaching spelling, maybe spell check's okay. So again, it's really about the skills you're trying to teach. I think you I think you got it all right.

So some suggested dues here's some suggested dues. And the the purpose of the repetition throughout this is so that by the end you kind of get the message. So I don't mean to like be too redundant, but I do think, you know, telling the same thing in a number of ways hopefully something will click.

Um, do have a clear policy with reasoning. Use AI to critique AI rather than standing outside of this technology and critiquing it. It's way more powerful to use it and have your students use it to find errors with it, to find ethical challenges with it, to find places where it shouldn't be used for other reasons. And we'll talk more about some specific assignments.

Um, using AI to critique AI. Learn from students' curiosity and perspectives. The probably the best part of my job is working with students and learning from them. And a top-down policy is not practical, and it's probably not going to be very good. And that doesn't mean students should author the policy, but it should be a conversation, and we should learn from what students are curious about and what their perspectives are. There're usually a couple of steps ahead of us, especially when it comes to technologies. Not to sound old.

Um, know some of the basics so you can help demystify hype. There's a ton of confusion and hype out there. I'm also just going to um make a quick like um mention of this terrific book that just came out called *AI Snake Oil*. I don't know if anybody's seen it, but it's so good. Maddie, maybe you can pull up the link and drop it in the chat. I'm kind of obsessed with it. Um, and talking about um demystifying hype, um it's actually like the best AI book I've read in a long time. So you could read that book, you'll know a lot.

Um, avoid being punitive. A lot of policies are punitive. What about leading with trust? Like start framing things from the positive, understanding the ethical dimensions and challenges we just talked about that, and be open to some reasonable change so that your teaching can evolve too. Again, going back to this historic time, questioning and challenging how you've been teaching before.

Okay, so here are some policy recommendations. All right, so we talked about this. It's a historic time. Hear from your students. This is an AI Pol this is a proposed AI policy that I created with students. We actually did it as an exercise in class last spring, um in a Harvard general education class. And essentially, I had the students work in groups. It was a class of about 60, uh, and we'll also put the link in the chat to our this policy, which you're welcome to use and adapt. And there's also a QR code on the screen, but since you're on your screen, the QR code might not be that valuable. And essentially, I the students were and again this was last spring, so this is a year and a half ago, which is a really long time in generative AI terms, spring of 2023, but I still stand by this policy. I think it's it really makes a lot of sense. And basically what the assignment was is had this students break up into small groups and come up with an AI policy for her, look at the current academic code of conduct, discuss what they thought should be used and why, and sort of debate it and talk it through. And one of the groups could was so like in such firm disagreement they actually split up into two groups, which is amazing. Again, these are undergraduate students having a chance being asked like what do you think the school's policy should be and coming up with ideas. Then they presented their policies to the class, and they were really interesting and good. And then as a follow-up, some of the students reached out. I was I was just a guest getting like the guest um week on AI in this creativity course. The professor of the class is David Atherton. And some of the students reached out to David after the class and said we'd like to have a follow-up session with Newman to like do something with the policies we came up with and send it to the administration because we haven't heard anything from the administration about what we're allowed to do. Everybody was waiting and and uh so we did a bonus class, which was super cool and fun, um and a bunch of the students came back who were really invested, and we basically came up with a policy. We converged on policy. Um, Kathleen Fah, who's a PhD student, was also involved with sort of helping shape this. She was a metal lab research assistant last year, and of course me and David, um, and we this is our policy, and it's online. You're welcome to use it. You're welcome to adapt it. We have canvas templates. We have general recommendations, and um we did send it to the administration. They did send it out to all faculty. Um, it is they did link it on their site. It is not the official Harvard policy. It has a lot more detail than the official Harvard policy, but anyway, I want to offer this to you as one place you can start or look for guidance. And then my colleague Lance Eaton, some of you may have seen this, and we'll also drop the link to this in the chat, basically has just crowdsourced AI policies from anybody who wants to drop it into this growing Google doc of AI policies from different institutions. So this is another great place to look if you're thinking about coming up with a policy. Here's a resource.

So we we talked about this. Um, we should talk about what students should do and instead of what they shouldn't do. And I think, you know, this is easier said than done, but really thinking about like principles framed in the positive will be much more compelling and leading with trust, talking to your students about what your policy is, making sure that they have a chance to ask questions, make sure they understand what skills, not just like this is how I did it and like you have to suffer because I suffered or like that's not very compelling, but really think through why you're teaching, how you're teaching, when you're permitting AI, when you're not, and um you know there's a there's a lot more here. I'm conscious of time, time, and I want to make sure we have time for Q&A. So one thing I'll say is um if Johnny or Sophie if the Q&A tool hasn't been launched, oh yeah, good, the Q&A tool has been launched. So um as I'm teeing up to race through a little bit more content, um actually maybe we'll do is pull let me pull a group. So using the thumbs-up tool or a physical thumbs-up if your camera's on, but just using the thumbs-up tool, um if you want to do like five minutes more of content quickly, do a thumbs-up; if you want to do two minutes more of content, don't do a thumbs-up. I don't know, I'm trying to figure out how to get a sense from the room about if you want to see the rest of the slides or if we want to move to Q&A. I think we should go through the slides quickly. Does that sound good, Maddie? Does that sound seem like a good idea? Yes, I think so. Okay. All right, we're gonna all right, and I see a hand raised, which maybe is getting ready for the Q&A. I have a bunch more stuff to go through, and this is also being recorded, um and we're also going to post on the AI pedagogy site some like a synopsis of this talk so that we have these resources available for you so you don't feel like you're getting whiplash by how much I'm going through. I get really excited about this, and so my slide deck is insanely long, so I'm going to go through it quickly. Um, again, there's some redundancies here. Be exceedingly clear. Don't be dogmatic. Again, this goes back to the candy thing. Have reasons for what your what your policies are. Have reasons for them, um good reasons go a long way. Discuss this openly with your students. Plan to revisit and revise your policies. The policy should not be fixed; they should be changing. The tools are changing; the world is changing; policy should be changing. Any specific bans should be clear and justified. Some other idea, so I guess like going back to this last one, for specific bands should be clear, justified. I it almost goes without saying, and I'm going to just give you some citation references in a minute. All AI use should always be cited, always and annotated. There should it should the policy should be your policy or your school's policy should be clear what is permitted and what's not, and the students 100% of the time need to be citing just like they need to cite any other user reference; otherwise, it's plagiarism, period. So that like almost goes without saying, but I'm going to say it anyway. All AI, but they shouldn't be penalized for using AI if they're permitted to, but they need to cite it, and I believe they also need to annotate their use, and we'll talk a little bit more about what that means. Um, that said, if you have some rules about when and how AI can be used, one idea is to also offer a wild card, like a you know, like you get for your class, you've get one wild card where you can use AI on an assignment where it's not allowed or in a way that it's not allowed, and they just have to tell you when they turn in their assignment that this was their wild card use. They also need to share not only that they used it but why, what they did, and how it went. This like opens up a little bit more space for some experimentation. It gives space a little bit for students to learn. It makes you feel a little less dogmatic, even though you're generally not into AI. You could say you can't use it on your final or you can only use it on one dimension of your final, but any other assignment you can use your AI card, but you need again to annotate and talk about why you used it, and um this is creating a little more space, a little bit more curiosity. When you're thinking about sort of designing your own policy, I would break it down into what is the assignment meant to teach and what tools can and cannot be used and why. Why for each why can certain be used and why can certain certain cannot be used, and then designing assignments. So always be clear on what specific knowledge each assignment is meant to teach. Create assignments that are critical of and reflective of AI, and I'm going to give you a couple examples. So the AI pedagogy project, which I mentioned, has around 40 assignments that are curated from other educators, and some of them are specific assignments that are well all of them use AI and at the same time are critical of AI. All of the assignments do, and you can search the site and um but here are some assignments that like this one is similar to the one I gave in the course, the the ethics of generative AI. So the the students engage in a debate, and you know below here if you scroll down if this is a screenshot, but if you scroll down you could read about how to conduct the assignment. So these are assignments that you can do in class that are kind of getting at the meat of this um AI in teachers' hands. Um, reflect on the impact of and ethics of a teacher submitting an AI generative reference letter without the student's knowledge. This is an assignment. Have the students talk about this. Uh, correct a bad essay, uh practice EDI skills and learn about LMS by gener having it the model generate a poorly written essay and then copy editing it with annotation and reasoning. So not only are you making the essay better, but you're theorizing about why certain mistakes were made. And AI sandwich, which is from John Apollo, who I think might be here, um use AI tools for the beginning and end of an assignment with the middle being grounded in human knowledge and expertise. So here's some ideas, and again you can go to the AI pedagogy site and search for yourself. Use, adapt, however you like. Assessment when policies are good, assessment is less of an issue. Um, that said, and we we'll we'll make this available so that um I don't have to take you through all of this, but um essentially for assessments, think about the parameters you're giving and whether you should follow the same ones. Um, think about whether um whether you might use LMS for brainstorming or lesson design ideas and look for curated sets like the ones I just referenced of assignments that leverage AI critically and creatively. Um, we're going to move move past that. Don't use AI detectors. Okay. Um, in short, I'm not going to read this slide to you, but in short, AI detectors are discriminatory. Well, first of all, they don't work well, and they have a lot of false positives, especially for non-native speakers and people who are neurodivergent, and there's a lot of data on this. So essentially, students who are already at a disadvantage are being penalized more and discriminated against by the use of these tools. Not only that, but as I referenced earlier, you don't own the copyright to your students' work; that is their intellectual property, and when you put it into an AI detector, you are you don't know what's going to happen to that, and you don't know how it's going to be used. So also students can totally game the system. So if it's coming from a place of distrust and like rather than leading with trust and having reasonable policies and having open conversations with them, if if you're trying to catch them like they don't feel trusted, that makes them want to be sneaky, and students are smart and they're sneaky, and you can really game these things. You can actually ask an LLM to rephrase itself so that it gets past AI detectors, and it works. So um I would say see clear these, although there is some new data coming out that's interesting. These are schools that have banned AI detectors, and this is a very very long list for these reasons, the reasons I gave. However, um there's a that just came out with um some new data about ways AI-generated text might be watermarked, and it's possible that this will allow for AI detection to get better. Personally, I still think it's sort of the wrong attitudinal stance to take towards students, but there's a lot of different context, and there's a lot of different um students. So I'll just wanted to flag this that this is new data that's new research that's just coming out, and it might change and make obsolete some of what was on the previous slide. Okay, I think I'm going to skip ahead so that we have a few minutes for questions because I imagine some of you need to drop at the hour. Um, I'm going to go down to um basically um this resources section just to say there's a lot of resources. Um, oops, I just sorry, can you still see my screen? Yeah. Okay. Um, so oh, there we go. So um anyway, the AI pedagogy project, you can check that on in your own. We've got a lot of resources out there. We have an LLM tutorial that's free, and you can click through, and uh your students can click through. You can learn a lot about some of the basics of using LLMs and some of the risks. We have assignments which we already talked about, um and here are some references, and Maddie will drop some of these links in the chat about how to cite um how to cite for like for ML um APA and Chicago. So there are like citation rubrics and metrics, um and keeping up. This is the last thing. Um, it's really hard to keep up. A couple places I would suggest, I mean this is stuff we already talked about, are there's an AI and education Google group. Oops, sorry, there's education Google group. You can join the hosts of this podcast, Sophie and Johnny and their team, um have a great Substack. You can subscribe to that. Maybe you already did since you're here. Great ways to keep like they have a lot of data in there, a lot great way to keep up, um and we also made a page on our site with a bunch of the citations and links to the recent um recent papers on AI and academic integrity so you can learn more. Especially if your school is having you automatically use AI detectors, you can say look at all of this that's saying that AI detectors are bad. So here's the link to that. Um, we went over this, and this is the last thing. Okay, you take away one thing from today: the better your own grasp of your reasoning for your for a policy and your ability to articulate it, the more successfully it will land, the more you'll be able to evolve when NE the more you'll be able to evolve when necessary, the more your students will be able to learn and be equipped for the world they're graduating into. So this is a challenge to you to be curious about your instincts and reasoning, be open to trying new things, and be open to revisiting and r in sorry, be open to revising either now or sometime down the line. Thank you, Sebastian and Maddie, my and everything else, and with that we are going to take a few questions because I went really long. Thank you so much for your time, and I will turn it to Sophie. Thank you.

That that was great. I really love how we are really getting into the nuance of this AI versus plagiarism and try to bring in more asset-based lens. Um, we have a question from Connor in the Q&A, and we'll be alternating between the Q&A and um people who raise their hands in the chat room. We'll get to as many questions as possible. Um, we have a question from Q&A saying this is adding to all what you mentioned. I think in order for K12 teachers to create their own policy for their classroom department, they must understand the capabilities and limitations of AI. How do you suggest that K12 teachers learn more about AI in order to inform their policy? And an additional point to this is if we allow students to use AI in classroom, let's say they use it for specific writing, like translating sentences, translating sentences, teachers must have a basic understanding of prompting to teach about this AI usage. This is another training that teachers would need, and this is a pretty big ask for an already overworked population. Yeah. Great. Thank you. And I know some folks are dropping, so I just want to say we will share the slides with everybody who attended today, um so Johnny or Sophie maybe you can grab a list of the the people who were attendees. We can share the slides with you, and we will also um share the poll results because that was requested, um because I know folks are dropping, and the recording is going to be posted as well. And I also see uh I'm I'm going to get to answering these questions. I know a lot of people have a hard stop, um and I also saw the comment in the chat like this is way too much content. I agree. I admit it. I'm really passionate. This was like a f a little bit of a fire hose. Hopefully something landed, and again we'll make all the resources and content available to you so you can consume it at a pace that's not going to make you sick like binging on Halloween candy. Um, hopefully I'm not making anybody sick. Okay, so to answer those questions, um I think the AI pedagogy project, which is AIPpedagogy.org, is a really good resource for K12 educators. It's it was designed a little bit with higher ed in mind, but it goes through some really basic stuff, describing what AI is, giving some recommendations for educators, linking out on our resources page to some other places to look. Um, so I would start there, and that's not the only place, but that's I think a really good place for educators to start. It's kind of like 101. And then related to the second question, um and again not to like push it too hard, it's just the resource I'm most familiar with. Learning about prompting is is hard, and it's not something you can do in five minutes. I mean, there's there's one develops an intuition about prompting, and that takes time, and it takes experimentation. You're right, and everybody is overworked and has way too much to do. A few kind of shortcuts is I think our LLM tutorial on the AI pedagogy site gives some basic sense of how LLMs work. If you click through it, it's like a seven-step clickthrough and gives you a sense about prompting and how if you prompt this if you ask the same question multiple times why you'll get different results; if you change the way you ask why you'll get different results. And then our assignments where you search on the assignments page has suggested prompts for the assignments, so you're not just starting from square one. And there's a million other prompting uh prompting resources out there, but I would start there because that's the one I'm most familiar with. Amazing. Um, I don't see anyone raising their hand, so I guess I'm going to keep going with more questions from the Q&A. Um, we have Katherine asking, do you have examples of for ways in which it makes sense to use AI for research purposes? I think this is more for higher ed folks. Uh, yeah, I mean this yeah, this is a little bit outside definitely. I mean, it really depends on what kind of kind of research you're trying to do. Um, so you might need to clarify a little bit more. Uh, I mean, traditional AI like machine learning is used for a lot of different kinds of research to find sort of patterns and big data and make predictions about it. Generative AI specifically, um you can use it to synthesize um you know, big texts. You can use it to find certain types of things; it's sort of like the new text mining type thing. Um, I mean, I don't like I think there's a lot of applications. It really depends on the kind of research you're trying to do. I think often the sort of brainstorm creative brainstorming is a useful feature, like just really thinking outside the box. I'm teaching a a course or proposing a course for the the J Term, which is our January term at Harvard, that's um AI designed for ocean solutions, and um we're going to be using sort of AI creativity opportunities to actually think about really hard climate problems, not that we're immediately going to solve them, but it's like a way that you could potentially think outside the box and just you know, add a little randomness essentially, infuse a little randomness into your thinking, but it's it's hard to speak in such generalities, but there's there's a lot of applications for it. I'll also mention there's a a new tool that's really great called CiteS, and it's a large this is for the research question. It's a large language model, and I think it's cite.org MD. You can probably grab it um sc, and um it basically is a large language model that was trained on peer-reviewed research papers, so instead of getting like the garbage, the scraped internet, you're going to it's going to be pulling from a much higher quality repository of content. Awesome. Maybe I mean, I'm I'm happy to um to to stay and like take some more questions if for folks who want to stay, and again I know people are dropping, so for those folks have to drop, thank you for being here, um but I'm happy to stay for another you know, five or 10 minutes and do more questions, um if there's there's a lot in the Q&A. Um, we do have a few more questions in the Q&A. Um, we have we have um Gabriel asking, what's your advice for teachers who are just they don't want to change? Uh, yeah, um this is a very general question. I think um it it might be helpful to narrow down this. We definitely have people uh whether in K12 or higher ed who are just like sentiment wi more resistant to change. Yeah, I think I think it's a it's a really good question. I mean, one thing I would say is like listen. I mean, I think our instinct when we have people that like are resistant to change is like what can we do, and I think it's an opportunity for us to ask them like where are they're coming from, listen to them, like learn about their reasoning, so it's less about like we need to have the right argument and more about making space to hear from them. Like I think that's

The beginning—that's the beginning of like creating a proper conversation, and really learning from each other. Because if we come at it, come at it like we're right, they everybody needs to change, and we're just going to like mow them down with our all of our good reasoning, that like they're just going to dig their heels in. But if we can listen to, to what the reasoning is, what their instincts are coming from, and really hear them out, I'm sure that they have really good reasons. And then sometimes some curiosity opens up. A lot of times, I think, you know, resistance comes from not feeling heard. And so making space to hear folks, then they might be like, what, what do you—after you, you know, this is in like any kind of negotiation, this isn't limited to this, but like if you may really make space to hear from somebody, then often they'll be like, oh, why, why do you like it, or what do you—you know, like there's sort of like this fusing of horizons that um happens, um if you if you really make space for that kind of conversation.

I love that. I, I really love how we're going from the needs to really looking at what is, what is the value add of AI, and then go from there. I love that. Um, I, I know we're already at 10:05, would you be open to take another question, or yeah?

I'm happy to go till 15 past the hour for folks who want to stay, just because I, I, I talked so long, um, and obviously folks you need to drop, no no offense Tak, in um, but happy for those people who want to stay on. I, I think, I think Q&A is like some of the best stuff, and I'll also just point to the chat, there's some really great stuff going on in the chat, um, so thanks for those of you who are putting references in and resources in and questions. U, I'm looking forward to reading it properly after, but um, there's like a sidebar conversation that's very good there too.

Yeah, there was pure gold. A lot of people really love the information density. Do we have anyone in the um, in the room here who want to just raise their hand and engage in a conversation, or do people prefer if I pick questions from Q&A? Okay, looks like I will go for the Q&A. Um, we have another question. It says, "Hi Sarah, thank you for the talk. I'm really interested in uh to learn that you came to this work by a fine art SL insallation art and philosophy. And along those lines, I'm working on developing a decision tree for AI use in Art and Design. This is very subject specific for the college Art Association. Are you familiar with any resources to help with this? Um, I've gathered many samples of decision trees from AI in Ed group, but haven't seen any specific to Art and Design. I have interested to hear uh that your policy development discussion happen in the course about creativity, for example, like um, what does AI use for creativity, like is there like new policies um for this type of um subject area, what does plagiarism, or what are the red lines, or what would you encourage when it comes to Art and Design?"

Oh well, I mean, this is a really, um, this is a really big topic, uh, and this is a, this is a good question. Um, it's hard to speak in like such generalities. Um, let me take a look at the question here. Um, I think it's a really cool idea that you're working on a decision tree, um, so I think that could be extremely helpful. Um, I haven't seen anything related to Art and Design either. Um, Maddie, who's a student um at Harvard College who studies philosophy, I'm curious, have you seen any kind of like decision tree setup specifically for AI use and art design, or do you have any other thoughts about, about this question?

I haven't seen anything um super relevant to, to this specific question. I guess I've seen people using AI when it comes to creative projects, um, I think a little bit like the, the sandwich model that you were talking about in terms of like things that can spur us beyond the normal limits of our own imagination. So, um, if you're trying to think about something to write about, out um, I think it can be a really helpful prompting tool and also really a helpful editing tool, but I think the, the major engine of, I think the creative force I found is usually best left outside of AI. But um, I think yeah, in terms of the, the decision tree that sounds like a really cool project and I, I haven't really seen anything like that.

Thanks Maddie. Sorry, I threw you such a hard one. Um, yeah, I'll add, I'll add to this that, um, I mean Maddie's actually been working on like this ethical decision, sort of decision tree for another research project we're doing um related to AI. I think for creativity some of the big things that come up, um, for the, the person who asked this question is some of the big things that um come up are around the models, like especially the, you know, the image, their models are trained on other people's work, excuse me. So, um, we can use them to spur our own creativity, just like dadaism and other sort of art movements used random and use chance to spur creativity, we can use them for that. Some of the sticky stuff comes up when you think about if it's at somebody else's expense, um, like if you're using it to do something that you would have previously paid a human for, or that human labor was used to create, or human creativity, human creatives were used to create, like making a logo or something, and it's like trained on all these logos that were taken from the internet, and then logo designers are out of a job. So there's some kind of ethical questions that come up that I think there's different ways, and one, like one of the things about being, you know, a lifelong learner is that we're learning about how we learn best. So some people learn best with a blank, some people like, or how we work best. So in the case of creativity, some people work best with a blank slate, some people cannot start with a blank slate, they need a bunch of content to just get their brains going. Some people, like me, like take a nap and like figure stuff out when I'm sleeping. Some people like need to be sitting up at a desk with a cup of coffee. We all have different styles of working. So I think that like one of the interesting things to explore in the space of creativity is not a single path, but sort of understanding like the many dimensions of creativity and how varied that is from one individual to another, um, and uh, Lee, I hope you will share out what you come up with because it sounds, it sounds really cool and really important.

Thank you.

n yeah, absolutely. I, I happen to have been reading up on um literature on a gen AI for creativity specifically, and the truth is we have mixed results depending on what you're looking at, what how you use gen, you can both enhance and hamper creativity in your students. So yeah, I do see how it's a very nuanced conversation, and, and I do wish we have more time, and we are already at 10:11.

Sure.

oh, s sorry, Sophie. Oh, sorry, I was just, I didn't mean to interrupt you, I was just going to say there's a couple in here that I'd like to take. I'm looking at the Q&A, can I take those, or do yes please? Okay, um, so I'm going to take some of the anonymous ones, um, so well, um, okay, well, the off-topic question about we use the API, we use the open AI API, so that's for the llm tutorial, and you can write to us at hello AI pedagogy, and we can tell you more, but this question, how this is, you know, I didn't get to this, this is why I wanted to take it, how do you address students not following the policy for example, to cite AI use if you can't tell if they used AI or not?

Okay, well, this is kind of like students cheating for all of time, or paying somebody else to do their homework. Some people will get away with cheating, like, you know, we, we do as much as we can to create like a culture of honesty and integrity and trust and being reasonable, and some people will still cheat. We won't always be able to catch them, unfortunately. At some point in their lives they're probably going to get caught. It's better if it happens earlier, it better, it's better if it happens when they're in school then when they're stealing from their employer in the future or something like that. People will cheat, um, and we won't always be able to catch them. However, I do think you can ask, so with the question how do you address students off, if you suspect that somebody might be using AI, I think the first thing to do is invite them for a conversation and just say, how's it going, you know, can you tell me about the process for using this assignment, etc. Um, there were certain things that I didn't use an, an detector, but there were certain things about what I saw that made me wonder, can we just have an open conversation about it? Some people will come clean, you know, and just say, yeah, and I did, and I was embarrassed, or I felt like you would think less of me, you know, so like a lot can happen through conversation. You're not going to catch it 100% of the time, and just underscoring, you know, before the different assignments how important it is to cite. You can also refer them to your school's honor council. I mean, if you, if you really think some somebody's continuously not citing work, you can take more action. I would say it's more like how people cheated before AI, it's hard to catch them, but if you suspect somebody, for example, if their in-class writing is very poor and then their essays are very good, um, uh, that might make you suspect something, but oral exams, in-class assignments, there's other ways to test skills, and we can think about integrating those better. And, um, there's one thing that I, I kind of like, you know, ran past, but it was, it's like a new idea around like having like an AI affidavit, and the idea was like, what about when, because affidavits feel very formal and very serious, um, I mean, I think some of us have had to use them before, but when you have to like state something and sign your name on it, you know, on that document, it like feels very formal, and so I've been thinking that in addition to having a policy when somebody turns something in, having an affidavit that they have to fill out and sign that says that they follow the policy might be enough to just really make people want to be honest, because somehow like violating an affidavit, which feels like such a formal document, uh, just could be another, could be another motivator. So again, it's, it's not, it's not a perfect answer, it's not going to be a perfect solution, but those are, those are my instincts there.

I, I, I love that. Thank you for catching that question, that was a very good question. And I, I do also have, have some thoughts on this. I feel like it's never the students' intention to cheat, because students are spending time, they're paying tuition, they don't, they're not here to cheat. Cheating is like, there are a lot of other underlying reasons. Sometimes it's because there's also too much on their plates beyond just the academic area, uh, sometimes it's just, it's hard because they are not, they don't make the same connections in their head when it comes to some subject area or subject content, like it, it's not as apparent to them why we were doing this assignment. Sometimes the value add is not as apparent, and I know it's already like a lot on teachers' plate to have already designed the assignment and everything, but um, sometimes maybe just allow more room for the students to make the connection for what it is in there for them when they learn without AI, clarifying that aspect was sometimes also like just like make co, make cheating less, less encouraging. And I, I want to um thank human again for time to date and everyone else in the audience who stayed for an extra 16 minutes. We, I saw someone in the chat asking for a part two of this webinar. You're asking for a what, a part two of this webinar?

Part two? Oh, that's a nice request. Oh, thank you. Well, probably it should have been split up into two parts. This is pure gold. I, I also, because like we have a lot of people focusing on the asset base, but um, I really like the idea of like, what if you sign on the policy? I really like, like the idea of um making a wild card idea available, asking students to articulate what's working, what's not. There are a lot of really good, just like you can use it next time, kind of tricks that, that you mentioned, and, and I hope everyone else have something they really enjoy to take away at the end of this webinar. Um, once again, I'll have you, I hope you a really great day or evening wherever you are, and we look forward to seeing you at our next event. Thank you.

Thanks everybody. Thank you. Thanks Sophie and Johnny.