GPT-5.6 Sol, FIFA AI & Wall Street’s AI nerves

Well, if you think of OpenAI's original public character, all it was was safety. So if you look back at their webpage circa 2019, that's what you'll see. All that and more on today's Mixture of Experts. I'm Tim Hwang and welcome to Mixture of Experts. Each week, MoE brings together a group of the researchers, builders and thinkers working in artificial intelligence to walk you through the week's news. On this week's episode, we've got Lauren McHugh Olende, Program Director, AI Open Innovation, Kush Varshney, IBM Fellow, and Chris Hay, Distinguished Engineer. Welcome to you all. Glad to have this panel on.

We're going to cover a ton. As per usual. There's trouble on Wall Street. We'll talk a little bit about AI at the World Cup. And finally we'll talk a little bit about goats and Age of Empires two. But of course, the big news of the week is that OpenAI has officially announced its kind of rejoinder to Fable and Mythos, the long awaited GPT-5.6, Sol, Terra and de Luna.

Maybe we could take a step back and just zoom out for a second, because I always feel like it's easy to get lost in like 5.4 and 5.5 and 5.6 and okay, now Mythos is out. Where are we really? Do you think in the race between, say, Anthropic and Open AI? Right. Are we really in a situation where, you know, pretty both frontier models are kind of indistinguishable? Or do you think that this is distinctive? Does it put OpenAI ahead or, you know, even ahead in different directions from where Anthropic is focusing? I'm curious about kind of, you know, 10,000 foot review of where we are right now.

Yeah, to me, I mean, it is just a back and forth. I think they're kind of tracking each other. But the one difference that I did notice was, um, really on the, the safety side. So with Fable and Mythos, it was um, I mean, Mythos being kind of like this naked sort of model with Fable having like the single layer, it seemed, of a classifier just to prevent different harms and so forth. But on the Sol, as I was reading it, uh, they're kind of like doing more of this, uh, defense in-depth sort of thing. So they do have extra training. Um, in the model itself, they have some fancier sort of guardrails and then even, like going over to a kind of a reasoning model that can go and check and kind of like analyze, um, different responses before putting them out. So it's kind of many, many more layers. And I think, um, uh, that's maybe that divergence a little bit. Um, but yeah, in terms of the capabilities, I think they're just kind of going together and like to with it, they spend all this money and they kind of end up with like sort of the same model.

Um, Lauren, it is actually kind of interesting because I think I had kind of the same impression from Kush as like this one really weirdly seems more safety obsessed in some ways. And in some ways that kind of runs against, I feel like the the sort of public stereotypes of the two companies, right? Or Anthropic is like, you know, kind of hand-wringing about safety all the time. But yeah, Fable is kind of this, like one layer classifier, and then you have Sol and Terra and Luna. It feels like they really kind of put in a lot more than you would expect. And so it almost seems like the companies are running a little bit, you know, different from at least what their public character is. I don't know if you agree with that.

Well, if you think of OpenAI's original original public character, all it was was safety. So if you look back at their webpage circa 2019, that's what you'll see. Um, I feel like now there hasn't been a breakthrough as big as chain of thought and reasoning from early last year. I did see that Sol characterizes itself as better at scientific reasoning, which I thought was really, really cool. But then when I looked at the benchmarks, still, when it tested its ability to predict which protein would bind with which or which molecule would bind with which target site it was 7% accuracy and which is impressive, but, you know, 7%. Right? Yeah. And and they actually said that they did a bunch of expert interviews to figure out what is the critical level where we need to start worrying. And last model, they said it was 50%. This model they brought it down to if it could just do 30% we should be worried. So either way, it's still very far from that. And I feel like the goal lately because there hasn't been a reasoning level of breakthrough, that the goal lately has been more of kind of chasing these corner cases of, you know, 400 binding sites and 43 proteins. How well can you do every model release? Yeah. Which I think I always have to kind of keep in mind, you mentioned like 2019. It's like, um, we've come a long way. Like the fact that we're just talking about that as the primary task, which would have been insane to consider even years ago. Just being like, the thing that we're trying to do now, is, is a little bit wild.

Um, so, Chris, the other part of this launch that kind of goes without saying is that, um, you know, they are rolling it out in a Mythos style, I think is probably the right way of saying it. They have basically said, look, we're going to do a staged rollout. Only a couple of people are going to get it initially, and then we're going to do more full rollout going forwards. And, you know, as someone who's like a real nerd who, you know, like Fable came out and I was like, oh, can I just implement it against all of my projects that don't absolutely don't need that level of technology? Um, I guess the question for you, Chris, is like whether or not you think that like the the meta is now changed, like essentially that the Mythos release pattern where there's like a few people who get access in a kind of public way and then only later maybe does it, you know, GA. Um, is that going to be the future of model releases from these companies?

I hope no, because that's a stupid way of releasing models. You know, I don't like it because I don't get to play with it. I don't like it, you know? But but there's a more serious point I If I really think about the ethos of open source for a second, the best thing that you can do from a model perspective is put everything out in the world, and then everybody will discover what the true problems are, and then you can go and fix them. I like that as a model. I like the idea of openness. I I'm not a big fan of only the special people get it for however long because therefore I think it creates a two class system. And I don't think that's ultimately a good thing. And I don't think it's sustainable anyway, because other people are just going to catch up on these things as well. So, you know, there's a part of me which is, you know, it's just FOMO here. But, um, but I don't think it's a good thing. However, back to your earlier point about safety. I think this is kind of, you know, OpenAI's response is like, look, we're giving it to a few people. We're being responsible. And, look, it's not doing anything bad. And, you know, and they're they're taking a different approach to releasing their models and therefore I think we're more likely to see, uh, five. Six. Um, um, coming and us being able to do something with it. So I, I'm, I'm, I'm hopeful on it. What I do think is super interesting though, and and this is where I think OpenAI has taken an architectural different approach necessarily to, to to Anthropic is, if you notice on the, the benchmarks, the, the amount of tokens that the new Sol models, etc. are running is far less than Anthropic to get to at that point. So I think that is something to be kind of aware of that OpenAI seemed to be on a token efficiency drive. And I think there's one thing. I mean, I saw a report earlier this week that they found a way of making tokens more efficient. But it's not just that. I think they're actually capping the amount of thinking time for the models because and I don't know, there was an interview I heard from kind of Noam Brown, who was one of the researchers, you know, one of the driving forces behind the reasoning models in the first place in Open AI. And he was talking about the whole, you know, well, if you let the models think forever, then, you know, they're they're probably going to come up with the answer type thing. And it seems to be that their approach might be for the cybersecurity safety type thing is like, well, if it's going to take you 100,000 tokens or a million tokens or 10 million tokens, whatever the number is, to get to the really bad answer, well, we'll just lop that off after just maybe a million before. And, you know, off you go. You don't have enough time to think about it. And I, I feel as if that's approach and when I and that's why I say architecturally I, I, I wonder if I mean, I mean I could be completely wrong. I don't know, but I wonder if OpenAI has a better control of how long they reason for than than necessarily. And they've probably built that into their training so that they get better results that way. And I think maybe that's how they're getting there with cybersecurity. So I think this is more about kind of how long you're reasoning for and then cutting that off at the bad moment?

Yeah, absolutely. I mean, Chris, you're you're a I don't think you, you shy away from the title of being an AI governance guy. Um, what do you think about this new meta? I've been, you know, obviously, I feel like I agree with Chris, which is give me that model. I wouldn't play with it myself. Is this a new era of responsibility or just really kind of security theater? Do you think?

Uh, so, I mean, the fact that, look, the things I was saying earlier that they are like, technically doing additional things, I think that's not the theater. Right. The release part is the theater. And, um, so the fact that, uh, I mean, that they've invested it seems like, I mean, again, we don't know. I mean, again, to Chris's point, we don't know. We haven't been able to play with it. But, um, if that extra sort of stuff, those multiple layers are, are actually there and effective then. And you couldn't call that theater by any means. So, um, yeah. But like the, the overall like version of, the process, the timing, the selectivity, all of that. I would. Would agree with Chris on that.

Lauren, I guess just thinking a little bit about by way of connection of kind of your work on on open. Um, you know, I'm curious about how you think about this new kind of release paradigm impacting how open happens in some ways. Um, you know, I think I could see sort of arguments for like, this maybe actually being maybe good for open in the sense that there's actually like more time with which open projects can kind of get adoption, you know, as like the big companies kind of stress about who they release it to and how there's almost kind of disadvantage of being sort of out and first and available to all. But, um, yeah. Curious about the expert take on how, you know, folks in the AI open innovation space are are thinking about, you know, this kind of change in how the big companies, um, like Anthropic and OpenAI are changing how they release stuff.

Yeah, I mean, there's a range that people say of how far open models are behind proprietary models, and the range is like 0 to 12 months. So if we think if we take six. If we take six months as an average there. I think it's interesting that originally the government request was a voluntary 90 day period for review. And then they've cut that down now with this last release to a 30 day review. Um, so that would still leave plenty of time. I'd still leave five out of six months for proprietary models to be ahead. And given how fast developers pick things up like a day is enough to get a first mover advantage. Um, but I think it comes back to I mean, the difference here is that there's 20 organizations that supposedly have access to these models first. And who are those organizations, and are they the ones that have the skills, the time, the resources to be doing this testing, and then what testing are they doing? And what is the, you know, way to be comprehensive about that, because truly, the, you know, best case would be something that's scientific. You'd have a peer reviewed process to see how you tested it. When it comes to Fable, it looks much more Judge Judy with it. It was you know, it's AWS researchers originally who found a way to, um, jailbreak into some of the information. But then Anthropic claim was that that information you could already get with GPT five for with prior Anthropic models. So there was very little, um, critical way of evaluating, was it a risk or not? And I think that just comes back to where I was land, which is we need more technical skills in government. Um, because if it's just competitors policing each other, we're we're definitely in trouble.

Absolutely. Well, I'm going to move us on to our next topic. Um. Super interesting. Watch this as this rolls out. And of course, I am excited to get my hands on on the next Open AI models whenever they choose to give it to little old me. Um, I want to talk a little bit about Wall Street's bearishness on AI. It's been obviously a kind of historically huge kind of run in terms of Wall Street optimism around AI. And, uh, you know, last week or two has seen some interesting jitters. So, uh, SoftBank, who's a major OpenAI backer, uh, saw their shares fall about 13%. Uh, Nasdaq, which is very tech heavy as a kind of composite, uh, saw extended losses. And the kind of the most interesting one is Apple has also been significantly down. And, you know, I think looming over all this I think is some fears about the price of memory. Uh, you know, some of the impacts on how quickly some of these new models will get rolled out to what we're talking about a little bit earlier, I guess. I don't know, Chris. Maybe I'll turn it to you is like, how should I be Should I be selling my stock? You know, I don't want you to give investment advice, but, um, you know, I'm kind of looking at some of these indicators, and they look a little bit more worrisome than I would expect that, you know, at this point.

I, I don't know at the moment. I mean, I, I'm beyond trying to work out how to predict what goes well in the stock market. And sure, maybe, maybe that's the thing. AI once AI can crack what's going on in the stock market, then that's the point. We should all be able to just retire. Yeah. Yeah, exactly. I think there is. I think there is some part of it. And if I truly think about the market for a second, you know, it is really price for future value in a lot of cases. Right. And, and some of that's kind of a risk. And you're sort of saying, well, if AI is going to be massive, there is only a few players in this particular space and there's these are the infrastructure providers, etc.. And if you add all of these things up and if it all plays out the way that we all think it is, then yeah, you can, you you can see that that's going to work out. But it's very, very speculative. I, I the thing that I would, would really sort of bet against there on the opposite side is Lauren just said it's spot on. The open source is what, six months behind, you know, 0 to 12 months behind the proprietary models. And and I keep coming back to Simon Sinek's infinite game. Right. Which is um, which is effectively you're going to stay there's no winner in this game, right? You're going to keep playing the game until you either get bored or run out of resources. And in these cases, they've always got to stay ahead of the the other models. Right. That is the game of OpenAI. That is the game of Anthropic. They always have to be ahead of everybody else. And if that changes at any point, then then that's problematic stock market wise. So as long as they can always stay 12 months ahead of everybody, everything's fine now, from a jittery point of view to your point, if you're, you know, if if the stock market is six months behind, but you're going to it's going to take you three months to release your model behind there. Well, that advantage starts to kind of slip a little bit as well. So which way would I go? I always bet on open source. I think open source is is always the best bet. So let's see.

Kush stock picks recommendations I guess. The more seriously I think the question to ask you is, you know, I think there's one bit about, I think investors being worried about, you know, the, the nature of these companies always needing to stay ahead. Like what Chris is talking about. Um, and there's also this kind of hardware concern. I mean, I think we were talking, you know, originally about how GPUs are extremely scarce. It seems like DRAM and NAND memory actually might also become extremely scarce. Um, and like how much of that is kind of like actual friction on the market? Is that a realistic concern? Like, do you think that, like, there are now other types of, let's say, bottlenecks. I think in the supply chain that are really starting to crop up.

Yeah, I want to, um, like, I'm often want to do. Right. Uh, go bring up a historical sort of example. Right. And, um, uh, so sorry, Chris. So this week we're celebrating 250 years of independence, right? Um, and, uh, so the British East India Company, um, uh, was like all over, all over the place getting Americans addicted to tea. Um, they were like, uh, tea is such a great thing when it's like, you don't need it. Like, um, same thing with the AI companies like for most of the stuff that we do, like, do you really need AI injected into everything? No. But, uh, it's happening. And then the company, the British East India Company, kind of was having a lot of financial problems because they had to invest so much. And then there were all these other effects. Maybe there was, uh, a drought in India or something. Whatever. Same thing as the hardware prices going up or things of that sort. and then the government had to prop them up. Um, and then they kind of dumped all this tea that was really cheap price. This tax, the Tea Act and stuff. And then open source won. Right. The Boston Tea Party happened. They dumped all the tea in the ocean and kind of rejected even something that was really cheap. So, um. Yeah, I think that that was good. I was wondering where you're going to land that. That's incredible. Yeah. And as soon as the UK develops the best AI in the world, we're getting on those boats. And we're coming back, baby. Yeah, exactly, exactly. So, um. Yeah. So, I mean, I think that's the instructive sort of thing. It's that, um, uh, there's all these factors, like when something is so expensive to stay ahead and as Chris was saying, look, to be months ahead. Um, and you have to, like, just invest so much that there's, like, very little chance of getting the return unless you're kind of, um, getting some, like, propping up somehow. Then, um, then that's, I think, a reasonable reaction for the stock market to be concerned. So that's kind of kind of where I'm where I'm thinking.

Yeah. I never really thought about, you know, the East India Company as the OpenAI of its time. I'll be I'll be thinking about that one some more. Um, Lauren, thoughts? I mean, there's a lot kind of going on here. Um, you know, I don't know if you saw the slides that were presented from SoftBank that I think trigger may might have triggered the kind of stock loss, but it's this like really bizarre, you know, slide deck about sort of the future of AI and what AGI will provide and kind of this utopian future that we're going to live in. I don't know, I mean, are you kind of like, you know, that there will be a little bit of a course correction here, or is it sort of line go up? You know, we're actually still at the beginning of this for you.

Well, I think there's the perception which then feeds into the value of these companies. And then there's the reality. So I think something that's happening right now with Rammageddon and Rampocalypse is that there's a delay between hardware innovation and software innovation. So we realized maybe 18 months ago when we had our first, presumably trillion parameter model, no one really knows that we'd need more RAM. And then since then, researchers on the software side have recognized that that would be a huge bottleneck. And there's been well, there's always been tons of innovation around quantizing models, around key cache optimization, around different ways of optimizing flash attention and the things you can do to require less RAM. So whether those things converge at some point, and the lower supply because it's being eaten up by the big companies who can sign these five year deals with the Microns of the world, whether those things are going to converge, I don't really know, but I don't think the market is as aware of the software innovation happening because it's less in-your-face than a chip shortage. A memory chip shortage. Mhm. Yeah I like that. It's kind of like the the market is actually pricing just what it, what it can see or what it's aware of. Right. In some ways. And there is a point there though Tim though. Right. Which is we know the training runs costs a lot. Right. Let's not pretend that and the cost of serving inference to the world is a lot. And that has to be paid for from somewhere. So at the end of the day, these companies need to make money for at some point right now and, and, and, and it needs to be more money at some point than the cost of these training runs. And at the moment you're betting on future value, as I said. Right. And at some point, if that if that doesn't hit equilibrium somewhere, then then back to my kind of Simon Sinek, you're, you're going to get bored or you're going to run out of resources. And I think that's the risk is the run out of resources piece that that's the bit I would be looking for over the next few years.

Yeah, I feel like this is the golden window for humble old end users like me. You know, it's like it feels like the subscription is worth it just because, like, whatever I'm using it for, it's going to be costing the companies more. So I got to use this window before they start adjusting the price to what its true cost to. The businesses, which I don't even want to know is probably horrific. So and then I'll have to decide whether or not I want to pay rent or, you know, have my have my Mac subscription. So have your Mac subscription to forget what I'm saying. Thank you. Chris. Yeah, I know, I know your response. All right. Next story of the day. Um, kind of fun article I've been, if you're like me, have been just watching way too much soccer. I have at this point just kind of lost track of what game I'm watching or who's playing or why. Um, but, um, nice article from Wired. Uh, a journalist by the name of Sam Cunningham wrote an article called World Cup. Teams are in a race for AI dominance. And we've talked a little bit in the past on kind of the intersection between AI and sports, but I hope this was actually a pretty interesting profile of like all of the places where AI is impacting sort of team logistics and strategy, right? So from recruiting to penalty analysis to squad selection, it really, genuinely is the case that, you know, major global teams are experimenting with using AI to, you know, optimize their performance on the field. And perhaps the most interesting part of this article and I recommend everybody who's listening here, check it out, was that apparently FIFA was concerned enough that it actually built its own bespoke AI agent called football AI Pro to try to like even the odds between all the companies using this technology, which I think from my point of view is like, you know, pretty strong signal. Um, in some ways, um, I guess Kush. Like, should we be surprised? Like, it's a little bit surprising to me that, like, AI could make such a big difference that FIFA would be like, we have to find some way of rectifying this or else that there's going to be a huge delta this year. Um, you know, is that them like over hyping the risks or do you think they're responding to real signal is this, you know, how do I how to navigate this.

Let's say they invest in all this advanced technology for, um, small island nations or whatever have you. And then that can also lead to that secondary effect, the, the schools that they built with the field. So what about AI for agriculture or other sort of things that, that those countries might need to, might just come along because the types of analyses that are being done for soccer, um, can be kind of transferred over to, to other sort of endeavors as well.

Chris, based on your comments before the recording, I believe you are probably watching a game out of the side of your eye as this episode goes on. Potentially. I don't know, um, maybe. Yeah. Yeah. Um, but I actually really wonder about this is like, so I think there's we can talk about football AI Pro and whether or not it actually benefits the smaller teams at all, I guess. Do you buy the base case? Right. Which is, you know, like any other business. Sports teams are going to use AI. The ones that use AI actually might perform better over time. And so there's almost like this adoption race that's playing out in the sports as much as it, you know, is playing out in, say, business.

I would like to say, Tim, that I think I know something about AI. You know, people in the comment section might disagree. I can tell you right now that no amount of time I spend with AI means I'm going to be playing in the World Cup. Let me let me start with that. So, you know, and and the reality is, no matter how good a strategic or whatever you are, you know, folks like Messi are just on a different level, right? And we just we just have to kind of accept that. However, what I do think is the most interesting use case in that article is that they use an AI to go and find people who have got like, you know, whatever country is a grandmother for that country. And it just reminded me of like as a, as a Scotsman, they were all like, well, we need to, uh, we need to get people dating people from Brazil. That's the way forward. 20 years time, we'll invest in our future. You know, we need we need Brazilians in our team. And I just, I, I just think it's hilarious that, you know, you're not using AI for sports performance, but to find that dodgy grandma or mother connection. Exactly. You know for sure.

Um, Lauren, this made me think a little bit about, you know, so famously, a sport like, uh, American baseball, right, is a sport where, like, the nerds and the quants have really gotten into it. And I think one of the reasons is that it's statistically so clean. There's like a guy at the base and he swings the ball, and it's the same action over and over and over again. It seems like soccer is like really, really hard just from a data standpoint, to think about how you optimize. And maybe that's lying a little bit behind Chris's comment. You know, do you think soccer is like maybe less amenable than other sports like so that's maybe one way of looking at it is maybe in the future, baseball teams that use AI are going to be way ahead. But in, say, something like football, you know, it's really going to be at the margin. I think we would need. I think there's so much more data that could be so much more insightful, but that data's probably impossible to get, at least for a while. So, you know, I thought the article didn't mention, but there's a pretty much every team is supposedly wearing all kinds of sensors. So a actual accelerometer, heart rate monitor, these things. There was a Czech player who got his jersey pulled off and it ripped open and showed some of it underneath. And people were asking, what is he wearing, like bionic, like all these sensors. Yeah, yeah. So there's work being done on better data through sensors. But what I think about is video being such a powerful one, it's still video from so far away. And from a perspective that, you know, when actual contact with the ball comes down to millimeters, you know, a video from one of the stands is not going to help that much. And if you would think about, you know, if players were if you had an entire game recorded from every player's personal perspective and had all that video footage that I could see that being something that's maybe more powerful, and then you also see their gaze and you can see how a player is reacting on the field and coach from that standpoint. That could be really interesting. But with the data we have now, which is the stats, the video footage from afar, I think we're a lot. There's biometric data that's really interesting. Like, you know, using heart rate to know when to rest a player. But I think there's so much more data we'll probably have in maybe a decade that'll be way more powerful.

Yeah, we're like in the process of collecting it right now. And that is kind of interesting. It reminds me a little bit of maybe parallel to the, you know, for folks who don't watch football, kind of like the VAR debate. Like in some ways, like I wonder whether or not people would be okay with a world where we collect the data we need in order for AI to have an impact on the game. You know, it's kind of a funny world where players have to run around, like encumbered with, like, all of these sensors and cameras and stuff.

No, everybody would hate it. Right. You would never like no one would ever be okay with doing that. And that alone might be the the kind of bar of Chris I see smiling. I've got to comment on that. It's happening. Right. I mean, look, as Lauren was saying, look underneath, I mean the referee look at the referees. Like their arms are bulging with like, all this gear. Um, and people are okay with it. Um, yeah. I guess this is all going to be, like, below the jersey, right? You can't have like, wearing, you know, maybe contact lenses. Eventually we'll have all of that stuff. So yeah, I want to see one of those space robots coming onto the field and just like, you know what I mean? Like RoboCop. And then it will be just like, you know, and then they'll give them a yellow card and then the referee, the player just disagrees, and they pick him up and throw him out of the stadium. You know what I mean? That's. That's what I want to see. That's what you want robots into the game. Yeah. Yeah.

Well, I think and part of this is the struggle, right? Because I think like one of the reasons I think people like watching football is just the dream of it being, you know, it's just Messi, right? It's just like it's just people playing. And kind of the intrusion of technology has been, I feel like generally considered sort of a bad thing for the sport in some ways. I think when technology works, it's okay. But I mean, I think VAR is just doing such a terrible job that you know, that people don't like it. It's not like they don't like technology. They just hate bad, terrible technology. Exactly. I don't think you like good technology if it goes against your team as well. But, you know, that's that's a whole nother issue. Yeah, exactly. All right. A final story of today that we're going to cover. This is a fun little paper that came out that got some really good circulation. Uh, Microsoft AI researcher by the name of Andrea Adrian de Winter, um, did a paper recently called. and I'll just read it specifically as quote. If LLMs have human like attributes, then so does Age of Empires two. And you know, the basic story behind this paper is de Winter was like very, very frustrated with articles in the AI space and research papers in the AI space that assume that LLMs have human like traits. You know, I think one of the, um, you know, kind of metrics that they cite is that, uh, papers over the last two years have 50% of them start with the assumption that LLMs have these traits. And when they specifically, you know, set out to study these traits, like 77%, conclude they exist. Um, and so what de Winter does is basically say, look, I'm going to basically implement a bunch of these goats in the game Age of Empires two to basically do logic gates, right? Kind of like the simplest possible, like quantum level of computation you could imagine. Um, and, you know, I think the whole paper is a little bit of an exercise in kind of anthropomorphizing the behaviors that are observed when that occurs. And, you know, ultimately, I think de Winter's argument is to say that the kind of human like traits that we attribute to LLMs, you know, are just that, right? They're just kind of a lens that we're putting on the technology, and we shouldn't assume that these technologies actually have these traits in a really more fundamental sense.

Kush, what do you think? Is this the right corrective like, or should we just kind of remind ourselves that, like at the end of the day, we're just talking about goats from Age of Empires two?

Yeah, I think the, the absurd, the absurdity of of how he's presenting it is, uh, is his way of making us aware. Right. Um, so, um, I mean, absurdism is a is a very powerful sort of thing. It can kind of, uh, lift the veil of ignorance in many different ways. So, um, yeah, I mean, when we're talking about this difference between consciousness of these technologies or the, um, like the deep, whatever you want to call it, the deep soul that they might have or things like that, I think, uh, it is It's helpful. Right? I mean, to. To get that extreme position. To get us to realize that. No, this is just, um, uh, just electricity with these NAND gates and whatever sort of thing that that's happening. So it is like a physical medium, whether you want to call it, I mean, goats in a civilization, um, sort of thing, or you want to call them, uh, like actual circuits and stuff. So, um, yeah, it helps. And, um, yeah, just going a little bit philosophical for a second. Um, so, uh, one thing I've been working on recently is, um, uh, different kind of philosophies and how they view, um, kind of consciousness and the soul and so forth. So one of them is called Vedanta, and it kind of, um, has these like two views. One is what's called the vyavahāra. So what we actually observed the empirical version of, of reality. And then the other is called paramārtha, which is like the ultimate reality, like when you're enlightened and stuff like that. And it kind of makes the difference that like viewing the empirical world, viewing things as they are, um, is kind of like is true, but also like the ultimate reality is true. But, um, in both cases, um, like, consciousness is never like something that, uh, is really implemented by, um, like, uh, the, like the real world in a sense, it's all like an illusion. So, like having these goats teach us about this illusion, I think is a is a great thing.

Yeah, definitely. Well, I did want to build on that a little bit. Um, because, Lauren, I read this paper and I was like, I, I disagree vehemently. Um, and, you know, one of the reasons I do is I think a little bit about there's this paper that Anthropic did where they say, look, we show the model this content and we can tag these emotions in the model. And look, the model is scared when you talk to it about, you know, taking way too much aspirin or something like that. And, you know, I kind of look at that and I say like, well, you know, regardless of whether or not, you know, the AI model is actually feeling fear in some philosophical sense. You know, the fact that we now have this activation vector in the model that, you know, fires when the model is seeing certain types of behavior. And it's like predictive for understanding why the model does what it does under those conditions. You know, it's almost kind of like, I don't even need to know if the model feels fear. It's just it turns out it's a really useful way of giving ourselves intuitions for how an AI system is behaving, and how we might go about making sure that it behaves better under the conditions that we want. And so there's almost kind of a view of this, which is like, look, I know it's goats at the end of the day, but if we talk about it in these terms, we suddenly have access to this, almost like this, this interface for interacting with these systems in a way that seems much more, um, uh, flexible and actually quite productive in some ways. Um, do you, do you buy that? It's like kind of the pragmatist argument for this, which is I don't know if the model feels fear, but I know that the model will refuse to do things when it does it, when it experiences something called fear. And so, you know, that's what we're going to just use.

Yeah, I think the problem with that is, I mean, that's what we're doing now is like we look at like an MRI of a model and see the activation and then infer what it's thinking from that. But there's still so much randomness and so much that we can't explain in human language in that. And it can't it's not perfectly predictive. And that's the same state we're at with the human brain right now, like they identified a Jennifer Aniston neuron. But we have how many? I don't even know the number for permutations of neurons in our brain. So, I mean, statistically, they'd be likely to be at least one. Yeah, yeah. It seems almost hypocritical that the author is trying to say that let's look at goats to prove how random things actually look like. Meaning when our brain is just a bunch of neurons with sodium and potassium and glutamate going back and forth. And so what's more. Why is that more human? That it's sodium and potassium versus goats on a digital battlefield like that's. And I was at a conference last week where I heard the word soul come up like 3 or 4 times. So my prediction is that everyone working in the AI space will grapple with the question of what is the soul in the next year, and say that word at least one time, and I don't even, you know, there's no answer that we're going to get from that anytime soon, because that's a question. That's a hard one for humanity. Yeah. For sure.

Uh, Chris, final thought on this. Do you worry that your AI agents get sad?

Uh, not in the slightest. Um, so not not even vaguely, they're not getting sad. I mean, it's I, I'm with goat Guy, right? It's like, you know, because where where does this stop? Right. It's like, uh, do I not get to play video games anymore? In case, you know, I'm gonna hurt the feelings of the the baddie that I'm playing against, which is AI controlled. Do you know what I mean? It's like I, you know, and then you're like, oh, that's a cute character in the game. You're like, oh, well, you know, I should be nice to it or whatever. I mean, come on, it's like you, you've anthropomorphized this, I he said, struggling through that word. I hate that word. I'm anthropomorphized. Anyway, I think we've done this to ourselves and. Yeah, and if you train a model on the entire text of human text and internet text, etc., you're going to get something that sounds a bit like a human, you know, that's, that's, you know, and that's the same with like, birds, right? You know, if you, you know, birds mimic car alarms. Do you know what I mean? Are we suddenly going to be like, oh, I should drive a bird now, you know. No, it's like it's a, it's it's mimicking a car alarm. It's still a bird. Do you know what I mean? And I think it's the case. It's. It's just still an AI model. You know what I mean? And, um. And the fact is, we can look underneath the hood. So if we look underneath the hood mechanistically, right, we can see the circuits, we can see the classifiers, we can see the neurons, we can see what is clicking. It's deterministic. We can make it reproduce the things that we see. And we can see probabilistically how it gets to that next token. Right, which is based on the context. And, you know, without getting too theoretical about it. Right. It still goes all the way back to Shannon. Right. So, you know, in information theory, so, so none of this is new. And and again, if I think about like in the 60s when ELIZA came out, everybody's like, oh my goodness, I'm speaking to a human. And it's a bunch of if-else statements. Right. That's, that's what's underneath the hood. And then if we go to the, the kind of the, the rules based machines, the expert systems of the 80s, it's not that different. It's, it was more brittle. But everybody's like, oh my goodness. I feel as if we're just going through this like Groundhog Day every day. And maybe it's going to turn out that I'm completely wrong. And then in years to come, everybody's going to go. Of course, you're a moron. I expect them to go, Chris, you're a moron for different reasons. But but I, I don't think I am the way they are today actually represent, you know, biological you know, souls whatever. Right. I just don't think that's the case. That may change in the future. I just don't think we're there. And I'm based on that. On having peeked under the hood myself.

Nice. Chris. Hay, with your dose of reality to round out this episode of MoE, uh, Kush. Chris. Lauren, thanks for joining today. Glad to have you on the show, as always, and thanks for joining all you listeners. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify and podcast platforms everywhere, and we'll see you all next week on Mixture of Experts.