Transcription
And then something shocking happened. You train it, train it, it sucks, it sucks, and then it starts getting better on the training data. And then at a certain point, it suddenly also starts to get better on the test data. So it starts to be able to correctly answer questions for pairs of numbers it hasn't seen yet. So it somehow had a eureka moment where it understood something about the problem. It had some understanding.
So I suggested to my students, why don't you look at what's happening to the geometry of all these 59 points that are moving around in this high-dimensional space during the training. I told them to just do a very simple thing, principal component analysis, where you try to see if they mostly lie in a plane, and then you can just plot the 59 points. And it was so cool what happened. You look at this, you see 59 points just looking very random, they're moving around. And then at exactly the point when the eureka moment happens, when the AI becomes able to answer questions it hasn't seen before, the points line up on a circle. A beautiful circle. Except not with 12 things, but with 59 things now, because that was the problem it had, right?
So to me, this felt like the AI had reached understanding about what the problem was. It has come up with a model, or as we often call it, a representation of the problem. In this case, in terms of some beautiful geometry. And this understanding now enabled it to see patterns in the problem, so that it could generalize all sorts of things that it hadn't even come across before. I'm not able to give a beautiful, succinct, fully... I do feel that this is a small example of understanding. We've since then seen many others.
We wrote another paper where we found that when large language models do arithmetic, they represent the numbers on a helix, like a spiral shape. And I'm like, what is that? Well, the long direction of it can be thought of representing the numbers in analog. You're farther this way if the number is bigger. But by having them wrap around on a helix like this, you can use the digits if it's 10 to go around. And there were actually several helixes. There's a 100 helix and a 10 helix. I suspect that one day people will come to realize that more broadly when machines understand stuff, and maybe when we understand things also, it has to do with seeing patterns and then coming up with a clever way of representing the patterns such that the representation itself goes a long way towards already giving you the answers you need.
This is how I often, I'm a very visual thinker when I do physics or when I think in general. I never feel I can understand anything unless I have some geometric image in my mind. Actually, Feynman talked about this. Feynman said that there's the story of him and a friend who can both count to 60, something like this, precisely. And then he's saying to his friend, “I can't do it if you're waving your arms in front of me or distracting me like that.” I remember. “But I can, if I'm listening to music, I can still do this trick.” And he's like, “I can't do it if I'm listening to music, but you can wave your arms as much as you like.” And Feynman realized he, Feynman, was seeing the numbers, one, two, three. That was his trick, was to have a mental image as protected. And then the other person was having a metronome. But the goal or the outcome was the same, but the way that they came about it was different.
There's actually something in philosophy called the rule-following paradox. You probably know this. There are two rule-following paradoxes. One is Kripke, and one is the one that I'm about to say. So how do you know when you're teaching a child that they've actually followed the rules of arithmetic? So you can test them 50 plus 80, etc. 50 times 200. And they can get it correct every single time. They can even show you their reasoning, but then you don't know if that actually fails at 6,000 times 51 and the numbers above that. Interesting. You don't know if they did some special convoluted method to get there. Exactly. All you can do is say, you've worked it out in this case, in this case, in this case.
That's actually, we have the advantage with computers that we can inspect how they understand. In principle, but when you look under the hood of something like ChatGPT, all you see is billions and billions of numbers. And you oftentimes have no idea what all these matrix multiplications and things like this. You have no idea really what it's doing. But mechanistic interpretability, of course, is exactly the quest to move beyond that and see how does it actually work. And coming back to understanding and representations, there is this idea known as the Platonic representation hypothesis that if you have two different machines, or I would generalize it to people also, who both reach a deep understanding of something, there's a chance that they come up with a similar representation. In Feynman's case, there were two different ones, but there probably aren't... there are probably a few ones, or one or a few, that are really good. It seems like a hard case to make. But there's a lot of evidence coming out for it now, actually.
Already many years ago, there was this team where they just took... In ChatGPT and other AI systems, all the words and word parts they call tokens get represented as points in a high-dimensional space. So this team, they just took something which had been trained only on English books, English language stuff, and another one trained only on Italian stuff. And they just looked at these two-point clouds and found that there was a way they could actually rotate them to match up as well as possible. And it gave them a somewhat decent English-Italian dictionary, so they had the same representation. And there's a lot of recent papers, quite recent ones even, that are showing that, yeah, it seems like the representations of one large language model like ChatGPT, for example, are in many ways similar to the representations that other ones have.
We did a paper, my grad student Dawan Beck and I, where we looked at family trees. So we took the Kennedy family tree, a bunch of royalty family trees, etc. And we just trained AI to correctly predict who is the son of whom, who is the uncle of whom, is so-and-so a sister of whom. We just asked all these questions. And we also incentivized the large language model to learn it in as simple a way as possible by limiting the resources it had. And then when we looked inside, we discovered something amazing. We discovered that, first of all, a whole bunch of independent systems had learned the same representation. So you could actually take the representation of one and literally just rotate it around and stretch it a little bit and put it into the other, and it would work there. And then when we looked at what it was, they were trees. We never told anything about family trees, but it would draw, like, here is this king so-and-so, and then here are the sons, and this and this. And then it could use that to know that well if someone is farther down, they're a descendant, etc., etc., etc. So that's yet another example, I think, in support of this Platonic representation hypothesis, the idea that understanding probably often has something to do with capturing patterns, often in a beautiful geometric way, actually.
Just a moment. Don't go anywhere. Hey, I see you inching away. Don't be like The Economy. Instead, read The Economist. I thought all The Economist was was something that CEOs read to stay up to date on world trends. And that's true, but that's not only true. What I found more than useful for myself, personally, is their coverage of math, physics, philosophy, and AI, especially how something is perceived by other countries and how it may impact markets. For instance, The Economist had an interview with some of the people behind DeepSeek the week DeepSeek was launched. No one else had that. Another example is The Economist has this fantastic article on the recent dark energy data, which surpasses even Scientific American's coverage, in my opinion. They also have the chart of everything. It's like the chart version of this channel. It's something which is a pleasure to scroll through and learn from. Links to all of these will be in the description, of course.
Now, The Economist's commitment to rigorous journalism means that you get a clear picture of the world's most significant developments. I am personally interested in the more scientific ones, like this one, on extending life via mitochondrial transplants, which creates actually a new field of medicine, something that would make Michael Levin proud. The Economist also covers culture, finance and economics, business, international affairs, Britain, Europe, the Middle East, Africa, China, Asia, the Americas, and, of course, the USA. Whether it's the latest in scientific innovation or the shifting landscape of global politics, The Economist provides comprehensive coverage, and it goes far beyond just headlines. Look, if you're passionate about expanding your knowledge and gaining a new understanding, a deeper one, of the forces that shape our world, then I highly recommend subscribing to The Economist. I subscribe to them, and it's an investment into my, into your intellectual growth. It's one that you won't regret. As a listener of this podcast, you'll get a special 20% off discount. Now you can enjoy The Economist and all it has to offer for less. Head over to their website, www.economist.com/TOE, T-O-E, to get started. Thanks for tuning in, and now let's get back to the exploration of the mysteries of our universe. Again, that's economist.com/TOE.