On Memory as a Self-Adapting Agent

The caterpillar wants to encode, and the butterfly wants to decode in a way that, um, reuses the information adaptively. And we see this, we see this a lot in experiments and memory transfer. You know, they train a sea slug, and they and they extract the RNA from from its nervous system, and then they inject it into the into the brain of a of a naive donor. And then you can see memory transfer. Me, it's fabulous work. Wow.

Most likely, you don't just want to persist; you want to expand into other spaces, which means you want to alter yourself in a way that would allow you to colonize as as many other thinkers and as many other um action spaces as you can. And then, of course, they face exactly the same paradox that we started out with, which is that if you change enough, you're no longer what you used to be. And so is that okay? Is that persistence, or is that, you know, you've now you're gone now, and something else exists?

Michael Levin is a scientist at Tufts University. His lab studies anatomical and behavioral decision-making across biological, artificial, and hybrid systems. His work spans developmental biology, artificial life, bioengineering, synthetic morphology, and cognitive science. Today we'll discuss his recent paper, "Self-Improving Memory: A Perspective on Memories as a Gentle, Dynamically Reinterpreting Cognitive Glue." Please like and subscribe, and I hope you enjoy our fourth conversation together.

Michael L, thank you for coming on for the fourth time.

Yeah, thanks for having me. Good to see you again.

Great to see you. It has been a while, maybe 18 months to uh to 2 years uh since we last connected. And I'm so excited to cover the the papers, the recent papers um that that you've published here. We'll discuss today. I'd like to get us started off if if you wouldn't mind. You know, for the year of 2024, do you have any uh say broad highlights that you want to share with the about the lab?

Um, let's see, highlights about the lab. Boy, it's been there's been a lot of stuff. Uh, I know. Yeah, yeah. Um, well, a number of a number of papers. Some things have um really moved along um on the on the anthro-robots front um with respect to uh our ability to track communication between embryos. So so attempts to like really understand um collective intelligence above the cellular level. Uh, also some new work below the cell level looking at the gene regulatory networks and how they solve problems. Um, so yeah, some some some there's some stuff coming actually soon that was developed mostly in 24, some publish it, but some things on um novel transcriptomics of anthro-robots and xenobots and uh yeah, lots of lots of interesting things.

Lots of stuff that's so cool.

It's so cool to follow uh all of your work and your your your team's work. Um, and today in this discussion we'll cover, we're going to try to cover three of uh your recently published papers. And I'm going to try to structure it uh like past, present, future. So we have the past will focus on this paper called self self-improved devising memory, and then uh the present, second act will be the second paper on stress sharing as cognitive glue, and then the third uh AI, a bridge toward diverse intelligence and humanity's future. So that'd be ideal to cover all three of those. I know, touch, but if we if we get into one and we're you know, very in-depth into one, that's totally fine. We can know we can just cover one or two of the papers, but love to get a started with this self-improving memory, and uh could you explain the central thesis of this paper?

Yeah, well, it it it started when I uh when I started thinking more deeply about what's going on with the caterpillar butterfly transition. So for years uh I've been thinking about, so let's just let's just say what the what the data are. So the data are, sure, a caterpillar um it's a soft-bodied creature, has a particular kind of controller that it uses to move around basically in a two-dimensional world, eating leaves, uh which is this very specific type of brain that it has, and it it it turned into a butterfly, which is a hard-bodied kind of creature, completely different controller, um flies in three-dimensional space, and in order for that to happen, the brain is is is massively remodeled. So most of the connections are broken; many of the cells are killed off, and then it rebuilds, basically rebuilds a new brain. So the remarkable observation, this was made uh for um certain kinds of certain kinds of metamorphic systems years ago uh by uh uh by the shaman group, um and then more recently uh in in caterpillars and and by Doug Blackiston, who's currently a staff scientist in our in our Center, was that if you train the caterpillar, the butterfly remembers the original information. Now, one of the cool things about that, of course, is that uh we have here a computational medium that um where the information survives massive refactoring. So we don't have any Technologies like that where you can store some data, and then you you sort of rip up the medium, reconfigure it completely, and everything is everything works. So so I've thought about this a lot over the years as a um as a as a challenge to our computational kinds of architectures. You know, what what does it mean to store information in a way that survives that kind of uh that that that kind of massive damage and and remodeling? But um I I realized uh earlier this year that the much more interesting feature there isn't the fact maintaining the fidelity of the information, but it's actually remapping the information onto a completely new uh uh in a completely new kind of being with new problems, because the butterfly doesn't need the exact memories of the caterpillar; in fact, it can't use them. The specific memories of the caterpillar are of no use to the butterfly because it doesn't move the same way; it doesn't want the same things; it doesn't want the leaves that the caterpillar was trained to to find at a particular color; it's not going to move the same way. Uh, you can't you can't just keep the same memories; what you have to do is remap these memories onto a new substrate, so and and and make them make sense, make those memories be useful to you in a completely novel configuration. So if you think more deeply about it, it you you realize that this isn't some weird unusual um feature of this of this lifestyle, you know, this metamorphic lifestyle, but it's actually extremely fundamental in biology. And this goes back to a paradox, and and um uh I used to think it was Bates's Paradox; I'm actually not sure who who came up with it, but the idea of the paradox is is this: if as a as a species if you remain the same, you will eventually die out when the environment changes; you can't keep up, and you'll die out. But if you do change, then you're not the same species anymore, so once again you're gone, right? That that original being is is no longer there. So the paradox is how do you survive and persist while necessarily changing fundamentally? And this is true for all of us; this is this is the the the consequences of education, of transformative experience, of of puberty, of the the question is, you know, is is is the past me still still around, or is this or is this different in some way? This being different. So so I started thinking about this, and I and I generalized this in the in the following way um that that paper, the self-improvising paper, does a couple of things that I think are are are useful. One is that it um proposes this architecture, which is which is uh useful both on the kind of uh time scale of a single cognitive agent but also on the evolutionary time scale, and the and the and the system looks like a big bow tie; it's also uh it also looks exactly like an autoencoder architecture for uh used in in computer science and and machine learning. So what you have is you have this this big funnel on the left side, and um then there's a there's a thin um node or layer in the middle, and then there's a big funnel going out, right? So you can imagine that kind of bowing. And so the idea is this, so let's just let's just imagine this for our for the for the cognitive case. Um, you as a as a as a as an active agent, you have inputs, stimuli, experiences that you have. So these come in with very specific details. If you're a cell, it means there are specific molecular things going on, let's say on your membrane. If you are an animal, you're receiving signals from from certain receptors that you have in sense sense organs and so on. Um, so you so all of those are coming in, but but you can't you can't afford to store the exact details because by trying to track the micro details, right, by by trying to be a kind of Laplacian demon that that that just tracks the micro details, you will be eaten and and die. And no, the real world does not afford you the the time and the energy that it takes to do all that. So instead what you have to do is you have to generalize; you have to compress all of those instances into some kind of generative, some kind of very simple generative kernel that um is going to capture the what's essential about all the different details. Like lots of those details are are not essential. You know, if you saw a particular stimulus, the the exact shade of color and you know where the pixels were on your retina, those are not typically the useful things, but but there's something else about that experience that um maybe you maybe you've inferred a pattern of in the stimuli. You know, so intelligence does is is is it goes from from from particular instances to a general rule. So then you remember this pattern; you you learn and you remember this pattern. And so that's what you have access to uh it as as a memory engram. So you you you uh you you you store that as a memory engram, but then then later, and it might not be very much later, it might be, you know, this you can think of the center of the V as the now moment. You've got the past, right? You've got this now moment, and then you've got your future, and that now moment moves along, and and very very shortly thereafter you now have to re-inflate um those generative uh memory traces into some kind of um coherent story about what's happening now and what it means and how you can use those memories. And so another thing that I try to do in that paper is to recast your memories as messages from your past self, so to use the same kind of um formalism that we use to analyze communication between agents at a given time, so laterally, right? The way you communicate with others is um you can also think about um all of your memories as communications from a past version of you, and all of your actions and the things you learn now are messages that you are recording for your future self. Um, I long after I read that I I wrote that I I read a funny uh I think it was a tweet by Sam Gershman who said that your most important collaborator is you six months ago, and he's not answering emails, and that's you know, that's that's really very good; that's that's good. Yeah, if I had had that back then, I would have used it as this as the subtitle of a of a section in that paper, because yeah, it's basically what what what it reminds us of is that you don't have access to the past, but you have access to are the memory traces, the engrams that have been left by your interaction with the past in your mind, your brain and body, and now now at every at every given moment, and and this I think is interesting for kind of the definition of what it means to be an agent or to to to have a mind is to be in charge of constantly and and be be um driven to constantly figuring out what do my memories mean. You know, you don't you don't think of it typically; I think typically we don't think about it; we just assume we have memories, and we know what that means. Well, right, that might be true in our computational systems where the uh the bottom layers are highly reliable and and and there's this abstraction where when you program at a higher level, you don't expect uh you don't have to worry about the the data in your register sort of floating off and and changing into something else and so on. So we can get into that too with the difference with computation, but but the thing about biology is that uh you are you are always working in an unreliable medium, and so this is where so so um just finish finish the the thought on the on the cognitive side. So so you've got these engrams, and then you have to re-inflate them, and in order to re-inflate them, here's here's the thing about re-inflating them is that you've lost information. Uh, there are lots of um uh correlations and other things that were specifically squeezed out of that data when it was when it was written down, but now that you have it, you don't know exactly what it means; you don't know exactly how your past self interpreted it; you don't you don't have access to that; what you have access to is the recording itself, and now it's up to you to reinterpret it, and you might reinterpret it in exactly the same way, but you might not, and you don't have to; you don't have any allegiance to that; you you have to biology um tries to make use of information at the moment in whatever best way that it can, you know, and it's it's sort of like um I mean in literature they have this concept that when when when somebody writes something, you are not forced to interpret it the same way that they did, even though they say well I it I know what it means, and you say well, guess what, I got something else out of it. So so now I think this is what it means, right? So it's that's it's that sort of it's that sort of thing, and it it relates to the polycomputing um paradigm that Josh Bongard and I have developed, where the idea is um you you know there isn't there isn't an objective fact of the matter about what a particular physical event is computing, and it doesn't matter if if someone wrote the algorithm and says no I know what it does; I I wrote the algorithm. Well, there are multiple observers that could potentially look at those events and have a different model of what's being computed, and they're equally equally valid to the extent that they can make some use of it, right? You have to be able to adaptively use it. So um that means that the uh the left side of the funnel is largely algorithmic because you know, compression and inference and so on; you can imagine that as an algorithmic process, but the right side of the funnel is creative because it's underdetermined; you you can't simply uh deduce what was meant by the memory traces; you have you have to interpret them in the current context. So it's a view of cognition as a continuous sense-making where you're trying to make models of of yourself and your world, and you have to reconstruct them all the time because all you have is the traces that were left to you by the past self, and your past self may be very much like your current self, in which case it might not be too hard, but also it it might not. And so uh and so that that then leads to uh to an interesting uh thing where you can apply these concepts on a on an evolutionary time scale, and you can then see immediately the uh the origin of the incredible plasticity and the problem-solving capacities of cells and tissues and molecular networks and so on, because unlike um unlike in our computer technology, biology is working with a fundamentally unreliable medium, not only when you come into the world as a new being, not only can you not be certain of what your environment is going to be, but you also can't even be certain of your own parts; in fact, you're guaranteed that your own parts are going to change; they're going to mutate; there's going to be errors; they're going to all kinds of things. Um, and this is why, you know, I think we've talked before about like one of my favorite examples of this of this nephron kidney tubule, right? That that that you you can make these these nephrons with different copies of chromosome number, and basically the cells adjust their size to the amount of DNA, and then morphogenesis adjusts itself using novel mechanisms to make a perfectly good nephron out of cells that are completely the wrong size; in fact, different building the same pattern but in quite different ways. And so as a new as as a as a creature coming to this world, you can't be sure of much, you know; you can't be sure that of your environment, but you you also can't be sure of your own parts; you don't know how many copies of your genetic material you're going to have; you don't know uh how many what how what the size of the number of your cells; you have to construct a viable way to move forward on the fly, right? This is you know, play hands your dealt kind of kind of thing. And so that if from the beginning biology and evolution uh commits to this idea of an unreliable substrate where you don't really know what's going to happen, but you have to interpret that information left for you. So the genome was left for you by eons of experience of your ancestors, but you're under no obligation to interpret it the same way. And this is why with the exact same genome we can have planaria that make heads of the wrong species; we can have xenobots made from with a frog genome; we can make anthro-robots made of a totally normal human genome. The meaning of that information is not hardcoded at all, right? Issues. And so and so this is this is the the origin, I think, of that incredible plasticity in biology, and I also think that this provides kind of a um an intelligence ratchet where once you once you uh start down the road of making um active problem-solving agents, not solutions to fixed problems, which I think most most organisms are not; there may be a few exceptions um but most organisms uh and cells and even you know even unicellular organisms are problem-solving agents. Once you start down that road, the meaning of that information becomes less and less determined because the agents are going to be better at reinterpreting them. And so eventually you start projecting that kind of uh willingness to confabulate; I mean, literally, this is this is where confabulation is a is a is a feature, not a bug, because the ability to ignore whatever the information used to mean and to make up a story that helps you right now is is is really helpful in a wide range of of of contexts. And so I think that kickstarts what we eventually recognize as intelligence, which is increasingly um creative uh uh on-the-fly reinterpretation of information based on whatever problem you're trying to solve. And then the last bit which we can we can talk about the the the title of the um the title of the of the paper actually is is really only touched upon at the very end, which is uh this idea of the memories themselves as agents. And so we talk about that; that's a whole other thing we can talk about, as patterns as agents and so on.

Yeah, well, that was wonderful; you answered about six or seven of my questions already, so that was great, thank you, Michael. Um, and that actually actually is one of let's see, I go so many different places here. Let's start off with what you just mentioned about memories as a gentle, um, what do you mean by that? Yeah, how do you define agential memories, and what evidence do you have to support that idea?

Yeah, um, okay, so so let's first uh let's first just nail down um how how how I think about agency. So so there's there's a there's a feeling among many that uh there's there are certain certain kinds of things are agents, so let's say humans and maybe some animals and so on, um and that the use of that word in other contexts it's some sort of a category error. So so my feeling on all of this is that these categories are are not sort of given to us by on high and that then we are then required to stick by them. I think the category should follow the science, and that means that the way you know when you have agency is you take the the the tools that we normally have, so these are the tools of behavioral science, of cognitive science, and so on, the tools that you normally used to interact with agents, and you and and then you empirically, you don't sort of sit back and assume, but but empirically you apply them to other types of things, that being molecular networks, cells, tissues, organs, um cyborgs, biobots, whatever, and you see how far that gets you, and you see where where that gives you advantages and disadvantages. So I see agency as something that's only um applied, it's a term that can only be applied after experimental study, and you and you need to have a specific hypothesis about what problem space it's working in, and by the way, that problem space does not need to be three-dimensional space. So when people talk about embodiment and they mean some sort of wheeled robot or something that runs around and does things, I'm talking about solving problems in transcriptional space and anatomical morphospace, I all these other spaces that are that biology navigates but are hard for us to to to visualize. So so that's that's my take on agency. So now so now okay, so now let's think about the you know the spectrum of of agency, what kinds of things are on that spectrum? So um I've previously talked about placing some some unusual things on that spectrum, like and tissues and and you know and slime molds and and even molecular networks; we've shown learning in molecular network models and so on. Uh, but I think we can get much weirder than that even, and I think that's a good idea um because it's good to push through um push past our uh typical kind of limitations in thinking about these things, because I think you know we've inherited some some very specific firmware from our life on Earth and spec and and especially you know our our our um our latest history on the Savannah or you know all that kind of stuff; we've we've inherited specific ways that were very expedient for thinking about things, but I think I think they're quite um constraining actually. And so so now it's right now it's time to break through some of that. So so I started thinking about the following uh the following dichotomy because we typically most people typically think, well, okay, we have we have uh actual physical um beings, and so those might be biological beings; they might be some sort of you know um engineered machines or whatever, but you we have things things that do things, um and then we have patterns; we have data; we have we have information that these things process; we have uh patterns in various media, so so whether they be in a cognitive medium or patterns in in a you know in a digital memory medium, what we have patterns. So we have patterns, and then we have we have cognizers or we have thoughts, and we have thinkers, and um William James said something interesting; he said that thoughts are also thinkers. So how would that how would that work? So so so first let's um just to kind of warm us warm us up, let's let's think about a science fiction story, and this is I think based on a story that I read many years ago; I'm actually not sure if that's really the case or what the story was, but uh and and no doubt I I I've bent it completely out of shape, but anyway, but but but here's the but I think it's good, and here's the here's the story. So so just imagine uh from the from the core of the Earth, from the center of the Earth, come these beings; they sort of work their way up, and uh they're incredibly dense because they come from you know the center of the earth. So so they're incredibly dense, um their vision is I don't know in the gamma range or something like that, and they're walking around; what what do they see? Well, the first thing they see is uh that the Earth is enveloped in this very

Tenuous kind of plasma, um, that's all of us and everything that we see as physical objects. They don't see any of this; they, uh, to to them, they are so dense that to them all of these kind, all of the things that seem to us are just um, you know, just ethereal, wispy kinds of gaseous patterns that exist around the earth. And um, uh, much like much like when you walk through a garden, there's all sorts of patterns of of of of pollen and smells, and there's all this stuff, and you just sort of walk through it; you don't even see it. It's, you know, these patterns in the in the in the media, so they're walking around and and and uh kind of stomping through everything, and and one of them is a scientist, and he says um to the others, you know, I've been I've been I've been watching ing the gas that that our planet is surrounded by, and I and I see these kind of patterns in the gas. I say, what what what kind of patterns? Well, they're sort of temporarily persistent patterns; they kind of hang around for a while, and they seem to be doing things. It's almost like they almost they almost seem a gental; they almost seem like they, you know, they move around and they try to protect themselves from from dissipating, and they have certain goals, and it almost looks like they're they're doing stuff. And the others say, well, that's well that's crazy, but like we're real, we're physical patterns; patterns can't be agential. And by the way, how long do these patterns stick around? Well, about a 100 years. Well, that's nuts; nothing nothing interesting can in the space of 100 years. Um, and so and so uh, you know, and and and I've I added to this uh to this story; there's a there's a blog post that I have where where I kind of do a dialogue between, you know, um the one of those one of those core scientists and one of these patterns, and and he says, look, uh, you know, I feel I feel crazy talking to you because because you're just a pattern in this in this medium. And the human, which is the of course the pattern, is trying to convince them, no, we're we're we're real too; it's just a matter of perspective, like we you know we're we're real.

Um, and it's funny that there's there's another story which I'm quite sure is is a is a is a is a real sci-fi story about a a a a stream of plasma pattern that gets ejected from the Sun, and the humans are flying by in some sort of spacecraft, and they don't realize that this is also a sensan being that's just been sort of ejected from this uh from from its home and and so on. So anyway, so so the point of all of that is to re remember and to remind ourselves that we are actually also temporary patterns; we are um temporary patterns in metabolic space, and we persist for some amount of time, let's say on the average of a 100 years or so on the scale, and uh and and we try to keep ourselves together, but much like hurricanes and solons and gliders in The Game of Life and various other, you know, temporary um self-organizing and self- persisting patterns; one can take that view. And so that that reminds us that this this a distinction that everybody makes very categorically between real be, you know, real things or thinkers and the patterns within them or thoughts is really a Continuum; it's very much a continuous measure that's up to the eye of an observer to to note. And so that then uh suggests the following, which where now just beginning to um this is something that you know Josh bongard and I and and um and um Richard Watson and Chris fields and some others are starting to think about, which is what if what if you turn the standard Computing Paradigm on its head? So normally you have these Turing machines; you got the machine and that's the agent doing things, and then you've got its memory tape, and so it's writing things. And so what if um what if what if you look uh from the perspective of the tape? So in in fact, not not just the tape itself but the patterns on the tape, because in a certain sense they run the show; the machine is going to do what it does based on what the information on the on the tapes say. And so you can imagine uh with these so so now so now back to Memories as as agents. So so imagine this Continuum: so you've got you've got these um you've got fleeting thoughts, so these are patterns that run through a cognitive system and then you know wink out of existence; they disappear, so they're very short shortl lasting. But then you've got some persistent thoughts or recurrent thoughts that are kind of difficult to get rid of, and we know from Clinical Psychiatry that that there are there are those kinds of thoughts that once they establish in certain minds they um they're hard to they're hard to get rid of; depressive thoughts and you know obsessive thoughts and things like that. And in fact, some of these thoughts do something interesting; they do a kind of Niche construction, meaning that they they actually the more you have those thoughts the easier it is to have more of those thoughts; they literally change the brain. There have been studies on on how uh you know brain Ultra structure changes with with those kinds of those kinds of thoughts; it makes it easier to continue having that cycle, right? So these thoughts, these kinds of thoughts are a little more permanent um they're a little more uh they they contribute a little more to their own Survival; they're in fact changing the Thinker uh by their presence. And then then you sort of move up the Spectrum, and you can say, well, what about uh dissociative identity disorder, personality fragments? They are even more agential; they have they have goals; some of can talk uh they will certainly um affect the Thinker in ways that that changes how you know how they persist and how others persist. And then you know eventually then you have a full human personality, and then who knows what's after that, right? Transpersonal psychology suggests there may be something past that. So you can imagine these different much like like you have for um quote unquote physical objects, of which all of us living things are really just metabolic temporary metabolic patterns um you have you have a you have a spectrum of of of agency there, and then you can have a spectrum of agency in these kind of patterns too. So so that's so that's what I'm talking about, you know, this and this is just the very beginning of this research program. So uh uh the the conceptual part of it is to start looking at it as active data. So so yes, you have the machine that's moving the data around; what if you look at it from the other direction, and especially in um in systems like biological systems where the information patterns themselves and these might be patterns in the neural substrate, so these might be like fullon um thoughts, you know, traditional thoughts, but they also might be patterns in physiological State space; they must they might be um uh patterns of of of stress, or they might be you know all all kinds of things that physiologists and and uh and and different kinds of uh but you know therapists deal with, right? There could be all sorts of unusual patterns, and to what extent can you think of uh data as as driving the show and having its own uh its own life and trying to persist in its environment the way that the way that we P try to persist in ours. Um, so so that that has all kinds of interesting uh practical implications for for example regenerative medicine, and that's the kind of thing that we're working on now. So so could we you know could we look at some of the um you know one way to think about it for examp for example look at our bioelectric patterns, right? You can you as as we have for years describe describe the bioelectric patterns that we see during morphogenesis during regeneration as literally the thoughts of the morphogenetic collective intelligence. So you have a cellular collective intelligence; it's trying to navigate morphogenetic space to get from from an egg to a to a full body or to regenerate a limb or something, so it's navigating that space, and the bioelectric patterns are the thoughts of and we can read and rewrite them now to some extent; they they literally are the thoughts of that agent in exactly the same way that electrophysiology in the brain represents the cognitive content of beings navigating threedimensional space. So that's the the the more conventional story, as weird as that is, but but but that's a that's a more conventional story. The the new way that um I'm starting to explore now is what if it's actually backwards? What if the physical body that we're looking at is the tape and it's the it's the bi electric patterns that are really the driving agent and that what we see when we look at the consequences of that which are changes in in second second messenger function, gene expression uh chromatin, you know, epigenetic changes and then finally cell Behavior changes and morphogenetic changes; what if what if that's the tape, right? The physical body is the uh is the is the memory medium, and uh and and there's there's significant um significant action going on at the level of the physiological patterns themselves, and so that suggests some more applications uh and ways to test these ideas, and and that's what we're doing now. This is very very early days. H that's fascinating. So I have so many different questions I could go here, but I do want to stick on this um concept of memories as agents. So correct me if I'm wrong, and uh I'm going to try to restate some of this so we can think of memories as let's say Transmissions from the past, and we have to interpret those; they're not just given to us. Memory, and I'm sure many many many listeners know, but not not everybody knows that memory is not like a storage cabinet; you don't just go in pull out the thing; you have to actually not confabulate; that's going too far probably, maybe not, but you have to um recreate the memory um there's some trace of it, memory trace of it, and you have to actively I mean we do it we do it spontaneously, right? It's uh not a conscious construction, but we have to do that. So if we think of them as agents, is then do we have to think about say information patterns more broadly as being agential? So some of the examples you gave, memory is like say one example, but is any pattern of information potentially agential? Then yeah, that's a great that's a point, but potentially yes; we don't know; you keep you can't automatically um decide that that's the case anymore than you can do that with with with physical objects. But yeah, potentially that's the case, which means that you have to uh you have to try to apply the tools that exist for this to see to see whether that gives you an advantage, and if you find one then then then there you go. Um, you can imagine and and so this is this is a part that I left out about these um these patterns, these memory patterns that have to be reinterpreted uh from from from sort of time slice to time slice of a being. When you look at it from the perspective of the of the being themselves, you see that uh you see that um uh bowai architecture; so you see that okay, you're the recipient of a bunch of compressed information, and now you have to creatively expand that engram into what do I do now, you know the choice. Okay, that's that's from the perspective of the of the of the Thinker. Now from the perspective of the thought itself, it might be, and I'm not I'm I'm much like with our basil cognition models; I'm certainly not claiming that uh these that these thoughts are high high level agents like humans, you know, I mean some of them might be, you know, the dissociative um sub personality alars are close; I mean they're you know they they they they have a lot of those features, but but some could be very low-level intelligences; doesn't you know you don't have to be a high level self-reflective mind to be a to be some kind of intelligence. But but but from the perspective of that system, what what might your goals be? Well, one goal might be simply to persist; that that you know that might be a simple kind of darwinian way to think about it. So so so from that perspective, if you're a pattern, what you would seek to do is to change yourself and also the the thinker or the system around you in a way that makes it easy more easy for you to um propagate into the future, or in fact uh you know most likely that's not a sufficient story; most likely you don't just want to persist; you want to expand into other spaces, which means you want to alter yourself in a way that would allow you to colonize as as many other thinkers and as many other um action spaces as you can. So for example, you know so so that means that the the caterpillar wants to encode and the Butterfly wants to decode in a way that um reuses the information adaptively, but the information itself uh might work in to to uh to have features that make it easier to be encoded decoded and propagated into new uh into into into new embodiments. And and we see this we see this a lot in experiments and memory transfer um they kind of um you you like like like when when David gansman does does the the the RNA, you know, they train a se- slug and they and they extract the RNA from from its nervous system and then they inject it in the into the into the brain of a of a naive donor, and then you can see memory transfer; I mean it's fabulous work. Wow, yeah, and and there's been I'm not familiar with that. Oh yeah yeah so this is so this is David glans man's work um looking at the the basis of memory, and there are there's a lot of other work in the past that's been done about moving either either chemical extracts or pieces of tissue from a trained animal to a naive animal and so on. But but to me one of the most amazing things about that kind of work is that you know when you introduce let's say the RNA extract into the donor into the host recipient, you don't micromanage where the RNA goes; you don't put it into the right neurons to run the thing like a puppet; you just sort of you just sort of inject it somewhere into the brain, and yeah you know no it works; it's no problem; it just kind of picked out. And so so this idea we have we have other examples that are that are that are that are still unpublished of of of something similar in morphogenetic space. So so yeah you know I I think there are incentives on both sides actually for these to be uh reinterpret and for the for the agent to be good at reinterpreting them and for the memories to be good at uh at being the subject of that kind of process and thus colonizing the future, you know, colonizing the and expanding into into new spaces. And then of course they face exactly the same Paradox that we started out with, which is that if you change enough you're no longer what you used to be, and so is that okay? Is that persistence or is that you know you've now you're gone now and some something else exists? Yeah, it's beautiful; that makes me think of you say persistence a few times; the Persistence of memory by Sol Vador Dolly just makes me think of that uh that painting, and I know I wanted we wanted to cover three papers, but I actually think that there's so much meat here I'd love to stay on this one if if that's okay with you. Um, and the bow tie architecture in particular; that's something I wanted to definitely dive into because it's not a familiar concept to myself, and I imagine for many listeners it it won't be either, but it seems like there's something about this this shape this structure that you know cuts across; it's like a pattern across patterns, right? Um, so can you tell us a little bit like how did you discover this? I mean it also looks like a cognitive light cone; I mean it's just like this pattern that you see over and over again; how did you first come across this idea, or I don't know if you came up with it or um brought it together, but can you tell us more about it? Well, I certainly didn't come up with the bowai architecture, so so that's been been around for a really long time uh been around in biology; if you look at um things like signaling networks, you know a real um common one is there's a million different things that cause calcium fluxes, and calcium fluxes cause a million other different things, so you have this bow where like all this different stuff feeds into calcium and then and then it fans out again, and and there's a lot of discussion in the community, okay, but how does the specificity work? If everything boils down to calcium, how do you figure out on the other end what which which one of these things? And that's the whole point is that it's not meant to be a onetoone mapping uh it's not that the end the endpoint tries to figure out okay so I know you're encrypted, but which one, but I'm going to decrypt you to know exactly which micro State caused it; that's exactly not the point of these networks. Um, the other the other place that this cropped up, and I didn't invent that either, is um the autoencoder architecture which is used machine learning, where the idea is that you have these these layers of of a of a neural network like structure, but in the middle there's a very thin layer that forces um generalization; it's it's it's thin and it's and it's uh its information capacity is small, such that you cannot afford to remember details; the only way the information is going to come through in a in an Adaptive way is that you uh you compress and you generalize. So and that and that forces the generalization by not by by putting a layer or or several layers in the middle that are um that are uh very um very very thin in terms of how much information they can propagate; you're forcing the system to generalize; it's a bottleneck that that that that requires you to uh uh to learn Concepts and not try to not try to remember individual details. And so that's very important for intelligence because the whole point is that you should abstract patterns and what's happened before and apply those patterns to scenarios you haven't seen before. So so that so that architecture has been used in a few different um in a few different systems, but what I do think is new in this and and there's another there's another paper related to this um that's that's come out; well it's a it's a preprint that that came out recently by Kevin Mitchell and Nick Cheney that also looks at this at this concept where uh we can now use that architecture to understand what's going on in biology; the idea that these really are I mean the I I I I think there's two fundamentally um unconventional things in this in this paper, like two big themes; one is the Symmetry or the invariance between cognition and uh and and and development. So um development broadly speaking, the you know morphogenesis and so so the idea is that uh there's a reason you know uh why um uh there are there are there are deep deep in deep fundamental similarities between how you construct bodies and how you construct minds. And so what I'm after is the what those those principles and what is what is happening with the information that requires morphogenesis to be an intelligent process and how that works during on an individual scale but also on an evolutionary scale. I mean that's the other nice thing about this is that you can apply this to whole lineages; you can apply this to an individual being uncertain about what their memories mean, or you can apply this to a um uh an evolutionary lineage where you come into the world and you have this DNA, but you know you you're going to need to reinterpret it, and this is why by the way this this is this is um something that's going to come out in the next uh in the next uh month I guess or so uh is some of our work on transcriptomics in um in xenobots and anthr robots. So the bottom line is that they have a they have a r both of them have a radically different transcriptome than um than the tissue of origin in Vivo, and so the G DNA is the same, but in your new environment and more I mean the environment's not that different actually; the environment's almost the same; what's different is your embodiment; you have a new shape, a new a new way of getting around, a new a new function, new behaviors, and how are you going to use the affordances you receive from Evolution, all the DNA and all the the other uh cytoplasmic components that you have, how are you going to use them for your new for your new life? And so that's so that that I think that that invariance and that that scaling across space across time, the the the movement of of Concepts from from cogn and behavior onto the construction of bodies, you know morphogenesis, so I think that's that's kind of that's kind of new. And also uh this this idea of uh the information that moves through these um these these kinds of B eyes as being potentially uh an agent too, which means asking ourselves what does the world look like from the point of view or from the perspective of that information. That's interesting; some perspective information. I mean I think a very it's just like comical, but going through it to a bottleneck, right? The information I mean as if it had a perspective or as if it could see, but um imagine that being very quite scary actually to be condensed down and compressed down, but then of course there's there's the way through and then more expansion on the other side of it. Yeah, so right so so I think I think scary is the is the right term because this and and you know I not to get into matter that are kind of above my pay grade, but but uh it it does sound like a lot of things that um people who and and I've had a lot of um contact from from people who work in therapy and psycho Psychiatry and and and on on these kinds of ideas, this idea that you're going to go through a bottleneck; what comes out on the other end if you if you want to compare details, it's not going to be you because you're not going to be the same on the other end of it, um but that's the price you pay for Improvement, for learning, for growth, for projecting into new problem spaces, for creating new meaning and so right so so that is scary, and it's especially scary if you commit to a kind of um uh object permanence with respect to the self. So if you think you are a stable thing then yeah you're in trouble because no matter what you're not going to be here for very long, but if you have a more processed view where what you are are a kind of pattern with certain features in you have the ability to shape those features over time, and that already means that you the old you is not going to be here, but you get to shape the new you and and and but the environment is going to try to shape you as well, so there's some some tensions there, but but yeah yeah I think I think I think you're exactly right; it's this it's it's back to that same Paradox that that that kind of architecture gives you the plasticity and the intelligence to uh to adapt and exploit other uh other Realms and other domains of of activity and and and so on, but that means you are not going to be the same, right? There also this brings to mind of course it's a specul very speculative idea uh the idea of like white holes, black holes and white holes on the other side of them, and I see this pattern; I'm I have the paper in front of me, and I can't help but think of that connection there potentially. I know that's way out there um sci-fi land here, but do you think gives any Credence to that idea or it's it's interesting? I mean I I don't know; this is this is the kind of thing you know Paul Davies or somebody would probably want to want to talk about that.

Too. So I, I don't have the physics to know what's supposed to happen to patterns as they as they go through a wormhole like that, but um, I I don't think it's crazy to to ask the question, uh, with respect to, I mean, especially if, okay, under under normal under sort of conventional theories, you wouldn't expect anything like like that. Uh, you you wouldn't expect there to be any reason why the the these things would map on to to that to that Wormhole scenario. But if you buy into some of the um evolutionary universes approaches, like I think Lee Mo and hwood and some um, then then it becomes uh, I think perfectly a perfectly reasonable hypothesis to say that the same dynamics that led to this bowai architecture in the biological world, if if those Dynamics exist at the scale of of whole universes, maybe maybe maybe they give rise to exactly the same kinds of information Dynamics through the through these wormholes. I mean, this is like way beyond you know, anything I know as far as realistic physics, but um, I I I think I think if you take the longer view of like some of those some of those some of those models, then then I don't see why not. Yeah, it's fascinating.

The other thing too, I'd love to, a big topic um, in the paper you talk about confabulation and actually would you mind potentially defining confabulation uh for the audience? And you talk about in the paper a bit, but I just wanted to get like um, can we think about it like as being like hallucinations with AI? I know that's something that's brought up or I've heard that term discussed with AI uh before. Is that the same kind of idea? Are those different? So I I I I okay. Um, I I don't have any reason right now to think that the kinds of phenomena that we see in uh in in current language models are the same have have the same origin where where the confabulation has the same origin as it does for us. I will put an asterisk there that we can talk about, which I think is that we have to be very humble about our claims of even though we write these things and we make them and whatnot, for for a number of reasons that we could talk about, I think we have to be um quite agnostic still at this point about what's actually going on there, but um but but the end point is is actually I think quite quite similar, which is the desire to uh or the or the functional drive to uh output behaviors that are um more adaptive given current circumstances versus the circumstances that gave rise to them. So an allegiance to saliency and adaptive quality, not to history or veracity or uh or Fidelity of the data. And so so so let's just let's just Define um what what we mean when we talk about um confabulation.

Um, here are some here are some examples that uh that that people have found in in human human patients. Uh, some of the some of the um earliest ones were from split brain patients where you sever the Corpus colossum, and so you have the the you know, there's a speaking typically there's speaking Hemisphere and one that doesn't, but but the one that doesn't is operating half of the body. And when that half of the body does certain things, uh the the speaking half makes up stories about what's going on there, even though we know. So so so so so we can put, you know, we can you you put a piece of a piece of cardboard between the eyes like this and you show one side of the brain uh some kind of thing and then you ask the uh the the opposite um you ask the opposite hand to pick out a relevant object and then you ask the language speaking side, hey, why did you pick up this object? Well, it has no actual idea because it did not see the prompt, but it'll come up with some story that vaguely makes sense, and it doesn't feel like lying to the subject, it just feels completely natural because we are driven to make stories about ourselves in our world that make sense. That's a that's a fundamental thing. And um uh it's another example I can think of uh there's a there's was a video on um that I saw where a patient had a um he had an electrode uh I think it was for epilepsy um in his brain and it happened to be uh touching a region that corresp that that induces laugh laughing behavior. And so the uh the scientist pushes the button and the the person's mouth starts laughing and then you ask then then he's asked so why are you laughing and the answer isn't gee I don't know I was sitting here thinking of serious things and suddenly my mouth starts laughing. That's never the answer. The answer answer is Oh, I thought of a funny joke. And and again this isn't this isn't them trying to fool anybody, this is this is what it feels like to them because because because all of us are trying to uh uh continuously modify our models of ourselves in our world to make it to make things make sense. And so so that that kind of uh basic fundamental feature I think is really important. Now, when it goes too far, when the Horizon gets really short and you lose track of long-term patterns, then that's not adaptive either because because then you end up with explanations that have maybe immediate value but in the in the long run they're you know, and and I think this is functionally I think this is what's happening with these language models. They they tell you something at the moment that is plausible of the kind of thing you want to see, but but big picture they're not if they're not tied to what's actually if if they're not good at reinterpreting the past, then then this then this doesn't work. I mean, it's a deep skill to be able to do that. So so I think it's I think it's fundamental, but I think we we still don't understand. I mean, the biggest mystery to all of this is like the one the one thing that um you know, I've hardly uh cracked this this deep issue here, but all I've done is is is draw attention to a new way of thinking about it. The Deep issue is how the creative interpretation actually works when you're handed these engrams, decoding them in a way that uh that is adaptive. That's really important. I think if we understood how that works, we would have much more insight um into into ourselves, but also into new uh computational Frameworks that would do a better job than than than our current efforts. Okay, interesting.

Okay, so it's a little different than it's a little a skew from what I was thinking of originally. Are you familiar with it all with um Greg henriquez? He's a psychologist at James Madison University. Yeah, this reminds me a little bit of his one of his course Concepts around uh that we're self-justifying apes. And so when you say something like yes, one part of the brain is explaining something that has no access to the thing that we're doing all the time, much of the time is is is sort of justifying stuff, is that right? Or is am I I mean I mean yes yes, I think I think there's a lot of value in that, but also um and I don't remember who said this, it might been Yuval horari Harari or somebody said that you know humans are fundamentally storytellers. Like yes, but this isn't just about humans, this is about all agents. So all good agents, if if if you're not a good Storyteller in a in a in a primitive way, right, meaning making models making actionable models of yourself in your outside world, you will never get out of the Single Cell phase. In fact, in fact, you will not survive as a single cell. And I don't think you'll survive as a persistent you know, chemical pathway either. You this this this storytelling uh you know, by the time you get to humans we call it storytelling, but but that fundamental thing that active inference kind of loop that causes you to to store some priors and to try to figure out what exactly is going on in a way that is going to allow me to make the decisions which are you know, are coming up very you know, constantly, you have to right in in the real world you're going to run out of energy and die and be eaten very quickly if you're not constantly taking actions uh that requires modeling and that does not wait till we get to human stage. That was there from day one of evolution and possibly before that. H storytelling are there any let's see I'll say patterns or huh in terms of these information patterns, do you notice any patterns that resemble anything like storytelling Frameworks or elements of Storytelling that we are more familiar with as humans?

Yeah, well I I I do think it would be a it would be an amazing project for somebody uh and and you know maybe there's there's a few people I've I've talked about this around here that that might want to do it is to take something like Joseph Campbell's um righty of of of archetypes or or you know these kinds of things and try to recast them as what what what do those things look like for single cells? What do they look like for Pathways? And and I mean here's right I like I think I think that would be that would be completely fascinating um but uh you're speaking my language, that's like amaz that that is a great idea. Yeah, I think I think that would be really interesting. And you know like like one pattern I mean I'll I'll tell you one one thing that I can think of right off the top of my head uh and I'm no I'm no you know expert on on myths or anything like that, but but but one thing that I think is really fundamental is uh seeing agency in the world, telling stories about agents doing things I think is really critical. And here's why if and and I think and I think any any real realistic agent that that evolved under constraints of energy and time is going to need to do this because again just to come Circle back to the beginning if you you you as a living system that is vulnerable you know these mortal computations as as a few people have called them um you do not have the luxury of being a lassan demon that says I don't believe in mesostates or large scale pattern. I'm just going to track microstates. All I care about is every particle every atom I can measure and and I'm just going to track them. Okay, well you don't have time to do that. You you'll be dead if you if you try that strategy. So the only things that survive that filter are agents that are good at core screening. So what they do is they say okay I'm going to take a bunch of these these all of these states I'm gonna I'm G to ignore what's different about them, but I'm going to establish a category that I'm going to treat them all the same way. I'm going to generalize and I'm going to say all of this is you know that it's this this is hunger or this is a chemical attack or this is danger or this is stress or this is um you know what whatever right and so and and and then later on it's like oh this is a tiger and I don't care if the pixels are this way or that way or there's lighting shadowing you know and so on so um you have to get good at it in order to survive. You don't have the computational uh resources to to avoid that and and that leads to that that kind of thing. If you become an agent that is constantly telling stories about other agents in the world doing things that then leads to you making models of them. Well, how do what are the properties that these agents have? Do they what do they notice? What do they you know can I hack them? Can they hack me? Are they dangerous? Are they positive whatever uh and then eventually you turn that on yourself and you say wait a minute I'm an agent too that does things and now you've got a model of yourself as This Magnificent you know uh Moral Moral being that that exists right this kind of self-reflective loop. But but you can but you can in fact must do that long before you're capable of of doing that um that self- referential you know that self- referential Loop. So I think that one you know uh that fundament that that concept of an agent in the world doing things not just not just uh standing back and saying here's a bunch of stuff that happened and I've totaled up all the atoms that zigged and zagged and whatever, but but but I've I've course grain them into a into a some kind of a larger scale pattern where the pattern makes decisions, it has memories and by the way it pays off if I try to make a model of well what kinds of things does it remember for how long? What does it like? What does it not like uh some level of you know uh some level of of predictive description that I think is is extremely [Music] fundamental.

Yeah, and even well and I think about it in terms of more broadly the the effectiveness of great storytelling and how yes you might have a con concept a complicated thing in physics right, but someone a great educator can hone it down and break it down and have an anecdote about it and maybe give some personalities to the quirks and electrons so that we can kind of glom on to certain things and we interpret those and we kind of make sense of them in a better way even though they don't have personalities per se. Um I like to think of about them as having personalities or having yeah not not personhood but um attributes let's say or or things that um like I can understand um I'm kind of talking in circles here but yeah yeah it's a really cool idea. I like that. Yeah, I mean it makes what what you just said makes makes sense for in two ways. One is that like in that paper we give a table um or or I give a table of things that act as uh as these bow ties and uh scientific papers, language, literature, all of those things act in this bow MTI fashion. You have some very complex mental states, you have some very complex um synaptic you know molecular States. There's no hope of you communicating those States and mapping them onto my brain so that we understand each other. My brain is different, it's not going to work anyway. What we do have is a very thin interface at you know, however many bits per minute we can do it, that's language that allows this very complex set of events to come forward and to then in then to be expanded in my brain into whatever it takes for me to understand whatever part of what you said I understood, you know, whatever whatever I'm supposed to get out of it. And so the same thing with science papers right, so these brilliant people have these incredible ideas, they boil it down to a you know to a nature paper that's like this compressed you know squee a couple of pages representation and then all everybody else the rest of us read it and try to reinflated it and say well what does this mean? What am I going to get out of it? And and right so so a lot of these things are are like that. Yeah yeah great. I know you have to run. I just want to ask you one more one last question. Um So based on the conclusions of this paper uh are you going to follow us up with any research? Do you have anything uh what comes after this basically?

Yeah yeah we yeah we have we have a ton of stuff. So so well conceptually uh what comes off after is some computational work that we're doing to tie um uh the uh the poly Computing framework into all of this and so um a Tusa parksa who uh was was a student with Josh bonarden was responsible for a lot of the actual primary work on the poly Computing. I mean she she drove the the early poly Computing work um she's she's now in my in my group and we're going to uh basically turn this whole thing into a a model of an evolutionary model a model of for a new type of um you know computational platform and so on. But then but then the biology of it uh becomes really a search for the mechanisms of this creative and a so when you do have uh when you do have these engrams what are the mechanisms by which those get mapped onto whatever the novel um scenario and the novel problems are and that's that's something that we're using in our um synthetic morphology models like anthr robots, xenobots and some other things that will come out this year where we can actually start to ask because because they're I mean one reason for making those things is that there the problem is is stands the starkest. You're you're you're a zenbot, you've been given some DNA was that any of that DNA about how to be a zenbot? Most of it wasn't at all. There's never been any zenbot. There's never been selection to be a good zenbot um you know there's some physiology in there that you have in common but but but a lot of it and the same thing the same thing for the OTS when you one of the values of making these uh synthetic models is that you force them to break free of a specific evolutionary history and then you get to find out how do they reinterpret the the affordances that they've been given um and where do these novel patterns come from? They have new patterns of behavior and and so on so so memories that were not specifically encoded for them right because if you I mean just think about this if if you have the ability to reinterpret these little these little um engrams that you've been given, that creative ability you can now nucleate that off of a lot of other things you know you could there are lots of prompts that that you can apply that same capacity to it doesn't have to be the same the same materials that you were given before. If you have the ability to interpret specific memory and grams that were used for Behavioral memory you can turn that capacity onto almost anything. So I almost visualize that like the right side so you get this bow tie you know they meet in the middle. I almost like visualize unmoral and disease and and and so on. Oh so cool. All right Mike, thank you so much for your time. We we want to cover three papers I want to but we got we got deep into one which I think is I I prefer that in terms of going deeper into one thing than glossing over maybe three, but maybe we can talk again and we go into the uh stress sharing paper and I can reach out to uh to you and Emma and uh sure sure yeah yeah yeah yeah got some time uh and we'll we'll we'll talk about the other stuff. Yeah, no problem. Great thanks so much Mike appreciate it. Thank you. Good to see.