Transcription
Okay, so multi-scale logic of collective intelligence, and it's what we call the recursive trace logic. So, we've had the trace logic for a couple of years, but in the last couple of months, we discovered a recursive aspect to it that will lead into the notion of agency that's novel. So, this is different, Chris, than the conscious agent theory. It's a different notion of agency that we've had before.
Um, so, the big topics I'd like to—Can you guys see me? Yeah. Okay. So, so I'm going to talk a little bit about, um, collective intelligence, our model of collective intelligence, how it involves coarse-graining, which is important to you guys, how it involves generative models, minimizing surprise automatically, bending problem spaces, a recursive notion of agency and self, a new intelligence metric for agents we'll call lambda sub two, and its relationship to your measure K, and then, um, how this is all beyond space-time and quantum theory. And I'll start there, just briefly, about why I'm thinking entirely outside of space-time and quantum theory.
The idea is that high-energy theoretical physicists have are done with space-time. They say it's not fundamental. So, here's Nima Arkani-Hamed at the Institute for Advanced Study: "Space-time is doomed. There's no such thing as space-time fundamentally in the actual underlying description of the laws of physics." And he makes it very, very clear that he's saying, you know, space-time and anything inside space-time, and that includes anything with unitary evolution, he, quantum theory in particular. So, he's going beyond space-time and quantum theory. And it's not just him. It's because of his success and his collaborators, the ERC has funded a 10 million euro initiative called Universe Plus, and it's all about going entirely beyond space-time and entirely beyond quantum theory, looking for what they're calling positive geometries. And so, there are over a hundred high-energy theoretical physicists and mathematicians now working on this, and they're finding stuff. And I can talk a little bit about how it's related to what we're finding, but they're finding these positive geometries that give you scattering amplitudes without any quantum theory whatsoever. And you get it much more easily and more simply than with quantum theory. So, it's really quite striking. So, so I'm stepping entirely outside of space-time.
Yeah. Sorry, just a quick question. Maybe maybe naive, but this, I just want to understand this idea of space-time being doomed. So, on that view, if that were correct, what is the status of, let's say, general relativity? What, what does it refer to? You know, is it completely to be supplanted? Like, what, what is that theory about then? If—
Right. So, the idea is that the very notions of space and time, even the combination of them as space-time, is not fundamental at all. So, general relativity will go the way of all theories. It will be, you know, like Newton, we still use Newton for certain cases. We'll still use GR for certain cases, but we need a much, a much deeper theory.
Got it. The, the hard fact is that when you bring together GR and quantum theory, you find that space-time has no operational meaning at the Planck scale, 10 to the minus 33 centimeters, 10 to the minus 43 seconds. It simply has no operational meaning. So, that means we have to find a deeper foundation. So, these are only at best approximation theories.
So, so Don, just to be clear, in the, the obviously content vein in which I know much of your theories, this is not only dynamiting the space-time at the empirical level, it's also ejecting it from any transcendental style considerations. It's, it is just a, until until we have something better, just the placeholder, but it will, it will die as a concept in our economy of thinking, even about our experience, let alone the empirical physical world.
Absolutely. That's the idea. We thought space and time were the fundamental nature of reality. We might have even [clears throat] thought they were a priori true or something, but that's just wrong. All right. That's just plain wrong. And science has a way of forcing us.
Yeah, if I could jump in here just to add one comment. Um, if one formulates basic ideas of quantum theory outside space-time completely, then there are many routes that are under study by huge numbers of people again, sure, for generating space-time as a consequence of, um, basically assumptions about in quantum information theory. And also many, many routes for generating Einstein's equations as either approximations or again, outcomes of other kinds of assumptions. So, GR turns into something like the status that classical physics has with respect to quantum theory in space-time, which is a limiting case, you know, something an approximation that's good in some circumstances for doing some things, which is basically how Don just characterized it. So, yeah, there's lots and lots and lots of physics underlying this, both in the high-energy community and in the quantum information community.
Right. And and what Nima and the ERC group are doing is even going beyond that because they're saying we're not going to even start with quantum information theory. Anything quantum itself is going to arise joined at the hip with space-time from something far deeper. So, so they want to show quantum information theory and general relativity arise together from something that couldn't care less about unitarity at all. So, that's that's what they're after. So, there is no locality and there is no unitarity, period, in these new positive geometries. And they don't care about unitarity and they show that then quantum information theory comes out as an approximation and special case at the same time that you get space-time. So, that's so it's different than the, you know, Carlo Rovelli kind of approaches and so forth.
>> [clears throat and coughs] >> So, so and that's the approach they're trying to—
Should actually say that it contradicts most of quantum information theory because quantum information theory actually has nothing to do with space-time. So, the two arising together would be very unusual.
Well, what Nima has shown, wants to show, is that unitarity and locality together arise from these positive geometries. And then because you get unitarity arising from it, then you get the foundations for quantum information theory. So, that, but, you know, we'll see. You know, the proof is what if you can do it, right?
Um, so, yeah, that's I'm just trying to be clear about where they stand with respect to current approaches to trying to build up. As you, as you say, Chris, most of the approaches that are trying to build space-time are starting with something quantum, and these guys are not. They're saying we're not even having quantum. We're starting with what they just call positive geometries.
Um, so, so that's just want to make clear how out of the box they're thinking is. So, John Wheeler, of course, was trying to think out of the box, and he was saying, you know, "Someday," this is 1990, in his wonderful book on gravity and space-time, he says, "Someday, surely, we'll find we'll see a principle underlying existence so simple, so beautiful, so obvious that we'd all say to each other, 'Oh, how could it have all been so blind so long?'" So, that's what we're looking for.
And let's see. I'm not able to—Can you guys hear me?
We can hear you, yeah, but the slides are not advancing.
Yeah, let's see. Okay, well, I guess it now it advanced. Okay. And he said about the same time in his famous book—
Don. Sorry, sorry. Ours, we're still, we're still seeing the title slide.
Okay, let me try this again. We, we only saw the title page so far.
Okay, I'll go back and try the—
That's weird. So, I'll—
Let's see. So, I'll go back to share.
Sorry about that.
No problem. Hm. That's weird. Let's—Can you—Can you see that?
We can, but it's still in, um—
No problem. Yep. Okay. Yeah, there we—Okay. Okay, so—
Right. So, Wheeler suggested that, you know, the note struck out on the piano by the observer participants of all places and all times, bits of the are in and of by themselves constitute the great white world of space and time and things. So, he was trying to start with what he calls observer participants. And he thought that maybe somehow that was in his "it from bit" paper, 1989. And he actually, in his paper, cited work that Shaytan and I were doing, our book, Observer Mechanics. So, he was already thinking about the stuff we were doing with observers and participants back there, back then.
So, what's a minimal observer participant? I'm going to have—we're going to start with just the absolute bare basics. They have experiences like smell of garlic, taste of mint. And these experiences can change. That's all I'm going to assume. That's the foundation of everything. So, my ontology is: there are experiences, and they can change. So, for example, maybe I have four experiences: a very, very simple observer, red, green, blue, and I'll call that yellow. And they change. So, now I'm seeing yellow. Now I'm seeing green. Now I'm seeing blue, and so forth. They keep changing.
So, a simple, in fact, the simplest and most general way of talking about that is just to talk about Markov chains. So, the Markov matrix there, the first row has, you know, that means if I see red now, what's the probability I'll see red next? The point three is, if I see red now, it's a three-tenths chance that I'll get a 30% chance I'll get green next, and so forth. That's just—so, it's just a transition matrix, probability of seeing the next color given that I'm seeing the current color. So, that's all Markov chains are. They are these matrices. Of course, a lot of complications come out of that.
And one aspect [clears throat] of Markov chains is that they immediately estab- instantiate a very interesting kind of goal-directed behavior. No matter how you—what state you start the Markov chain in, it has a target stationary measure. In this case, it's the thing on the left: point three, three, point three, zero, point one, six. No matter what state you start this matrix in, it is going to go eventually to that state. So, and you can perturb it as much as you want. It will resist the perturbation and head back to that target state. So, already we have in the very structure of this a goal-directed behavior.
So, and as, as you guys in your papers talk about, you know, William James mentions that intelligence is having, you know, achieving a fixed goal with variable means of achieving it. So, so that's the stationary measure, and if you all have an ergodic Markov chain, then you will have a stationary measure.
Now, the idea is I want to have multi-scale collective intelligence, and so we need a notion of scale. So, I'm just going to take an observer that sees a subset of the states that this—so, the first observer I was talking about has four colors it can see. Let's consider one that has only two. So, that's my notion of scale. How many, you know, the subset relationship among the number of experiences that you have.
Now, so here's the key idea of everything we're going to be doing now. Suppose I take the matrix on the right as describing, quote unquote, the reality. This is what's happening. And but and those are the transitions. But this observer on the left only sees two: red and green. What transition probabilities is it going to see? Right, there should be a formula given the matrix on the right, there should be some kind of computation we can do to give us a two-by-two matrix for the transitions on the left of the just in red and green. Does that idea make sense?
Yep. Okay, good. So, when you do the mathematics, it turns out that's the matrix. It's—so, you get this very two-by-two matrix. Notice that the numbers are completely different from this matrix, right? It's not, you're not just copying. It's a computation that you have to do. And so, here are the two matrices, you know, the one on the right is the big matrix, and if you just restrict attention to the red and green, then you get the matrix on the left, induced by the matrix on the right. And this is called the trace. So, the matrix on the left is called the trace of the matrix on the right. That's just standard in Markov theory, that's been around for more than half a century. So, this is not new to me or to us.
Now, this, you can actually—the trace formula is important. I'm going to go through it because it has an important conceptual thing for us. So, the way you compute the trace. So, I got—I'm going to take this matrix. I want to—I'll call it matrix P, and I want to, you know, get its trace on the red and green. So, first, I'll just notice that we can take this matrix and divide it into four submatrices. Um, there's a two-by-two matrix, you know, that has point two, point three, point five, point two. That's for the red and green and so forth. Um, that we'll call matrix A. So, that's going to be the states that are visible to the trace observer, right? So, A is the submatrix based on the states that are going to be visible to the sub-observer. C is the submatrix, um, relating states that are dark to this new observer. It doesn't see this. So, this is all dynamics that's dark to it. Okay? B is the matrix that is the exit. This is the exits from what you can see to the dark region. So, B is the exits, and D is the re-entrance. This is getting from the invisible world into the visible world. So, those are the submatrices that we're going to be using. And here is the formula. This works universally. The trace—so, the trace matrix on A, which is, you know, the visible states, is you just take the original matrix A. So, point two, point three, point five, point two. And you add this interesting thing on the right. That I is the identity matrix. So, you take the identity matrix minus the dark matrix. So, I minus C is the identity minus the dark matrix. And you take its inverse. That has the effect of being able to explore all possible paths. There's an infinite number of paths through C that you could take. So, you allow I minus C quantity inverse is exploring the infinite number of paths there, and then you pre-multiply by the exits and post-multiply by the entrances. And you add that all up, and that's your trace. So, that's the idea. You're basically—you get the trace by looking at all the ways that you can go outside of the trace and then coming back into the trace, the trace states, okay? That's the general formula. So, that's been around, again, that's not us. That's been around for a long time.
So, you have hidden memories and controls. B, C, and D are going to be hidden layers of control that the, that the agent A cannot see, but will be influencing their behavior. So, that's going to be an interesting hidden memory kinds of possibilities now with B, C, and D. So, there's explicit memory changes when you change A directly, but then there's going to be hidden—
Just, just one second. Can you please go back? In, in this B, C, D, so the exits, the entrances, and the invisible, is there any particular mapping to memory or control, or is it more of a blanket category you're using, hidden memory or controls? Like, is memory, for example, C, the dynamics, hidden dynamics in C? Or that's, or what's the control here?
I suspect the exits and entrances would be more like control.
Well, it, it looks—it turns out that you, if you, there's different ways to control. You can screw around with B, you can screw around with C, screw around with D, or all of the above in any combination you want. All of them together give you different ways of controlling. So, it's really quite fascinating the possibilities here.
I'm going to go. Thanks.
So, all of that is old. Here's the new stuff. We discovered just a couple of years ago that the trace relationship gives you a partial order on all Markov chains. That was the discovery. And that's what sort of launched this whole thing. So, it's a partial order, which means that there is a logic. So, the, so, so the definition is that the a matrix M is less than or equal to a matrix N in the trace order if and only if M is a trace of N. That's it. One trivial definition, but no one saw it before. It turns out that that definition gives you a multi-scale logic of minimal surprise. And the reason it's minimal surprise is because the trace is the zero surprise view of the bigger matrix. That's the key idea. It is the zero surprise subset matrix view. And the—we'll talk about the stationary measures as well. The stationary measure is identical to the—it is a normalized restriction of the original stationary measure. So, you have minimal surprise in the dynamics, in fact, zero surprise in the dynamics, and in this, in the stationary measure, again, zero surprise. So, the trace logic is the logic of minimal surprise for arbitrary dynamical systems. So, that's the power of this, because we're minimizing surprise, which is of course key to intelligence, a key to intelligence. But this is multi-scale. So, this is the multi-scale logic of minimal surprise.
So, what about this trace logic? The set of all Markov chains form a non-Boolean logic under the trace order. It's, it's non-Boolean. Um, that means that there's no global top, there's no global negation, there's many matrices do not have meets and joints or hands and ors. However, so it does have a notion of meet, join, not, and entails generally, but many matrices are not compatible. So, they may or may not have meets and joints. So, it's a very, very complex logic. However, if you take any particular Markov chain P and you look at all of its traces, they form a Boolean sub-logic. So, I can pick any Markov chain I want, anyone at random, look at all of its traces. All those Markov matrices together form a Boolean logic. So, the notion of and, or, not, um, are completely well-defined. And this Boolean logic has two to the N members. If there's N experiences, then there are two to the N members in this Boolean logic and traces. So, so that's—if you think about it, all we've got right now are—we don't have agency yet. Although you, I showed you that that notion of goal-directed behavior, which is sort of like a proto-agency kind of thing. Already these matrices are going toward their stationary measure, no matter how you perturb them. So, already there is this interesting notion of some kind of agency going on there.
But now, here's the key idea. Here's the—and this is only now two months old, this idea. And that's why this, when I had this idea, I realized that it was time to talk with you guys. Once we have the trace logic, I've talked about is a logic on observer windows. So, it's the, the, the space, an infinite space of all possible observer windows. There's this minimal surprise logic on all of it, the trace logic, cleanly well-defined. Now, how do I want to model agency? And this is, this is the new idea, just in the last few weeks. Agency is a matter of changing which window I want to look through. I want to have a policy for how, if I'm looking at the world this way, then how do I want to look at the world next? And how do I do that? Well, another Markov chain. The Markov kernel will say, what's the probability if this is my current window, that my next current window will be such and such. The way you write that down is again a Markov matrix. So, what we have is a policy is a Markov matrix on the trace logic itself. So, the trace logic is the entire logic of minimal surprise on possible conscious observers. That's what it is. And the first step of agency is to say, let's crawl along the trace logic. That's the first baby step in agency. The first ability to crawl along the trace logic.
Now, if we look at the collection of all—I'll call those Markov kernels policies. Each Markov kernel is a policy. It's a, it's a first order of agency. And since they're Markov matrices, they satisfy their own trace logic. So, we now have a trace—we have the first trace logic of observer windows. Now, we start crawling on that trace logic of observer windows. That's our first layer of agency. It has its own trace logic. That's its own—so, let's call this recursive trace logic. It's recursive now. And you can see we can do this ad infinitum. Once we have the trace logic of policies, I can now crawl on it and get meta-policies. And so, I can take agency to whatever layer of complexity I want. We can start with the baby layer. We can start with just the observer windows and explore those, then study policies, and then meta-policies, and build up recursively to ever deeper notions of agency.
So, just at top level, we can think of a policy is simply a path through the trace logic of observer windows. That's that's the simplest case, right? So, I had—I started off with a three-state window, and maybe I move to a two-state window, and then I move to a one-state window, and that was what my policy was. And so, I've got a Markov kernel that does that. Um, and then a meta-policy would say, I've got thousands and thousands of policies. I now have the flexibility to choose my policies based on whatever goals I might have. So, policies can model attention shifts, scale shifts, reparameterizations. It can, um, maybe describe a subsystem that I think is now driving my future decisions, my policies. So, the recursive trace logic is the collection of all policies with their trace logics, and then recurse, recurse, recurse again. So, it's a whole hierarchy of trace logics. Each trace logic itself is infinite. So, we have a choice of policy, meta-policy, meta-meta-policy, and so forth.
Um, [clears throat] so, we've talked about stationary measures, um, and, you know, they're sort of a minimal kind of notion of goal-directed behavior. Um, we can write down a simple intelligence metric, um, based on, um, Markov chains.
>> [clears throat] >> So, it turns out that for any probability measure pi, there are many Markov chains for which pi is a stationary measure. Okay? So, if you specify a stationary measure, um, and you ask, what, what is the Markov chain that has a stationary measure? That's the wrong question. There's an infinite class of Markov chains that will have that stationary measure, and they vary in very interesting ways. They, for one, they have different rates of convergence. So, some, some will have this goal-directed behavior where they're going almost immediately to the goal. No matter where you start them, they will go almost in just a couple of steps to the goal, and others will converge very, very slowly. So, we get to choose, in, in the trace logic, we can choose how quickly we want to converge to our goal. Right? So, this is very going to be very interesting because search efficiency is, of course, your measure K is, is a model of intelligence. So, we have a dial here that we can dial the intelligence. And it may be that, you know, you might be have high intelligence with respect to a goal, but there may be some sub-goals. It turns out that if you, if you go quickly to your, this, this stationary measure, you may not do other things intelligently. So, we're going to be careful which Markov chain we choose depending on what, what goals we're trying to get to.
So, there are many goals that you can get, and I want to talk about that and its possibilities. So, there's differing rates of convergence, and the convergence rate is dominated by lambda two, lambda sub two, which is the largest eigenvalue of the Markov matrix. You take the Markov matrix and do its eigenvalue analysis. The, the largest eigenvalue has value, eigenvalue has value one. Um, but then you find the largest eigenvalue that's less than one, and that gives you that, that pretty much tells you the rate of convergence for, um, that particular Markov chain. So, there are Markov chains with different lambda twos that all have the same stationary measure, and so they converge to it at different rates.
Um, so, there is then a connection between this Markov notion of intelligence, which is the lambda two convergence, and and your your your metric, which is K. And the, the relationship is just a simple equation.
Where T sub M would be, um, your, would be our lambda, essentially our lambda two, the rate of convergence. And T blind would be, say, you know, just a random walk that does not, that's not smart.
Yeah. Right. So, so there is a deep connection. But now, here's a little trick. We, we want to have, um, what, as you guys talk about, you talk about different layers. It's hierarchical, and higher layers can bend the, the geometry of the problem space for lower layers. And so, how do you model that with Markov chains? Well, it turns out, um, there you can have lots of different so-called community structures. So, again, for any stationary measure pi, there are an infinite number of Markov chains that have pi stationary, but that have differing community structures. So, now, community structure is roughly, is like, is like you probably know about it, but I'll just say briefly. You could have, you know, thousands and thousands of states in this Markov chain, maybe a few hundred are tightly connected over here, a few hundred are tightly connected over there. There's just a few cross-links. So, the whole thing is ergodic, but basically, you might have like 10 communities that are tightly knit. Now, within each of those communities, maybe my hundred-state community, if I look at it more closely, it itself is composed of maybe three new sub-communities. In other words, you can have an infinite number of communities, sub-communities, sub-communities, all the way down as far as you want. Um, and all having the same stationary measure. So, what this gives us is you might have one big goal, reach the stationary measure, but you could have sub-goals, which community. The way you get there is the different communities that you might emphasize as you go down. So, it gives you this multi-scale flexibility. And, and the community structure turns out mathematically is dictated by the eigenvectors when you do the, you know, analysis of the matrix again. The eigenvectors with eigenvalues close to one, because they involve slow mixing between communities. So, the communities themselves mix inside themselves, but they don't mix between the communities very much. So, we have state, we can have policies then that are trying to, you know, focus on stationary measures, community structures, convergence rates, particular dynamical models, and so forth. Um, so, policies can be looking at all these things and trying to optimize. And then the meta-policies can explore different policies. We can have meta-policies and meta-meta-policies exploring at different rates. Um, so, this starts to give us a recursive notion of agency. And in some sense, the reason I'm bringing this up is here is a framework of mathematical tools that give, that's incredibly simple. There's one definition, the trace. That's the only mathematics there. And then there's one observation, the trace logic. And then the third observation is it's recursive. That's it. And then all the tools—that's it. And all the tools are at your disposal. So, meta-policies can explore different policies. And the deeper the recursion that we go in terms of making deeper and deeper trace logics, we get deeper and deeper notions of agency. So, we can actually explore just policies for our simplest notion of agency, and then go to meta-policies to discover, you know, deeper notions of agency, and so forth. So, we can take it one step at a baby step at a time.
Um, now, in terms of how does this relate to notions of Markov blankets and the self versus the world? Markov blankets [clears throat], as you will know, are strictly speaking defined for directed acyclic graphs. And there, you know, they define a boundary between self and the world. Um, and I want to upgrade these notions to Markov chains. Right? So, and the idea of the upgrade is, Markov chains are graphs, but they're not acyclic. They allow cycles. So, this is one upgrade. We're upgrading from acyclic graphs to cyclic graphs. And then we're upgrading to labeled cyclic graphs, namely labeled by the transition probabilities. So, that's what I mean by upgrade. And we're going beyond directed acyclic graphs to something that's far more general. So, we have to—so, the, so, we'll want to move from, you know, the standard notion of Markov blanket to what I would call a trace blanket. And here now, um, we have to actually construct the self and the world. And we need to, the way we will do that is, and by the way, now, you know, I'm just saying at top level, we have to do a lot of hard work here. But it's going to be policies and meta-policies and what they do. And certain experiences, like experiences of pleasure and pain, will be part of the experiences agents have. And to the extent that certain actions lead to greater hitting of the pleasure centers, the pleasure, the higher, put it, the higher stationary measure for the pleasure, then they'll be sought. And to higher stationary measures for the pain, they will be less sought. They will be avoided. And so, the idea will be there'll be pleasure and pain guides, but there'll also be, uh, I'm thinking that policies, what policies do is they say, um, given that I'm looking through this particular observer window, what's the probability that I'll now look through that window over there or that window over there?
If— >> [snorts] >> Don, can I ask you—Yeah. Can I interrupt with a question? A couple of sentences ago was the first time you used the word "action." And is an action in this framework just a change in policy?
So, each, um, it is, but it's not just a change in policy. So, an action would be, um, a policy itself gives you an action on observer windows, because your action is to change observer windows. Okay. A meta-policy gives you a higher level of action because you're now changing policies. Right? And then a meta-meta-policy would be an even higher level of action because you're changing your meta-policy. Okay. So, so actions, so actions are all either changing what you're looking at or changing how you decide what you're looking at.
That's right. Okay. Great. It's a recursive notion of action now. Okay. Right. So, so now this, I'm just thinking through this last bit, but it seems like some policies, for example, now, so now I'm looking at just the smallest level of action. Um, some policies, if they have certain things that you, that always appear in your observer window. So, for example, in my observer windows, my hands and my body often appear, whereas other things that I call the external world don't appear that often. And I also notice that I seem to be able to directly control my hands and my body. But if I want to have my phone move, I need to move my hand so that I can pick up the phone to move the phone. So, what, what I am going to say is that we, we really have to—so, in the Markov blanket approach, right? The, the Markov blanket has a clean definition. It's, you know, you give me a set of nodes, their blanket is their the parents of the nodes, the offspring of the nodes, and the parents of the offspring of the nodes. End of story. That is your blanket. That's your skin. That's your boundary between you and the world. Here, it's much more complicated. Now I have to use the notion of agency in a non-trivial fashion and learn probabilistically what features of my sequence of observer windows that I'm having remain there most of the time. My hands are there most of the time. And they're associated certain actions with my hands are associated with pleasure signals, others associated with pain signals. So, I'm learning to do certain things with my hands and don't stick them in the fire, things like that. Other things are much more contingent. They, so I can use probabilities of what I'm seeing in my observer windows as a way of starting to construct myself versus the outside world plus these pleasure and pain guides.
Don, can I ask you another question?
Sure. Sure. Uh, you, you talked about actions with your hands. What does that mean in terms of changing what you're looking at? Since the only action is changing what you're looking at, what does it mean to control what your hands are doing?
Right, because so that's that's a great question, Chris, because what that means is I want an observer window where my hand is touching my ear. Now I want an observer window in which my hand is touching my leg. And so I transition to that observe—so, what's happening is I'm choosing what I want to see in my movie next. And that's what we call moving my hand. It's a completely—you have to really think out of the box now. This is—it's really—it's a choice of what I want to see next. And that's what the actions are.
Okay. Okay. Great. It's very—it's in a very austere. What I love about it is it's austere. There's only one equation and one logic, and so you have very, very tight guides. And yet the claim is we should be able to get everything out of it. But that's what I love is a theory that forces you to do it in a principled way.
Now, Bayesian inference, I'll just—we can talk about more if you want, but I'll just mention briefly. Bayes' rule falls out of the meet, the and of the trace logic. And we can go into how that's the case. It's beautiful and non-trivial, but Bayesian inference is effectively a special case of the meet of the trace logic. And if you want, we can go into that. And, you know, you guys talk about bending the option space, and I want to say that, yeah, I'm taking the notion of space. Of course, that was metaphorical when you talk about bending the option space, but there is a real sense in which I want to get space and space-time itself. And what I'm working on quite heavily, and with with a couple others, is I believe that we can actually boot up special and general relativity entirely from the trace logic. And so, that's that's the claim that relativistic space-time can be constructed entirely from the trace logic. And this would be then fulfilling John Wheeler's goal that starting with only observer participants, that we can build up all of space-time physics. And that's that's the goal of where we're headed.
Um, and I'll just give you—this will be the last thing I do, and then we can have a conversation about it. Just to give you a hint about how that would happen. Um, it's standard in Markov chain theory to have what are called enhanced Markov chains. So, you have a Markov chain, but you also have a counter. Every time your experience flips, you know, you change experience, your counter increments. So, here I've got a case where I've got the, the four-color agent, and then there's the sub-agent of just red and green. And notice that each—there's a counter for the red and green, and there's a counter for all four. And notice the counter on the left is going much faster than the counter on the right because it's seeing more experiences. So, the counters go at—the counters for the sub-agents or, I'm sorry, sub-windows, sub-sub-observers are going at a slower rate than the ones above them. So, if I'm less than you in the trace logic, my time counter is going less than you than than the one. So, the trace logic also is giving you a, um, a relationship among counters, and that we claim is the time dilation of special relativity and general relativity. That's where it comes from. So, it's all about observer windows and their counters. And it turns out that the distances can also be derived. And it turns out that the distances that you will get in the window, the trace window, are different than the distances you'll get in the bigger. And so, we're, we're—this is where we're hoping to get general relativity coming out of this. Just simply there's notions of, um, essentially something like the commute time between states. Uh, and and similar notions. The commute time, I'll just give you that concretely. It's the expected time of starting at green, getting to blue, and then back to green. What's the expected number of steps that I, you know, starting at green, I'll get to blue, and then back to green. And it turns out there's that's, um, that expected time can be viewed as the square of Euclidean distance. So, there are, there are canonical ways of getting Euclidean distances from commute time properties. And, and other, there are Dirichlet measures which are even more to the point, but more complex. I won't go into them. But there are ways of going from, um, the trace logic gives us effectively the time dilation and length contractions of special and general relativity, is the idea. Um, so, time runs lower on the trace. Um, gaps between ticks are—So, so I'm—I'll just leave it at that. I think that's enough for us to—I'll stop the share so we can, we can talk about it. But I just wanted to give you guys a feel, and I can send you guys some papers on this, but I wanted us to have a little time to talk about this. This is just a—we haven't solved the agency framework, the agency thing. What we've got is a language now that's principled for talking about agency.
Yeah, thanks very much, Don. That was amazing. Um, uh, I have a question that I can—the general question. What do, what do you make of the fact that you're apparently pulling out descriptions of physics and descriptions of agency out of the same starting material? Does that—
I think, well, you know, something I've been saying for quite a while is that space-time's just a headset. And we're, we're effectively saying we can build the headset. Space-time is not the reality that's independent of us. Then we're little tiny, little—there you go. Our our typical view is, you know, Hoffman is this tiny little 160-lb thing inside of a massive, massive space-time universe. And I'm saying, "No, no, no. This that's what we call Hoffman is just an avatar inside a space-time headset that's being created by consciousness." And the proof of the pudding is, can we build the headset? Right? So, the idea is that that space-time, what we have for this approach to go through, we have to be able to show that we can get special relativity, no hand-waving, just from the trace logic. And also general relativity and quantum theory. We, we have to be able to show that we can get entanglement and all of this stuff simply from from the trace logic and Markov chain.
Now, one, one objection that someone might have is to say, "Look, Markov chains, they're, they're, you know, in quantum theory, we have unitary matrices. You know, what are you dealing with? You just have Markov matrices. You don't have these nice unitary matrices. So, how are you going to do that?" And, uh, the idea is, most Markov matrices are not unitary, but there are some that are. They are measures. There are a subset of the Markov matrices that are unitary. And when you look at the long-term behavior of a Markov matrix, the asymptotic behavior, it turns out that the, the way that you, the eigen functions, this is now when we go to those enhanced Markov chains. And this is work that Chetan and I did back in 2014. Um, Chetan discovered that the eigen functions of the enhanced Markov chains are identical in form to the quantum wave functions of free particles. Identical. So, the idea is going to be that quantum theory arises as an asymptotic description of a Markov dynamics. So, the Markov dynamics gives you a step-by-step-by-step analysis of agency and consciousness. Quantum theory only gives you the asymptotic behavior, not the step-by-step behavior. So, that's going to be, that's going to be the the connection. Again, this is all a matter of theorem and proof. This is, you know, either we're right or we're wrong. This is theorem and proof or theorem and disproof.
Now, one, one might say, "Well, um, you know, you have the no-cloning theorem in quantum theory. You know, what about that in Markov chains and so forth?" And it turns out if you look carefully at the, look, no-cloning theorem, the proof of it does not require unitarity. It only requires linearity. Markov chains are linear, and they have their own no-cloning theorem. So, I see no obstruction right now. We just have to do the hard work, but I see we have a principled notion of agency. And it shows us how the nested community structure can give us nested goals and bending, you know, nested bending of problem spaces. And we can actually not only talk about metaphorically about bending the problem spaces, we actually can show, I think we'll be able to show that we can actually have real space-time curved representations of bending, you know, general relativistic descriptions of bending.
Mike, may, may I share? You, you remember the, we were taking this project and embedding it into the variational free energy principle and all that. So, may I share now the screen to show Don and Chetan what we already have? I mean, mind you all, we, this is work from one year ago. We still, I still didn't get to develop it in full. It's, let's say, maybe 80% done. So, on the left, you see this book. This will come soon. It's Karl's book on the free energy principle on the nature of things. It's, it's the, the big monograph, monographs of the latest version, I suspect. So, uh, yeah. We didn't, we didn't push this paper because I also wanted to have access to the latest form of this before we would push. So, the synthesis paper had only a few dropped names of variational free energy and and and the decomposable way in which you can assess intelligence and true scale-free quantification and and and recursive decomposition in that sense. So, in this project, we tried to do it within the free energy principle framework. So, within the variational base framework, right? So, we take all these problem space operators and embed them into a physical variational physics descriptions. And we also end up on some of the things that you don't and and Chetan mentioned, like the, for example, when we can have the issue of free normalization and then getting ways in which you can decompose and quantify across scales additive gains in search efficiency at different scales and then do it globally for as a whole total of the system depending on how you would how efficiency gains are cashed out at different levels. So, that can be embedded in the variational logic of the free energy principle for sure, and using using the more pedestrian thing in the sense of like the all the nest assumption and and the Helmholtz decomposition and then building on that, building a minimal Landau style floor of cost per unit operator and and efficiency gains. So, this is not done, but what I, I want to say is that seeing you present this just now, it is clear to me that you provide way more finer algebraic structure for us to probe even deeper into this decomposition and then look at what you mentioned, the community structure and all these sort of effects you would get. And what would appear to us at the scale and metric of observation as an efficiency gain might come from innumerable ways of tiling that problem space or bending the problem space or just doing, in your terms, just changing the observation frame by communities and certain communities having different policies and then meta-policies based on the higher order aggregates and so on. So, I think it's definitely valuable, even going, even if it were something that's built after this is pushed because this will be finished quite soon. Right. It's definitely worth looking into it, and the connection with physics is certainly, certainly impressive for sure. The
That's the hard work, you know. This is less hard. Well, I would certainly welcome any interactions you guys want to have on this once once your current projects are you you you get to a certain place because I think as I listen to your work, I realize our ideas are really converging quite nicely here and I think that there's a synergy.
The nice thing about the Markov stuff is that it's it's so well studied. They they've they you just look at the you the the eigen analysis of these matrices to get a lot of this stuff. So, there are lots of papers out there about the community structure and and so forth. So, we would just have to do our homework and understand and a lot of that stuff we could just then port in here and it's it's it's really quite well understood. The only thing they didn't have was the trace logic and the fact that it can recurse. That's what they that's what they were missing to pull this whole picture together.
>> [clears throat] >> Interesting. Very interesting. Yeah. What do you think would happen or maybe you've already done it but to apply some of the causal emergence metrics to to the to the dynamics of these things, you know, or like some of the stuff that you guys do Robert or some of the like more conventional stuff that we have have you you know, phi d and all that kind of stuff. Have you have you done that at all? Me or Don? Mike?
Uh well well Don on his stuff and then and then if they haven't done it yet, then I'm going to say maybe we should. So, you know, so say a little bit more about the causal emergence question. I want to make sure I understand that that question.
Well, you know, and I mean Robert's the better person to speak to it but there's a there are a variety of new newish metrics in information theory that basically try to quantify some important aspects of agency, right? So, the the extent to which the whole is in some sense causally more than its parts, you know, phi all that stuff.
Oh oh oh right. That kind of exactly right. That's what I'm getting at. Have have you tried any of those metrics on on the dynamical path of these things?
Well, yeah. So, so a lot of that work has been the motivation between that Tononi and and Co and so forth has has been to somehow have consciousness be a function of the amount of causal emergence, right? And you know, to the extent that you the phi is the you know, the system with the the the greatest causal measure. And those are those are very very useful. I think they have nothing to do with consciousness.
Yeah, I'm not making any claims about consciousness. I'm I'm just asking you know, just you know, just step one of of getting the measurements and seeing seeing what's going on as far as
Oh sure. I think that would honest as long as there's no claims about that and consciousness, I'm all for it. I think that that stuff is that's that's really good work. Absolutely. What I think is bogus is saying that consciousness has something to do with that. I think that's bogus. But [clears throat] but yeah. I I but we haven't done that ourselves. So, we haven't done that would be So, the answer is we haven't gone there yet and that Yeah. It may it may be interesting to do just just to get some some data and just to do some measurements. I mean, we've we've been doing it on gene regulatory networks and all sorts of weird things and there's some really interesting but I would we haven't said a word about consciousness we had with respect to that but just just the data alone I think are already interesting whatever the interpretation.
Your data has really inspired me the last few months. I've just really your work your whole team has really inspired me. It really forced me to think out of the box about what this thing can do. So, thank you. I mean, it's been really quite fun. Thanks. Your your your your podcast with Lex Fridman, I've listened to it like five times or something like that.
>> [clears throat] >> Don Don I mean on on this topic specifically, do you think you could use not the inverse trace as a sort of hypothesis generation for this kind of experiments and this kind of data because I mean I mean it seems like you do the construction sort of you would in this paper at least in the paper in this case, we did it mostly forward but you you you do have a calculus and an algebra for doing it backwards. So, you have let's say an observed effective kernel P sub A on the visible states A and then with the trace chain theorem says that any cons any consistent extension P with the hidden states A prime was it? Yeah, A prime. Then that must satisfy basically the the the theorem. So, then the A B C D sort of like a tuples is a sort of parameterized hypothesis space about hidden mechanisms and then it just becomes an an an an inverse problem, right? It is is just that. So, if that's the case and we have a lot of data in our in our Synthesizer paper, we used the the planarian regeneration example and I know we use the data in that literature but there is also way more data in in GRNs and stuff like that. So, do you think you could we could have a sort of a model in the inverse trace that would distinguish between different hypotheses that explain best the data that we see?
Yes, in the following sense. So, the the trace logic because it's a logic, there is the notion of not only the meet but also the join. The join, yeah. So, I can take two matrices and if they are compatible if they're >> [clears throat] >> for example part of a Boolean sub logic, then they they have a join. Now, Chetan has done the hard work of getting a closed form solution for very special cases and it turns out to get a general closed form solution is an open problem. But we but the the and and the interesting thing is so there there there can be a join. There can also be so, the join is the least upper bound, right? Between two. But you could also have in some cases Chetan has pointed out that there could be a whole say one parameter family of minimal upper bounds. So, it's going to be very very interesting. We we with the trace logic for certain matrices, there may not be a a unique least upper bound. There could be a family of minimal upper bounds and then we may be able to use other um factors to choose one that we want.
>> criteria So, other criteria for for what we want. Maybe we want something that's the the you know, some kind of minimize complexity or maximum complexity or or or or you know, causal causal structure or greatest causal structure or something like that. So, there are all sorts of things that we could do. So, so I we don't know if there is a general closed form formula for computing the join. Chetan has it for a special case. It it's a fair bet that there is not one. That's going to be interesting to to it's an interesting open mathematical problem to study the join of this thing. And and and we I can give you the the unpublished paper we have so far where Chetan's stuff is there and you can see what we've got so far and it's open as to you know, how to to generalize that to or to prove that it cannot be done, right? But it cannot be generalized.
>> [clears throat] >> Mhm. Yeah, I know. It's it's it's super interesting. Yeah, I I would definitely love to to to see those papers and I I read I read some some of your stuff and also the the latest on Faces of Consciousness. I I went through through most of it. Very Okay, you've seen the trace of consciousness paper, right? Okay. So, so it's in the appendix at the at the at the back of that paper that you already have, um you'll see Chetan's work um on on what we have so far on the join. Yeah, work in progress. Yeah. Of course. Yeah, I know I know if you work because I am one of my best friends is Robert Breadner, so I we talk a lot. We just submitted a paper on on IIT and and and causal agent's theory. Oh, yes, right, right.
>> [clears throat] >> Yeah, of course we know Robert quite well, yeah. So, I have a question, John, about the um definition of the trace. The the trace of any Markov process is also a Markov process, right? Yes. Yeah, okay. So, do you have Do you have an available model of observations that are or sequences of observations that are not Markov? So, uh sequences of observations that um violate constant probability of switching from one state to the next. So, so that the probability is not well defined.
Right, it depends on your your time window and how you want to coarse-grain the states, right? So, it's one one can say that look, there are many many systems that um aren't it's not the case that if you look at the the states I've given you, that the probability of the next state is can be given exactly just based on the current state. You might need to have look at three states or five states or 10 states or whatever, you know, you're bigger a bigger bigger window to to get uh the probability.
>> [clears throat] >> But but in those cases, you can always then create new states and then turn it Markov. So, basically
>> You can expand the the state space. Actually multiply it. And there's there can be a combinatorial explosion, though. You do need to be aware of that possibility. And
>> [music] >> um and this only works for finite memory. It doesn't work for infinite memory. But even for finite memory, you you do have to be a little careful that I mean, the combinatorics can get nuts. Which is seems to be often the problem in consciousness research. Right, you know, I I'm I'm of course mainly interested in uh issues like contextuality and quantum theory, where you have groups of observations for which joint probabilities can't be defined. And so, um you can't build a single self-consistent hidden variable theory. So, I I don't know whether the formalism will handle that sort of situation or not. Since Since you do seem to be always assuming well-defined probability distributions. Can you say more about the system that that that doesn't work, Chris?
>> [clears throat] >> Uh well, contextuality is is defined as as a as a phenomenon uh sets of observations uh over which no sets of observations for which a joint probability distribution is undefinable. Yeah. For which the statistics violate the Kolmogorov axioms. I suspect that the fact that joins don't always work might have something to do with that.
>> That's what I was thinking, too. Yeah, that that's that's
>> I I originally thought your question was about, you know, having the same probabilities every time. And of course, you don't need that with Markov chains. They don't have to be homogeneous, but that's not your question. It's It's much It's It's more abstract than that. Yeah. But I that you know, what I would my hope would be to to somehow find sub logics which
>> [snorts] >> actually look like quantum logics. Um Right.
>> But you know, that if that happens, then we could possibly answer your question in affirmative. I mean, that would be one way to do it. Okay. So, uh another way to think of it in your formalism might be if there are pairs Yeah, this would be part of what look like a quantum logic. If you have pairs in the trace logic network that don't commute.
Oh, easily. Yeah. That That may be a way of approaching this joint question, John. Yeah, that matrices have no joint. Yeah, to get back to your comment about unitarity um at the very beginning. And unitarity is really just conservation of information. And Kolmogorov probability is really just conservation of information. So, if you don't have situations in which information is actually lost in some global sense, uh sort of at informational singularities, [clears throat] if you will, then the system uh satisfies unitarity as it's used as an axiom within the information theory, which is just conservation of information. Hm.
>> [clears throat] >> So, that this is where I was I was trying to emphasize this disassoci- this complete dissociation, actually, of unitarity from any spatial considerations. Hm. But okay, anyway, that's that's sort of an aside. The The real question is about contextuality. Right, right. Yeah, it is striking that that Nima Arkani-Hamed and these high-energy theoretical physicists are are strident that they're not assuming unitarity. They're They're strident about it. They're saying, "We don't need it." And we'll show that it arises from uh these positive geometries that that are entirely outside of space-time. So, they're they're getting space-time and unitarity um together. I I've never seen a I've never seen a demonstration of of that fact as it uh you know, in itself. What I have seen is, you know, they they derive um scattering amplitudes and which match what's understood from, you know, the Feynman approach. That doesn't mean you derive space-time, and it doesn't mean you've derived unitarity. It just means that you've matched something. Yeah, I I I think they're referring to
>> is absent. Yeah. Sorry. Yeah, I think I think they're referring to unitary processes in space-time. Right. Right. That's what they're referring to. Absolutely.
>> Yeah. Right. That's very different from unitarity Yes. as a strictly information-theoretic concept. And And yet, the way they wave it around and say, "We We don't assume space-time or unitarity." Right? It's just a disconnect in language, I think. I mean, you you know, it's it's fine not assuming uh a collection B when you're doing a collection A. But if somebody like Dyson comes around and says they're equivalent, then then you can't say that we don't need space-time. It's just another way of looking at it. So, they have you know, their claim is is unfounded as far as I I know. I mean, you know, somebody needs to sit down and say, "This is how space-time emerges from the amplitudehedron." Yeah. Otherwise, it just just saying that I you know, I Schwinger could say I don't need to assume Feynman, and Feynman could say I don't need to assume Schwinger, and they're both right, but they're both equivalent. Right. Yeah, yeah, right now they don't give you space-time, they give you scattering amplitudes. That's what they give you.
>> Right. Which I still think they're defined in space-time. Fair enough. Yeah, I I mean, a facet of something, you know. I I suspect that eventually we'll generally be able to identify uh amplitudehedron-like structures with error-correcting quantum error-correcting codes. Hm. You You wrote a paper on that, didn't you, Chris?
Well, we're we're in the uh we have a preprint of it that was revised as of a few months ago, and we're still working on it. >> [clears throat] >> But the uh the current pre- current available preprint isn't bad. Um but the hypothesis would be we can go the other way from amplitudehedron-like structures to quantum error-correcting codes. And there are many ways to get space-time from quantum error error-correcting codes. So, Hm, interesting.
>> it it may be that the the inference ends up going in that direction through that direction. Yeah. What I'm hoping to be able to show is that some of these positive geometries like the associahedron are sub polytopes of the Markov polytope. So, that because the Markov polytope um could be describing the probabilities of of certain interacting processes that we would think of as scattering processes. And so, if if if that's the case, then I would you know, there may be a a deep connection between some of these positive geometries and the Markov hedron, which is itself a positive geometry. Yeah. Set of all possible Markov chains is a positive geometry. Yeah. Yeah, I I suspect that. Uh Any any such structure defined over any space of possibilities or any such dynamics defined over any such any space of possibilities can be thought of as scattering in some you know Right. meta metaphorical [snorts] but but formally sensible way. Yeah. I mean computation We can think of computation as scattering Right. data space. Interaction is scattering. Yeah. I mean, scattering is what is how interactions look in a physics lab. What what's surprising is how restricted the scattering events are that you find in physics, right? That that that's a very restricted set and it turns out that you the something like the standard model gives you all the components that you're ever going to find in any scattering that you ever do. And
>> We hope. Yeah. That that so far well, yeah, that's so far. Well, the all all all the ones that you see with the sorts of things that we call elementary particles. Right. Right. What what what I find interesting is just as when we had the Ptolemaic system and we had all cycles and cycles and cycles, you could get all the orbits of the planets. But it was ugly and just just a mess cuz you had to do add all these cycles and correcting cycles to correct those cycles and so forth to do it. And the same thing happens with space time and scattering. You would you look at the Feynman diagrams, it's loop after loop and you you have three or four particle interactions and 500 pages of algebra because you have all these Ptolemaic loop after loop after loop where you're enforcing locality and unitarity. So, Feynman is forcing locality and unitarity and and so we have to do all this stuff. And all of a sudden 400 pages of algebra turns into two terms when you let go of space time and
>> Yeah. And also it feels again like we've got this Rube Goldberg machine called space time and that's why things look so ugly in space time and the mathematics. And all of a sudden we're seeing some hint you know, it was a big hint when we went from Ptolemy to Newton. That's you know, all of a sudden the formulas got a lot we're on to something much deeper here than than Ptolemy was on to. And now when when we go from you know, Feynman scattering diagrams to these positive geometries, once again we're getting 400 pages of algebra down to two terms. A clear hint that we're on to something deeper beyond space time. Yeah. No, I space time's a kludge. It's that's I think that's clear. Yeah, it's it's it's But what's interesting is that theories of consciousness all the main theories of consciousness assume otherwise. We start with space time, we try to figure out what physical systems in space time could possibly have the right structure to give rise to consciousness. Yeah. So, we all we all of our theories start with the kludge as the assumption. And and then try to go from there. So, they're doomed completely doomed to failure. Yeah. Well, I I mean all all of science has done this before about what? 1970. You talking about the standard model since about 1970? Well, no, I'm I'm roughly dating uh roughly dating at least the first things I saw from Wheeler with the notion of observer participants in it. Yeah. Yeah, right right in the 70s, right? Yeah. Yeah, he he saw it. He he knew I mean, he wrote the book on space time. He wrote the book on gravity. Yeah. Misner, Thorne, and Wheeler. That's that's that is the Bible and he so he knew space time and he knew it was a a kludge. He was looking for something entirely beyond. Right. Yeah. And he was going to call it observer participants. That's what he called it.
May May I ask something? Yeah. Go ahead. Go ahead, Chris.
>> going to say good to see you guys again and good to meet you, Robert. Good to meet you, Chris. And I think we will meet in person in Spain if I'm not mistaken in July. Yes, hopefully. That sounds very exciting. Yeah. Um I I I I told them to to invite you specifically. Thank you.
>> [laughter] >> What what's happening in Spain? Uh there will be a workshop um organized by by some people um the Tatiana Foundation and and it's a workshop on the known unknowns in our fields of interest and
>> Oh. There will be yeah. Um more than one uh um more than one field is fields. Okay. So, Chris fields
>> why fields is uh
>> [laughter] >> Excellent. And uh And uh yeah, um discuss several things. But Don, I wanted to say that um I think this is the the biggest myopia in consciousness science. And I work in Anil Seth's lab. I did my PhD with Giulio Tononi. I did a postdoc with David Chalmers. I work with Georg Northoff and and the temporal specialty I I know I know all the people. Yeah. And all of them are sort of either physicalists or they they still think that consciousness is something we we you squeeze with enough dexterity at the end of a long tube that you operationally construct. And and they don't get they really don't get the point that consciousness is the starting point. And everything else must come afterwards. You build everything else from consciousness, not the other way. And It's hard. I know the I know the I know the pain. But could you do you see your uh con the platonic space as having something to do with consciousness in that sense? Or being embedded within it? That's Mike's assets.
>> [laughter] >> Yeah, that's that's him, Mike. Well, um So, okay. So, so I'm I'm not fundamentally a consciousness researcher. I don't have any strong claims on this yet. Uh and but if I had to say right now, I would say that uh I I I don't think that from the point of the platonic space formalism that I'm investigating, I I don't think that we are beings that occasionally get visited by platonic patterns that are something else. I think I think we are the patterns. And I think that these patterns are wide range of you know, static, dynamic, low agency, high agency things. And I think what we call consciousness is the perspective from the platonic space outwards into So, so what a pattern from the platonic space experiences when it interacts with the physical world is what we tend to call consciousness. Now, I'm not sure that like I don't think that exhausts all the possibilities. I think there are probably lateral interactions within that world. I suspect that when mathematicians think about the you know, abstract mathematical objects, what happens is you have two there are basically two two different patterns in resonance there, the human one and the you know, whatever it is that they're studying. So, there may be lateral interactions that don't even require the physical world per se. Uh but but basically what I think we mean when we talk about consciousness is what it's like to be a platonic pattern projecting into a physical world through some interface. So, that might be you know, the sense organs that we have or something completely different and so on. So, the platonic patterns transcend our experiential notion of consciousness. I I In the sense that uh Well, say say more. What what do you mean about that? Well, there's a platonic realm that maybe we shouldn't describe as conscious, but but we perceive but a human perspective on it is we we experience it as consciousness sort of looking at it somehow. So, that's just perspective. Yeah, I I I think I think I think ultimately I think some version of idealism is probably the the more accurate thing. Like I do think that consciousness is fundamental. But but but but I don't know what to do with that on a practical level right now, you know? Like I don't have a way of making use of that in the lab or anything like that. What I see is a way to make progress with a somewhat more dualistic version, which you know, I agree that it would be nice to have some kind of simple monism. But but what I see right now is that we have different but interacting realms so to speak that that and that model helps us to do new experiments and make new discoveries. So, for now, I you know, I do think that um the they're they're sort of two two separate things, but ultimately, if I had to guess, I would say that consciousness is is primary. Um I don't know how to do that reduction right now, so I'm sticking with two because that's what that's what we can we can handle right now.
>> [snorts] >> Yeah. My my one of our goals is to show that we can actually get space-time special and general relativity and quantum theory from this theory of Markov chains of consciousness. In which in which case then we could inherit all the work that's been done in physics, um but but see it arising from a a consciousness-first point of view. So, that's that's where I'm headed. Um Yeah. I mean, for us, you know, one of the one of the important pieces of our research program is to understand the mapping between the interfaces that we construct, and those are anything from sorting algorithms through cyborgs through xenobots, embryos, you know, you name it, like all of these stuff. To understand what what are the properties of these things that facilitate the ingression of specific patterns from the space. Like, so so what what you know, what what is it about this thing you've made that pulls down this rather than that? And the and the right then that allows different different degrees of and different kinds of of of mind state to to interact through it. And uh also to quantify in so we've got some we've got some wild stuff that's kind of uh ripening in the next few months that I'd love to run by all of you. We'll we'll all we should all have another meeting so you can see. But we've been we've been trying to quantify like one of the things that I think is really important about this Platonic space thing is that it's not just a redescription, you know, and a bunch of sort of useless you know, philosophy that you sprinkle on top of things that work perfectly well. It actually makes very new predictions in the sense that it suggests that what you're getting from that space is are things that using conventional um theories or that we have now of doing the accounting of effort of computational cost of physical cost, these kinds of things that you get you get free lunches or at least heavily discounted lunches. You get more than you put in. You you know, our our accounting isn't adding up everything that you get. We're missing something very significant. And we now have the ability to quantify how much are we getting how much how much free [snorts] memory, free compute, free you know, whatever. Um Uh we can at least in simple uh toy you know, sort of minimal models we can quantify that. In biology, you you can see it, but but but it's hard to quantify or prove anything. It's just too complicated. Whereas in these minimal models, we can actually quantify you know, what did we get that we didn't pay for according to the conventional way of totaling up effort. So, and and so that's really important to find out how much and and what do you get? You know, do you just get static patterns? Do you get behavioral propensities? Do you you know, algorithms? Do you get virtual machines? You know, do you get free compute that you can do in that space? Like that's a you know, kind of a crazy prediction of mine that I I think you can actually get you can do compute in that space that you don't pay for in this in this space, so to speak. So, that's you know.
I've [snorts] been listening to a lot of your podcasts on this and thinking about it and and I think that there's connection with the hidden states in this Markov system. So, that when you if we just see a trace, most of the intelligence is something you don't see. And so, to the trace observer, that's all in a Platonic realm because you don't you literally cannot see it. And yet what you're seeing is entirely a trace of that world. So, your your visible world is controlled by this quote unquote Platonic space that you cannot see. And so, that's why I was thinking that's what your stuff about the planaria, for example. You cut off the head and cut off the tail and you can change the the electric fields and and make it have two heads and so forth. I mean, it's just you know, how does it know how to do that? Right, where is that state? So, I've I've been thinking so, if somehow all we're seeing is the planaria in our trace, we're not seeing beyond what we can see. There's a whole Markov realm of of intelligence out there that is projecting down into what we can see. Um which is just a planarian, but so I'm really really left to explore with you guys you know, as this formalism matures, how we might use the formalism in concrete ways to model specific Platonic spaces for specific memories biological memories. Cuz I think this gives us the tools, right? The there are the exits, the the dark states, and the entrances. All those tools are part of the Platonic space. And if we learn so, we're new to this ourselves, right? We don't know how to use those tools yet, but to learn how to use those tools to model specific things, we be able might be able to get that Platonic space not just a hand wave, but here is the Markov chain, here's the trace, and this is why it looks like this Platonic intelligence that's guiding you. But it would be multi-scale. Right? The there's going to be multi-scale in the dark space.
Um Well, you know what might be really fun is so so definitely we should we should do that with some of the biology examples that we have. Um in particular, for example, with some of the synthetic things that we have so so xenobots and you know, neurobots and because because they raise what I think is is perhaps the most interesting part of this is, you know, where do the uh the goals and properties of novel beings come from where you can't just pin it on selection, you know, eons of selection. Like I guess where they come from. So, but but but in I would I would complement that with uh which I think would be even [clears throat] easier, the study of basically applying these things to some of the minimal computational models that we have. Yes. Right? We have some sorting is one and there's going to be a bunch of new work on that coming soon. Um but we have others. We have some really interesting stuff uh that'll come out soon on um giving embodiments to various uh very weird um you know, sources like like a mathematical objects and making a robots that are driven not by conventional algorithms and sensors and whatever, but but their entire behavior is driven by, pre you know, sort of pre-cooked static mathematical constants and watching how those kinds of mathematical objects and watching how those things end up navigating a world and adapt you know, and and what what cognitive features they end up having and so on. So, I think those things are simple enough that we could I think we could actually make a make a pretty tight mapping onto what onto what you have in terms of states and and and things like that. Interesting. Interesting. Interesting. It may be that when you get to into pure mathematics and sort of a hidden Platonic realm that's doing stuff, you know, like sorting algorithms that are doing other things you didn't expect. I'm having my mind stretched to think about how Markov chains could do that. That seems like that might be something even deeper somehow.
Well, you know what we could that was the other thing I was thinking what what would what is totally doable now is to take your system and apply the tests that we have. So, we so we have a range of assays that basically are taken right out of the behaviorist handbook cuz the one thing I think behaviorists got right is that they weren't worried about what the what the implementation was. And so, it's very easy to apply their tools to anything, right? Um so, we could we could actually look for habituation, sensitization, associative conditioning, delayed gratification, path planning, you know, illusion, you know, counterfactuals, all this kind of stuff. We could we have assays now that we can look for all that stuff. Yeah.
Well, and and um it's clear to me that we there's always going to be Markov chains that we can build to do that. So, it'll be like they're they're universal Turing machines. Markov chains are universal. Yeah. I So, I'm not talking about building ones that do it. I'm talking about finding it in simple or random ones that you don't think should be doing it. That's that's the trick. We're we're we're finding this we're finding these capacities in very simple and and no no design, no selection. You know, you usually the the three things you think you normally need you need rational design by an by an engineer, selection or you know, evolution or learning, right? Those are the three things you need. One of those three things you need. We're not doing any of that. We we're we're we're pulling it out of you know, I don't I don't know. You'll you'll judge for yourself where where you think we're it's coming from. But but but I think we could do that with with you know, random random matrices or whatever. That's that's right. We may be able to find matrices which um what we're seeing in the organism is the trace, but but the invisible states are having a lot of the intelligence that that leads to what you're seeing. And so, we write down the the matrix that even though the trace you can't see why it's doing that, but it does it. But in the big matrix, you see why it's doing it. Yeah. Yeah.
I'm going to have to jump off, guys. Great conversation. Thanks, Chris. Good Chris, always a pleasure. Yeah, see you. Yeah, thanks, Chris. Good to see you, Chris. I we we have the recording of this. Uh yes. Yes, it is being recorded. Yep, I will I'll send you guys a link. Yeah, if if everybody's okay with that, I'll put it up on our center channel, but regardless, you you can all have a copy you can have a copy of it. That's fine with me. Yeah, great. Perfectly fine. Great. Yeah. Yeah, I think there's yeah, I think there's a ton of stuff to do. So, why don't we, you know, go off and and think about some some specific directions and let's let's come back. Um I I already have some thoughts, but but it'll be even there'll be lots more in a few weeks when you know, when I can send around some preprints. I'm having my my students write these things up and so then Yeah. I'll send them out.
>> Very good. And if we maybe then pick a particular simple problem system that we can see what the trace logic might do on it and see where we go from there. That would be fun. Yeah, yeah, that'd be great. And I think a natural extension of of our our formalism like after we we do the the FEP thing is basically try to to apply all all this stuff to it and and see how how you can get even deeper more interesting things you can say about all this computing intelligence across scales in more fine-grained manner than even the FEP allows. Yeah.
>> Yeah. Yeah. I think I think As we bring you the paper, I would I agree that this seemed to be a natural connection with what you're the project you guys are doing right now. Absolutely. Yeah. Yeah. Yeah. Good.