Transcription
When you look at metadata that's taken from our phones and our location, couldn't you pretty much tell what I'm thinking?
So, you really have two types of of things, right? You have the images that you present to the participants and you have the brain responses to those images.
You're mapping the signals.
Absolutely. I think you're highlighting something which is actually an open question, which is the interindividual variability of neural representations.
Yeah, that's what that meant.
[Music]
This is Star Talk. Neil deGrasse Tyson, your personal astrophysicist. And I see to my right Gary O'Reilly. That must mean it's special edition.
Yes.
Gary.
Hey, Neil. How you doing, man?
I'm good.
Former soccer pro.
Allegedly.
Allegedly. No. Are you better here than you were when you were playing soccer?
As I get older, I I do get better.
Good answer.
Yes. Hey,
as you get older, you get
I get older. That is what happens to me.
That's about it.
Uh, so I'm looking at the title you proposed, reading your mind.
Ooh,
yeah. I thought this was a science show.
I know.
Start the seance now. Buckle up.
Okay.
I'm getting an M. An M. There's a You have a relative somewhere. Um,
somewhere in the hemisphere,
right? You had a mother. Uh, okay. Sorry,
but that's just it. You're not.
All right. AI, Gary. AI will be driving our cars, our trucks, our trains soon enough. And probably, if not already, it will help us solve our everyday problems.
Um,
it already is.
Exactly. And it'll probably solve some of our big problems. It may even help us tidy up some of the mess we've made over the years. But surely it's never going to be able to read our minds, is it?
H
Well, actually, yeah, it can. And uh our guest today leads a research team using AI to decode the language of our brains.
But before you start shouting at your devices, stop and think about the positivity that could come with this as a tool. Um but those who can think but not speak who will get a voice.
So for that and if that happens that would be truly amazing.
The ethics of that too.
Absolutely. That's what I'm talking about there. So if we would introduce our guest
delighted to you.
Thank you.
Jee King.
Oo king.
Oh you're going to be saying that for hours you from France. Did I say all that right?
Absolutely.
Perfect accent. Welcome to Star Talk. Welcome to my office here at the Hayden Planetarium.
Thank you very much for having me.
And you work for Meta.
That's right.
Facebook basically, but
absolutely. Yes.
But Meta I mean
I think it's not just one singular.
It's not a thing anymore. It's Meta. All right. You work for Meta in Paris. You you have a background in neuroscience. That's I love neuroscience. We have neuroscientists on the show all the time.
Oh, we really do. Yeah. We're all in the situation when we have a neur neuroscientist and describe to us what what your goals are
aside from world domination. So we have uh we have a lab at MEA which is called fair for fundamental AI research which is uh structured as an academic lab in a sense. The goal is really to uh understand more about the principles of artificial intelligence and within that lab uh I'm working with a team that interfaces two disciplines uh neuroscience on the one hand and AI on the other hand try to both better understand how the brain works and also try to perhaps improve AI algorithms in light of these uh principles.
How do you have any idea at all how the brain is processing information? So we have tools for this of course in neuros tools.
Interesting tools do you put on people's brains? Oh
this is not hammers and chisels.
Tools that's a euphemism for something and I want to know what.
Sure. Yeah. You have a really a wide battery of tools that you can use. The uh the one that we
on human brains
on human brains. Yeah. So the the one we tend to use the most in the team are non-invasive neuro imaging techniques. So from um magnetic resonance imaging like the big scanner you have in hospitals
um to uh electronography. These are like the small nets that you can put on on people's uh
little caps that you put on your head with the all of the uh
um
and how does that work?
Electrodes it looks for fields
and each of those
electromagnetic fields that come through your skull.
That's right. So each of those work with different principles. So for EEG for electronography and MEG the magnet magnetonography you measure the fluctuations of electric and magnetic fields which are elicited by neuronal activity. Uh so typically
thoughts the biological insensations of thoughts. Yeah.
So is every brain precisely what I said you know the biological
in what's the word?
Insensation. Yeah.
In sensation of thought. So that does that mean that every action in the brain has an electrical counterpart or so like the firing of a syninnapse is an actual electrical you know
actually you have action a lot going on in the brain which is uh not electric or doesn't doesn't lead to electric fields in fact even the neurons which are uh firing not all of them are are being measured with EEG or MEG and we tend to only measure those that are specially aligned. So in the in the cortex which is the part of the brain which is uh folded um you have a lot of neurons which we call pyramidal cells that tend to be positioned in the in the same way. So when they discharge electricity their electric field can build up over space because they actually are aligned spatially if they
strengthens it the signal.
Yeah. If they were facing any direction
get some cancelling.
Yeah. average down to zero basically. But because they all
all aligned with one another, then you can measure these electric fields uh at a macroscopic level even with electrodes that are positioned on the scalp. So not inside uh inside the brain.
So if that's amazing.
So you're an fMRI and you're you're offering images
functional magnetic. So that's like you are actively you're a wink talking to the person
while they're messing with your brain.
Well, they're not messing with your brain. and they'll offer you an image and that then gets picked up through through the data. But while you're offering an image to a patient, there's other noise.
Well, you declared something he hasn't declared yet. Can we get him to say it first?
Okay.
When you read the brain, what do you see?
We see a lot of noise. But maybe just
Yeah, I didn't say my brain. I said when you read a when you read a regular brain, what do you see?
We just
see a lot of noise.
But just a clarification on the fMRI. So fMRI is a different type of technology that does not pick up electric and magnetic fields like EEG and and MEG. He actually picks up a proxy of neuronal activity which is the deoxxygenation or the blood flow in in the brain. So when neurons are active they they consume oxygen and so you have a change in the the vascular uh flow which you pick up with fMRI.
So you're getting the geography of the brain as to what's happening
and where.
Absolutely. and and this is this is a very different type of signal that you would measure with EEG and MEG and it's very slow. So uh of course the the the blood comes in only like it doesn't change every millisecond let's say and so you have a very different type of uh signal that you would observe depending on the device of choice whether it's fMRI or EEG or MEG or intraranial recordings when uh you you can have access to this type of of signals
intraranial means you actually have probes inside the brain inside the brain
absolutely so this is very common
people say go ahead and do that
no so so in in No,
you you do have patients uh typically patients who suffer from intractable epilepsy who uh need to have the part of the brain which generates the seizures to be removed. And before doing this, it is common to have uh a procedure where you well the neurosurgeons and the epileptologist decides to put electrodes inside of the areas which is believed to be pathological in order to be sure that this is indeed the brain region that uh should be removed.
You don't want to cut out the wrong part of the brain.
Absolutely right. And so these individuals typically would stay about a week in the in the hospital during which uh these signals can be analyzed by neologist and during that week you can ask them whether they would like to participate in for instance an experiment that involves I don't know story listening or watching a movie. I
mean we're we're already in your brain so why not? I mean, we're already in here, you know. It's like when the mechanic goes, "Listen, I got to go in there anyway, so I might as well get the calipers done on the brakes, you know?" So, right.
So, when you're decoding the brain waves, whether it's blood flow or the magnetic fields
and you said there's noise,
how is how is your algorithm filtering out that and how is it breaking down? Because you said there's different data. the way the data comes is different from an fMRI than it does from an MEG. So how can you explain how the algorithm is reinterpreting that?
Sure. So so maybe just to start with the reason why I said that when you look at it it's looks like noise is because the these signals are impacted not just by neonal activity but by a lot of different factors. So for instance uh magnetic fields are constantly evolving. I shouldn't try to say this in front of you guys because you know more about this than I do but we are in a flux of magnetic fields all the time. Uh and the magnetic fluctuation that are being generated in the brain are extremely small uh orders of magnitude smaller than the objects that surrounds us and and and move around when they have metallic parts and so the signals that can be picked up basically contaminated by by all of these things. So when you look at the raw data, it's very difficult to to to guess anything. Actually, you you would probably uh need to start to do the very same task again and again to try to average out the the the noise and uh start to see what is the average brain response.
So they're really looking for patterns than anything else.
Then what better than to use AI to recognize patterns?
That makes perfect sense.
Wait, so let's back up for a minute. I understand you can look inside someone's brain and see the image that they're seeing as though you were somehow their eyes behind what the brain processed. Do I understand this correctly?
The goal is to try to understand how the brain represents uh perception. In the case of this experiment you're alluding to, the individuals are typically watching images one at a time. Each image is lasting for about a couple.
Did you do this on mice before you did it on humans?
But you only saw big chunks of cheese. Are you saying this because I'm a French researcher? The we do not work with non-human animals in our team. But of course in neuroscience you have a wide variety of approaches and uh a lot of people are indeed working on the visual system. Rather in macaks and mice mice are not so great for for for vision but uh yes there are a lot of different if I remember correctly.
Well I'm not an expert in this but I think they do see uh things but not they don't count on on vision as much as we do.
Right. Right. So what you're really doing is you're measuring these signals as a person is seeing something and that what you're measuring once you filter it you're able to determine that this is the pattern and if we match that from person to person where what are you measuring against is really my right my question. So you really have uh two types of of things, right? You have the images that you present to the participants and you have the brain responses to those images. And the whole goal is to try to find the linking function between the two.
Okay? So you could use the same person actually and just replicate that over and over again. If you keep seeing the same pattern, then you know from this pattern that represents a a sports car or a this or a that. So you don't have to
you're mapping the signals.
You're mapping the signals.
Right. Right.
And so wait a minute. Here's here's the real rub though. The human brain varies from person to person in uh not in its general regional response to stimuli but it does vary in how we actually perceive things. So how do you make sure that what you're measuring in one person is actually going to be what you're measuring in another person? Like if I were to lose my sight, my occipital lobe would go like dead, but other parts of my brain would take up that activity. And so you would be measuring a completely different data set because I'm blind, but in my mind, I would still be seeing stuff. So
I think you're highlighting something which is actually an open question uh at the moment, which is the interindividual variability of your representations.
Yeah, that's what I meant. the uh and it's so up to recently most of the human neuroscience research was really trying to focus on what was common across individuals. So typically the the very sort of standard experiment is you take 20 or 40 participants like you and me and you make them do a task for about an hour in the scanner and then you try to see whether their brain responds similarly to the same stimulus. For instance, if you present half of the images with faces and half of the images with houses, is it the case that the brain areas that responds to faces is similar across individuals? And the result is that there is a surprisingly common structure across individuals in in ways which raise questions. For instance, you have um an area in the brain called the face fusififor gyus uh which is an area that responds specifically to uh faces and uh this area tends to be located in the similar part of the brain for every individuals
which is fine. You can say okay maybe genetically this was pre pre-programmed. We have some neurons in the brain which are specifically tuned for this.
But it also is the case for reading for instance for athography. So if you present words um you can find that indeed some part of the brain are specifically responding to uh letters or the letters that you know or the words that you know and this is uh this tends to be in a in a brain region which tends to be similar across individuals but this is this cannot be genetically programmed right because words is something that emanates from culture. This is a recent trait. So trying to understand why the same uh highle representations end up being represented in the same place in the brain is a major question. Now having having said that it is the field is is shifting towards uh more and more focused on individuals and we do realize that indeed the representations are very specific to some extent to individual brains and that so far we may have emphasized too much the similarity across individuals and and not pay enough attention about uh the individual specificity.
But if you have to calibrate against the individual for the individual's thoughts then you can't just come up to a stranger and know anything about them. So we we would so for instance we would know that uh auditory inputs so sounds that comes into your ear tend to be processed in the same brain regions at first right it's not that the ear is connected to a random part of the cortex uh it it tends to arrive ultimately in in the primary auditory cortex and this would be common to most people except if you have brain lesions or or a variety of pathologies and that would be the same for for vision and that would be the same also for the sense of numbers for instance if you have a sense of magnitude This is typically hosted in the uh in in a pal cortex and this tends to be the same across individuals. But as soon as you want to get more specifics uh you want to uh really try to to get more fine grain level of representations then this becomes really um specific to individuals and it's difficult indeed to to transfer the knowledge that we observe from one one participant to another. If we step back into the the offering an image to a patient, how accurate now is your algorithm in terms of replicating as much of that first image and and how much does the algorithm say, well, I'll take a calculated guess at filling in the blanks.
That's a very difficult uh question.
How many blanks are there right to fill in? Yeah,
because the uh metric that we use for evaluating how well we reconstruct the images in this case is not well posed. So if you take for instance a pixel level uh you want to compare how good your image the image that you manage to decode from brain activity is compared to the true image you may get every individual pixel wrong because perhaps I don't know the color is slightly off and and and the objects are slightly to the left or to the right and so you would have a very bad decoding metric. But if the image has the same content, if it's, I don't know, the true image has a horse and you also decoded a horse. You don't want to say that this was a terrible reconstruction, you want to say, well, it's maybe not pixel accurate, but it tends to to have the right concept. Uh, and so there is uh for now a difficulty in even quantifying the uh the quality of the uh of the reconstructions. However, what is striking is to see that when you have a lot of hours per participants, typically 20 40 hours per participants of them just watching image images in the scanner the and you have a very good uh scanning technique like a ultra high data.
Yes.
Yeah. This would be a huge amount of data for neuroscience, not for physicists.
The universe is bigger than your brain.
I was going to say they're only mapping the entire universe. So, of which your brain is a part of So once you have a lot of data per uh individuals then you can really start to reconstruct what they perceive in a surprisingly accurate way. Uh however going beyond perception currently remains uh very difficult.
H
okay so if you've offered an image to a patient
you get a certain set of data back depending on the subject matter of that. What's the difference if the patient is asked to imagine
right an image and do you get a variant
seeing it through your senses?
Yes. Yeah. We're talking mind's eye for one of a better term.
Right. So in the case of perception, this is where the most progress has been made. So when when you watch an image or when you hear a sound, it is becoming increasingly easy to decode what the person has seen or has heard. However, when you do the same type of tasks uh but on imagination, you can get performances above chance level from a statistical point of view. But frankly, it's it's it's not very convincing to uh to anyone uh who don't want just to look at the at the stats and and and just want to see the reconstruction. And the reason for this is well there are two reasons. The first reason is that the signal to noise ratio in imagination is much lower than in perception. So when you look at the brain signal on average they are weaker when you try to imagine let's say an apple than
some people have vivid imaginations though still
and I don't I don't think we know this. So I think this uh trying to evaluate whether the people for instance who claim not to have any visual imag imagination indeed do not uh have uh representation that would be decodable at all
because I just learned days ago that a colleague of mine
he went around the room and said imag picture an apple in your head. Picture an apple.
Okay,
picture an apple.
He can't picture an apple in his head. And I'm right and I is this some rare
not even the computer
he cannot conjure an image on command in his head. We all thought of Apple red apple right
but any image on demand.
Well he used that as a simple one but I so I didn't know this was an issue.
Yeah I think this is actually quite common. Uh I am not an expert in this but I think that the term is aphantasia. I think this is something which is uh more than 5% of the population. And I think uh that claim not to be able to uh to imagine visually uh objects.
They don't become artists.
I don't think uh artists are are restrained to just imagining objects in their head. You have musicians uh that may not engage in this monology.
How much more in terms of a percentile do you think your research is going to take to how the brain interprets images?
This is a very uh this is a very difficult question. again the um
sorry the question was easy is your answer that's difficult
probably yeah um I don't know about our research specifically but what what is clear is that there is a a huge progress which is being made in
thanks to AI but not as a tool like you would see in other sciences so for instance in I don't know in biology in in um in cosmology in sciences where you have a lot of data you use AI as tools you have a lot of numbers you don't know how to crunch you train a system to do whatever you you're looking for and it helps you process this data. In other science, we we also do this in the pattern matching that I that I was uh that we discussed earlier, but we also use this as a modeling framework because the the AI system in a sense is also trying to do something that we do. We train AI system to perceive the world to try to recognize objects to reason upon the world to to to discuss with us in in a linguistic form. Um and so this creates basically systems that can then be used as models of how the brain works. This is really um yeah accelerating the the I think the understanding of how the brain functions.
So you talked about linguistics there. If you presented a sentence to a patient, then you're going to have that sort of perceptual stage of where they perceive what this sent they see the sentence. Then you go through what they call a lexical stage and then a contextualization stage.
That all makes sense.
Good.
Are you basically how we communicate?
I know. But are you able to get the
seven syllable words though? Are you able to get the algorithm to feel the nuances of the brain and actually see how that breaks down?
Is that just the future?
I'm waiting for this answer.
Yeah, maybe I can I can say how how we how we do this in the first place, right? So, we can have uh individuals like you and me and I'm often a subject of my own experiments going into the scanner and reading a sentence, right? And so you flash a sentence word by word once upon a time and so forth. And for each millisecond you can see okay what is the brain activity now? What is the brain activity now? So you end up with an activation pattern associated with each moment of time
and that you can time lock to to words or to syllables phonms and then you can do this same approach in the AI algorithms. You can present a sentence uh and deep learning networks nowadays have activation patterns inside of them which are known to be difficult to interpret but nevertheless we can do the same trick. we can time lock the activations of the deep nets in response to words, syllables and so forth and then we can do the comparison between uh the activations of the AI systems to the activations of the brain and we don't know what these two things uh represent but we can still try to do uh correspondence to try to see whether they tend to be similar in the ge geometrical structure that they they that they hold and what we observe is that this helps us decompose the uh stages of processing that you mentioned. So we can first see that you have algorithms that are trained to do visual processing but know nothing about uh about words about about language that you can map and and corresponds to the activations of the perceptual system. And then you can do the same type of comparison with an algorithm which this time is not trained to recognize images or or pixels or to transform pixels but is trying to analyze words and combine them together. And you will see that the activation patterns of these algorithms that are processing really at things at a language level and not at a perceptual level, they do have activation that corresponds to other brain regions and other time moments. And so we can try to do this sort of onetoone correspondence between the model and the brain to try to understand the structure of these representations. And where exactly in that process do you get the language model to I'll say mimic uh perception and the nuance that we have which is experientially based. So when you w look at once upon a time there is an activation uh uh pattern right right and you can replicate that activation pattern in the AI but what you can't do in the AI is replicate all the different things that once means to you I went to the movies once really only once you went to the movies once upon a time I know that as the beginning of all fairy tales so it brings in a completely different contextual meaning so where along on that line of comparison do you get to interject what we do that machines don't which is intuit and find nuance
right that's that's a great question and maybe I should emphasize one thing which is that when we do this comparison we don't actually train or tune the algorithms to resemble the brain we don't actually try to inject this knowledge okay we just have these AI algorithm that we can use uh offtheshelf open source models uh either produced by our colleague colleagues uh or by the rest of the of the scientific community. And the these algorithms, they they're not trained to mimic the brain. They're trained for whatever other purposes to be chat bots and to recognize cats from dogs in in images.
But what we observe empirically is that training these algorithms tend to make them generate representations which are comparable to those of the what we do in our brains. Okay, first of all, that is scary af. I mean, it's fascinating and it's really cool, but it's also kind of scary.
Tell them what AF means.
Uh, it's scary as But the reason why it's a little scary is because on the one hand, it kind of diminishes us as this crowning jewel in all of creation with the zenith of intellect that we believe that the
zenith of anything. That's Yeah. Couldn't we do with a little bit of humbling every now and again?
I don't know about you.
No, no. Here you go. No. Here's how you get out of that. Here's Go ahead.
We are so brilliant,
right?
We created something more brilliant than ourselves.
So, I would I wouldn't I wouldn't say this quite quite yet because AI is really limited in in many ways uh today in spite of the hype. I understand the emotional reaction, but frankly, I also think that there is a source of marvel here, right? For for the first time we have AI systems or or systems that we we trained for a task, right? The task is surprisingly arbitrary or or even mundane, right? For instance, trying to predict the next next word given the preceding words. Like that sounds like
I mean that is what all LLMs do.
Exactly. Yeah. Large language model.
Thank you. Yes. Large language model. And this simple task pushes the algorithm to generate hidden latent representations which resembles those that we have in our own heads. And that suggests something to me which is very profound right which that they they exist general principle that push these uh systems biological artificial systems to generate a similar computational path a similar set of representations. So,
wow.
Is there a similarity between the brain and how it processes data in its architecture and that of a large language model that it's learning in a very similar way to the human brain?
Because, as I understand it, the original idea of neural nets as invoked in computers was an attempt to mimic what we thought our brain wiring was doing. And we learned that that's not really how our brains work, right? Yes.
So it's just dangling there now as its own thing with its own utility but it's no longer biologically biological analog.
Yeah. So the the history of of AI and neuroscience intertwined uh quite a lot but for a long time it was these links were metaphorical like the idea of a neural network was it wasn't I think a useful concept but it the goal was not to be as close to the brain as possible. In fact, it's really a huge simplification this this idea of artificial neurons as compared to what was already known at the time and and you've had these bridges uh between uh the two disciplines in for many decades. What is different now is that this this comparison is not just like conceptual like kind of loose. It's very precise. is we can quantify the extent to which the activation patterns in the brain and the activation patterns in these AI systems do look alike uh or not and even even though these systems are not built for that for that purpose. Now having having said that I I also feel the need to mitigate this result because this is a tendency that we have but we also see a lot of edge cases where this does not work. uh so typically if you take the very best model the largest model this uh similarity tends to break down. So we we do have cases where the what we call the convergence of representations between AI systems and the brain is not monotonous is not uh systematically the same. Yeah.
All right. One thing I haven't sort of got to grips with the speed at which image back a brain and how how quickly that is and how quickly you're able to then process that data back through an algorithm
in the head.
Yeah,
in the head it's it's quite slow actually. So uh when you look at reading for instance uh you flash a word onto your retina uh this takes for about 70 100 millconds to really blow up the uh visual cortex in the occipital lobe and from there you'll get another uh 50 milliseconds for this visual information to be processed.
A millisecond is a thousandth of a second.
A thousand of a second.
So 50,000 of a second would be 500ths of a second.
Yeah, that's correct. Okay. Yeah, I'm not good with math, especially not in my native language.
We mess it in my head if you're not good with math. No,
but so yeah, around 100 millconds, this is really when the activity peaks in the uh in the visual cortex for the sensory processing, let's say. And then this information is being analyzed into u edges that will eventually construct the the the representations of letters and of morphes of words. And this is around 200 millconds. So 1/5if of a second and then it takes another 200 millconds. So around 400 millconds does uh the semantics part of words really uh rise and and rise in the brain and is broadcast to a wide variety of of brain regions. And so this this process is relatively slow slow. It takes about half a second for you to analyze.
How fast would a how fast would machine learning do it? Uh or if let's say an an OCR how how fast would it know? Yeah, in terms of inference, the the machine would be much faster. It would be just a few milliseconds. It was a
few milliseconds to do the whole process and we take like half a full half a second.
Absolutely.
So, we're basically like duh
well,
at the at the infant stage. So, what we
stop giving AI ideas about what to do with us when it when it becomes our overlord.
What what is uh fast is what we call the infant stage, right? So when once once you already train the algorithm using it is actually very fast. What what is typically slow is loading the information onto the graphic card. But when once it's it's there, it's actually very fast.
However, training these algorithms is ridiculously slow. Right? If you want to train an LLM today, a large language models today, you need trillions of of words, which represents
many many many lifetime of just reading uh all of the text that we created in humanity. That's that takes us back to what we were talking about earlier. So in order for it to know, it has to see all the words. In order for us to know, we just have to see like a word and then something similar and we're like, "Oh yeah, it's that." You know? So that's what we're, for instance, a ball. If you show us a ball, you can show us one ball. We've never seen a ball before in our life. And you show us a ball. And then you show us a basketball and we'll say, "That's a ball." And then you show us a baseball and we'll say, "That's a ball." But the machine is like well I have never seen that before. So that's the difference.
Yeah. This is one of the many differences. I mean when when we emphasize similarities we emphasize the similarities because because we are in a field of differences. Everything is different. They are the architecture is different. The type of data that they receive is different. The the training the of course the physical instantiation uh is is is different. This is also highlighting why we are all the more surprised and interested in in the fact that in spite of all of these differences, we can still find similarities in the way they process information.
Wow.
And you've got
when you've got these sort of caps that you're putting on, they're sensitive enough to be able to operate at that sort of speed. I know you say it's slow, but that for me is really quite fast.
So for for it depends on the device. So with functional magnetic resonance imaging fMRI you get a snapshot of brain activity approximately every 2 seconds. So a lot can go on within 2 seconds. However, if you take magneton photography you can get a snapshot every millisecond. So you'll get a much more wellresolved uh signal in time but the spatial resolution now is much lower. So you tend to have a blurry image let's say of of brain brain activity. So you have a trade-off between uh between these different technologies.
Wow. So the it's kind of like cosmology. The better your like looking tools get, the easier truth of science.
It's just going to be so much easier for you to figure out anything you need to figure out. It's just a matter of we got to be able to see it faster and then know more clarity. I
got a question. I when I think of the brain I think of it as this organ and there are these parts of the brain that are similar from one person to another even if there's differences in detail. Are we to believe that the brain knows that in advance how it would divide up its territory or are we all just socialized the same way? We all grow up in a civilization and so we all have the same influence on our developing brain
for it to take the shape the way it does. So this is a very profound uh question. There is a tension. I'm not a I'm not a historian of of science, but there is a tension in in the field that dates back to philosophy between empiricists, people who think that the uh structure of of representations come from the data with to which you are exposed and rationalist the people uh who would rather emphasize the importance of innate representations and innate structures. Um so you have for instance Plato on the one hand if we take this back to uh to ancient Greece there would really be in the rationalist point of view with this idea that you there exist innate representations in ourselves and ultimately we can approximate them with with reasoning whereas other people and I think that the whole study of of AI is really on the extreme empiricist side is just let's take a blank system and just press a lot of data onto it and ultimately it will manage this system will manage to to perform a task. And what's is interesting nowadays is to see that irrespective of whether the representations are innate or acquired through uh through exposition not necessarily culture but even just sensory data um they they seem to at least have some some some similarities. This is what I I think is interesting in the case of this comparison between AI and the brain for language. uh the the brain is obviously very is structured very differently to uh to these to these AI algorithms and obviously there must be some innate structuring in our brain. This is why only human have language in a sense of being able to combine words together um uh in order to to reason and to to communicate. This is not
reminder comic two
dolphins swimming together,
right?
They're in a water show.
Oh, okay. Like like the SeaWorld type
sea world they're swimming together. one says to the other of the humans, right?
They face each other and make sounds, but we're not sure they're actually communicating,
right? That's pretty funny.
But but there are a lot of of experiments.
Their brain's bigger than our brain. Yeah.
For some of them, not not all of them. But so the the reason why uh there is I mean there's been a lot of of experiments on behavior with dolphins but also with apes to try to see whether they would be able to combine concept and and there are some experiment that show that in in some edge cases they are able to do this but for now we don't have any evidence suggesting that you have any other species that can learn this vast amount of concept and a be able to combine this concept together in order to produce a sentence or to understand a sentence a new meaning that they've never heard before. So this ability must be to some extent input in in our genome and uh and and be um an innate an innate structure.
Well, it have to be. I mean, we're the only you and it's so funny because it's disassociated from everything else that we are and have language for instance. I can be deaf, dumb, and blind and you can teach me any language. I don't have to have an actual reference like everybody else does. So, I mean, we are truly unique in the way that we do communicate. I don't know if I mean, I'm sure other animals communicate, too. I'm not sure. All animals. All animals communicate, but we're very unique in that if I don't know how to communicate with somebody I meet from halfway across the world, we will find mediums that allow us to know each other's language.
Right? And this is coming back to the whole empiricist versus rationalist tension. This is why um there is something very interesting here. So we established that the uh human brain must have some genetic or innate properties for it to acquire language. And this is why it differentiate itself in part uh from from other from other animals. And we also know that these deep learning algorithms they have very little what we call inductive biases. the the architecture that we use in in deep learning, they are remarkably blank and versatile. And so it's really the data with which they are trained that pushes them to build the representation that they have. And nevertheless, it seems to be comparable at least to some extent to those of the brain, not in every way, but in in some ways. And so that suggests that no matter where you come from, whether you come from this really rationalist type of approach to uh to cognitive science or much more from an empirical empiricist sorry uh uh approach you there seems to be some sort of convergence between these two approaches.
What what I want to get into now is the application of your research where it could go as we progress with this. Now I said in the opening about people who can think but can't speak. Is there an opportunity with this research to give them a voice to have their understandings made made public made aware?
Right. So you have a indeed a lot of patients who uh suffer from an inability to communicate typically because of a brain lesion. So either traumatic brain brain injury or anoxia uh that will lesion the the part of the brain which is responsible for for instance motor control. So they will be paralyzed or um or or lose an ability to uh to to move their facial movements. And there are now a few team that have shown that it is possible to put a set of electrodes in the motor cortex and to use these neural signals to feed an algorithms that can then be used to to do a brainto text translation and allow the uh the the individuals to to regain communication abilities. So this is already something which is happening with invasive uh approaches. So with electrodes which are surg which are implanted with neurosurgery. Uh one of the goal of course is to try to see whether it would be possible to push this approach with non-invasive devices which which do not require a brain surgery in order to rehabilitate communications in patient but also perhaps to diagnose. So sometimes you have patients who do not respond but they are awake. It's a parad parodoxical uh state which occurs sometimes after u a coma. And in these patients you want to know whether they don't communicate because they're not conscious of the environment or whether they don't communicate because they are fully paralyzed for instance
or they just don't like you
and perhaps they just they just don't want to which is actually an issue right if you if you have lesions to the part of the brain that are intrinsically linked to motivation that could also be a cause of
demotivated of a of a I'm tired of talking to you.
I'm just done.
And so for these patients uh having devices that would uh allow us to well allow them to communicate but even to allow us to to know whether indeed they are conscious or not of the environment is certainly of of a of a prime use. Yeah.
Are we likely to find that sort of ability in the near future or are we having to wait
for invasive electrodes? This is this is already happening.
Yeah, You know, our boy Lee El Elon, that's what he wants to do.
Nurling.
Yeah. I want to put a link. I want to put a chip in everybody. Everyone's got
You can do that through the vaccines.
Yeah. Is that's Well, no, that's Bill Gates. Let's Let's get our billionaire straight. Okay. Elon wants to put a chip in your head. I mean, an electrode in your head. Bill Gates already did it. I think the limits of the technology as Neil's pointed out will be reduced because of AI and they'll find solutions sooner. But it's the ethics of being able to potentially sort of decode the brain's messages and then reverse engineer it so as you can read someone's mind. It's the ethics of that being possible because I think that's going to freak not just Chuck out. You know, I'm going to be honest, though. Isn't that already
happening? When you look at meta data that's taken from our phones and our location and other phones that are around us, couldn't you pretty much tell what I'm thinking?
>> Well, I don't I don't know. But what I can I I can say is these these are certainly topics that that come up uh very often. Uh and there are many several things to say. The first thing is that what is possible today in terms of decoding brain activity is really limited to specific cases like perception and motor control. And the reason for this is because we have uh we know what the person sees. So we can attach the image to the the brain patterns. However, as soon as we try to do this in imagination, for instance, as I as we mentioned before, uh then things becomes uh drastically more more difficult. Not just because of uh an inability of the algorithm to work, but really because a signal is just not there.
>> That means it's it's not likely that you will anytime soon read someone's dreams. So
>> until you get a signal booster
>> for from a stat okay
>> from a statistical point of view for fundamental research there there is research on the science of of dreams. However there is all of the evidence point outs to the fact that it will be uh very difficult with the state of knowledge that we have to have a device which can uh read your mind in the in the way that people think like with your train of thought and all this. And the reason for this is because even with the largest multi-million dollar type of uh devices that are being used, the signals remain extremely uh noisy and it's very difficult to to go beyond this. So the the physics of the signal that we pick up is really the main constraining factor not the AI algorithm uh part. So the AI algorithms can be used as a useful tool but in terms of the signal that you can pick up this
>> you can't generate the input necessary for them to to do a good job.
>> Yeah the data that can be collected with these devices remains extremely extremely noisy and so uh from that point of view the risks seem limited. Now this is the current state of affair but our role here as as scientists is also to say uh what what is possible what is the uh the state-of-the-art and to share this uh through uh through the research through open sourcing and all this that's that's the reason why we we do this work
>> in science you're always limited by your signal to noise
>> absolutely
>> the signals you'd have to add up days weeks month months of measurements to pull a signal out of that noise
>> but you want to have then this only works if you have the same the same signal that comes up again and again. Whereas
>> when people think of mind reading, they think of reading the mind at a given instant. You don't you don't think of the same thing again and again and just repeat this until your noise average is out. So this is this is why at currently all of the evidence suggest that there is not a systemic risk. However, technology continues to evolve and we want to make sure that the the risk are limited and this is also why we engage in in these kind of discussions of course to ensure that uh the discussion does not just happen within the scientific community but with the rest of the
>> so when is the time to make that determination is it now before you actually have the equipment to measure this determination
>> so the determination as to the ethics like codifying the ethics themselves
>> guard rails going forward when do you come up with those guardrail Because if you come up with them after you're able to do it, it's the you know the barn is they the horse is out of the barn as they say.
>> Yeah. Absolutely. So this has already uh started right. There is already a lot of regulations on what you can and cannot do. For instance, I'm work in France and and so we have the GDPR in Europe. It's constrained the way the the the data that is being collected from brain imaging can be used. In France, for instance, you're not even allowed to do neuromarketing. The the you can you're not allowed to use brain data for marketing purposes. So this discussion is obviously already engaged and along the way we need to to continue and update these decisions with the the state of knowledge that uh that continues to evolve. Yes.
>> What was the movie Minority Report where they had this sort of
>> precogs? Yeah. Precogs.
>> I mean that's great movie.
>> That's everyone's default thought as regards this research and where it leads to and I think that's what scares them and I think they'd be grabbing not just for the guard rail. I mean, do you look
>> Well, precogs are you were not digging out of their head what they saw from the past. You were digging out of their head what they foraw in the future.
>> In the future,
>> right? So, that was different. So, they would see you committing a crime that you haven't committed.
>> That you haven't committed yet, but you were definitely going to commit
>> and then they just go arrest you,
>> right?
>> They started doing that. The crime rate went to
>> kind of like immigration in America right now.
>> Oh, how interesting.
>> You pre-arrest people.
>> You just pre-arrest people. So what is the endgame for Meta in this?
>> I actually don't know uh why Meta hired me in the first place. So I can I can only tell you what what we're trying to do within our within our team. So the goal here is well the goal is well posed, right? We have now some preliminary evidence suggesting that they have you have similarities between uh AI systems and the brain. And that suggests something which to me is very intriguing that there exists these general principles that shape the information processing in AI system and the brain. So discovering what those laws are and also trying to understand what is missing in AI systems for them to be as intelligent as efficient as us remains a major topic of research. So this is why we're pushing on this frontier to better understand the brain and make better AI algorithms. So if you're able to achieve that, people are going to feel an invasion of privacy. They're going to feel thought security becomes potentially compromised. I mean, you said you've, you know, there's discussion over the ethical point of view. Are we looking at those sort of features as well?
>> So yeah, it's the same uh topic that we briefly discussed before. We have an ongoing discussion to try to see whether AI and neuroscience uh developments are changing the risks associated uh with for instance mental privacy. As of now the discussion is is ongoing but I don't think we have a change in a technology that changes uh changes the the risk. What we observe is that it is possible to decode brain activity in certain cases typically for motor control or for visual perception. But it is not possible to decode uh what you are thinking at a given moment or your train of thoughts or to extract your password from from brain activity. The reason for this is because the signal that we have
>> the thing that just takes the password out of your head,
>> right? Like the readers that they have now that steal your credit card and ship information.
>> That's radio frequency. All of the physics on which we base this analysis prevent us to work outside of the lab, right? So with an MRI, you need to plunge someone into a very high magnetic field. This is not something that can be translated for, I don't know, a consumer product. But what you could envision is a dystopic future where the state who has the power and the money to actually have a machine that could read your brain and during an interrogation extract information from you that you don't want to give up. You know, basically like, you know, you violated my mental privates. So, you know, that's actually foreseeable based on just what we talked about today. Yeah, I mean if we if we go down to the dystopic possibilities, I I suspect that the states will not need an MRI to force you to give away your password. But but it is
>> a good point. I'm just like, I'm not going to your MRI machine. I refuse. Just like Yeah. Okay. Yeah. This baton says different.
>> But but but still if the risk does exist, we should try to characterize it to ensure like what is the path to to that risk? And this is part of the scientific enterprise too. Yeah.
>> Cool, man.
>> You've got some new research that you're about to to release into the public domain. Can you sort of expand upon that for us, please?
>> Sure. Yeah. So, so far we've done this comparison between AI systems and the brain with adult participants and to some extent this is frustrating because there is something uh which is missing here in the picture which is the the learning process, right? So in the case of of language, we don't just want to understand how the brain process language, but we want to understand what makes it able to acquire it so efficiently. Like we just with just a few words uh we we acquire language. The average number of words that we hear is typically around a a few uh thousands per day. A few dozens of thousand per day. Um and uh if you compare this amount of data to the data that which is input for the training of AI models, this is really a droplet of of information compared to the oceans of data that these algorithm use. And so what this means is that fundamentally the architecture or the training principle that we use for AI, they are really really mediocre, right? We need to understand much better how you can get to a system that learn learns much more uh much more efficiently.
>> So if you train your AI on children, you you may end up learning how we actually learn or acquire language. But then you're also going to have AI saying things like I hate you so much. I hate you. You never let me do anything.
[Music]
>> But certainly what we it would be uh important to understand not necessarily to train AI uh models with with this data but to understand the principles that allow young children to acquire language so efficiently. This is one of the of the big marvels of our species and and this is certainly what we're trying to do.
>> Yeah. So this is actually a work with a hospital the rush hospital in Paris um that has a unit for epilepsy in in and and young patients down to two two years old patients. So you have these patients who suffer from intractable epilepsy. Again, same patient as we mentioned before. We have electrodes that are implanted inside the brain in order to identify the location that is generating the seizure and who can uh stay for about a week in the hospital and listen in in that context to uh audio books and then we can time lock the brain responses to uh to each and individual words to try to understand how the representations of language are processed in these uh in these young patients and how this evolves with age. Let me see if I can offer a perspective here. I'm as big a champion of AI as the next person. But I still enjoy being human and whatever I can do to distinguish being human from a machine, I will embrace, leaving me to wonder whether the true creativity of what it is to be human may actually lurk within the noise that can be never read by a machine. the first person to paint an impressionistic representation of reality. Could a machine have had that first thought? Or is that a human being rummaging within the noisy confusion of our own brains, pulling out something that no one had done before, no one had imagined before, and in the end genuinely creating that which is human and can never be machine. I just wonder that is a cosmic perspective.
>> And that was beautifully said except the first impressionist was just some dude who was nearsided.
>> Was all just fuzzy.
>> Just fuzzy. It's just how he saw the world. People were like, "What an incredible interpretation." He's like, "What are you talking about?"
>> So, thank you for visiting. All right, Chuck. Always good to have you, man. Always a pleasure,
>> Gary.
>> Pleasure, Neil. Thank you.
>> Thanks for to you and and and Lane and others for coming up with these topics.
>> Oh, they they keep coming up with them. So, we're going to keep finding them.
>> We chase them down.
>> Yep.
>> All right. This has been Star Talk special edition. Neil Degrasse Tyson, your personal astrophysicist. Do keep looking up and in.
[Music]