Transcription
Hello, I'm Christian CA from the Swart Center for Computation and Neuroscience, and I'm going to give you a little bit of a teaser of what this lecture here is about. So, this is an introduction to Modern Brain Computer Interface Design. And the first question is, what is a brain computer interface? And you can think of it as, um, in a sense, a direct link between the brain of a person and a computer. And more precisely, it's a system which, um, measures system activity or brain activity and then processes this activity somehow and converts it into an output, um, that can be used by a computer. And so, it can be used, for example, to, uh, restore or replace the brain's natural output, um, modalities, and for example, if they're damaged somehow. Or it can be used to derive outputs that are normally not available to a computer, such as the person's mood or things like that. And there's various other applications that can be built around that, some of which I'm going to show you.
So, when you, when I say measuring brain activity, how does that actually work? Um, there's a lot of different sensor technologies, uh, that can be used. And here's one example, that's EEG, or electroencephalography, which is one of the most useful. You might know this from the hospital, where EEG is used a lot. Um, but there is also much sleeker, um, devices like this one from Emotiv here, which essentially measure the electrical potential at, at the skin here, at the scalp, if you will. And the sensors produce readings which look somewhat like these traces here. This is 10 seconds of EEG over multiple electrodes, multiple sensors, where a person is trying, um, to imagine to speak. And so, as you see, there is basically nothing in the signal that really tells you what word or vowel the person is speaking at any given time, or thinking of at any given time. So, it's really hard to analyze that data and say, convert it into a series of letters. But that's basically the job of a brain computer interface. But I should say that, um, doing speech recognition from EEG is considered to be extremely hard, and as far as I know, no one has been able to demonstrate that, uh, challenge conclusively so far. But there's other things, such as reading out the current attention level of a person or so, at any given time, that you can, uh, manage.
So, uh, these traces that we saw are generated ultimately, um, by, by the brain, right? And so, um, the reason why BCIs can be built, brain computer interfaces, is, um, because there's billions of neurons in the brain that are engaged in their various tasks, U, and giving rise to, say, everyday experience. Um, and these neurons, uh, like here, um, in this picture, which, by the way, normally they are not multicolored, um, this is a particular way of coloring them. These neurons radiate electromagnetic fields, each of them when they're active. And, uh, that's what we can measure, and that's what we can analyze and, and basically reconstruct some aspect of the person's cognitive state from. They are also consuming chemicals and produce chemicals, which are, which give rise to other ways of, of building brain computer interface traces. And of course, all that has to happen in real time.
So, the question now is, how do you take the traces like those that we saw and process these, um, into usable outputs? And it turns out, this, this is a very hard problem. And, um, not only is it hard, it's, um, it's a problem that is just about as hard as taking a picture and asking a computer to label the objects in the picture, say, this is a cat, this is a dog, or as hard as taking a speech waveform and translating that, you know, sound and translating that into a series of letters. So, um, all these different disciplines, uh, that are in machine learning and computational intelligence and artificial intelligence and so on, all these different areas share the same building blocks, and many of those are also necessary to solve the BCI problem. So, that involves statistics. Um, we'll talk a bit about that. We'll also touch on optimization, which is very important to, to design well-performing brain computer interfaces. We also use some linear algebra, which is actually prerequisite, um, for this course, and this is perhaps 80% of the technology, you know, these areas also information theory and things like that, which, which are universal to, um, across various problem types like speech, vision, BCI design, and so on. So, that's generally useful knowledge. And lastly, there's also models and techniques and methods that are specific to brain function and brain dynamics. Here, in other areas, like in, in speech recognition, you need to understand that the physics of speech production in the throat and all that. And here, in this field, you need to understand the nature of brain dynamics to some extent to do a really good job.
So, at the heart of, of brain computer interfacing, like in many other areas, is what is called pattern recognition, or also machine learning, where the, where the idea is that, um, the, you use some algorithms to learn what kinds of patterns you need to be looking for in brain dynamics, and you look, you learn these patterns from example data, like from EEG, as we saw. So, this is a representation that may show up at some stage inside the brain computer interface, where you could imagine each chunk of the EEG, say, say 5 seconds, gives rise to one of these dots in this space. And, um, you might have chunks that were taken on different, under different conditions, like the person was excited, or the person was bored, or the person was in a particular emotional state. And you could imagine that these different labelings are represented here as colors. This data actually comes from a different task, um, but it's the same story for brain computer interfaces. So, the goal is to be able to create representations like that from data with which contain interesting structure, and then to be able to learn, uh, the patterns in here, and learn to recognize the patterns, and map that onto useful outputs of the BCI, like the person is excited right now, or the probability that the person just made an error is 85%, or things like that. So, that is sort of the core of, of a brain computer interface and machine learning.
And there's an interpretation to many of these, um, representations. For example, here's a pattern that allows you to discern between two conditions of a person. And that pattern is actually a dynamic process in the brain. It's a connectivity structure over time, which, um, basically explains how, um, different areas in the brain exchange information, uh, over, uh, over a short period of time. And so, this is a pattern that was actually learned using machine learning from, from point clouds like we had before. And in fact, this pattern here corresponds also to a single point in a space of 30,000 dimensions. So, this is the beauty of mathematics, in some sense. And we'll analyze these data with, um, uh, using, you know, MATLAB code and things like that. So, we'll be, especially in the exercises, scripting a few things by hand in MATLAB. And there's also a very advanced toolbox that we created at the Swart Center BCI Lab, which we'll be using extensively in the later parts of the lecture to build a well-functioning brain computer interfaces.
And before we get completely lost in algorithms and mathematics and all that, um, let's, let's at least bring up the practical uses of, of brain computer interface technology. So, one of the most serious applications of BCIs is, um, for people who have lost the ability to control some or all of their muscles. And when I say all of the muscles, there's people who cannot even raise an eyelid. And some conditions that lead to that are diseases like the locked-in syndrome or ALS. And this is, in fact, an ALS sufferer who is using a brain computer interface to spell. So, these people need a way to communicate with their family or with their caregivers, or be able to control their wheelchair. And BCI technology is one of the very few technologies that allows them, uh, to do these things. Uh, there's a few other ways to, to use residual muscular activity, but, but this is, um, one of the biggest applications in the whole field.
And there's also uses for healthy people who, um, don't necessarily have to use BCI. But here, let's say, you have cases where people are in demanding jobs or demanding situations where they might zone out for a second or so. And that's things that we can possibly detect, um, from the EEG, and where, where we can tell the car, say, and the person is going to slam the brake, um, and that can save you a few meters. This is actually from a braking or lane change type study. And then there's, of course, also leisurely, um, uses, such as computer games. Here is a, um, an old Star Wars game that has a BCI hooked up to it. And the idea here is, the force power of the player, um, is modulated by his, um, relaxation level. So, the more relaxed he is, the better the force works for him. So, um, and there is a variety of game uses, um, that exist for BCIs, and that's also one of the very first things on the market, this whole area.
So, I hope that gave you, uh, a nice little introduction of what this lecture is about, and I hope you enjoy. Thanks.