Transcription
Hello everybody, Adam LK here, and today we're going to be talking about applying different memory frameworks towards agentic setups. The idea of memory has really become a core concept when building agentic language model-based systems. The reason behind that is because when you consider a language model and actually interact with it, every time that you run inference or call the language model, it's considered to be almost stateless. What we mean by this is that language models don't have any prior recollection, any prior knowledge, or any prior contextualization past what you have actually given to the model itself to do its generation step.
In the process of creating agents and agentic flows, essentially what we're trying to do at the core of it is simulate the way that humans actually approach different problem-solving techniques. The limitation of not being able to contextualize what's going on in the current environment really is detrimental to the language model's ability to fully execute on a given task. In comparison, when you and I actually approach a task, we bring all of our general knowledge that we have about the world, gained through all of our lived experiences. We also have the understanding of prior similar tasks that we may have done in the past. We have the takeaways and reflections that we've gone through after executing prior tasks, as well as what we've actually specifically learned how to do or have been taught. This allows us to instantly contextualize and shape our approach to any given task while we're working through it.
At the heart of it, what all of this is coming down to is that we, as humans, have advanced memory and the ability to learn from and apply those learnings towards new experiences, whereas things like language models do not. To compensate for this, what we can do is actually take some psychology concepts and model different forms of memory recall and continuous learning within agentic system design. More specifically, what we're going to do through the course of this video and this notebook is create a retrieval-augmented generation-based agent that models four different kinds of memory:
1. Your working memory, which covers the current conversation and media context within it.
2. Your episodic memory, which in the case of a chatbot would be historical experiences and their different takeaways.
3. Semantic memory, or the knowledge context and factual grounding.
4. Finally, procedural memory, which outlines the rules and skills for interaction.
All of this is essentially going to be an approach to and an exploration in modeling, architecting, and understanding cognitive design when applied back to agentic applications. We're going to break down each type of memory and provide an example of how to implement them, as well as how it relates back to how our own memory works. But right before we get into that, much of this was actually inspired by this paper here, "Cognitive Architectures for Language Agents." So if you're interested in some of the different frameworks that these cognitive architectures offer, I'll definitely be linking this paper in the description below. Go ahead and give it a quick scan.
The first type of memory that we're going to be modeling, and the simplest form to actually model, is what's called working memory. Working memory essentially encompasses all of our active, direct understanding and contextualization of immediate information that we are processing. For a chatbot, this represents the maintenance and manipulation of conversational context observed through the real-time interactions that are going on. For humans, this is essentially our short-term memory or the immediate things that we are experiencing as they're being experienced, and how we're able to contextualize everything that's been going on in the immediate or the short term to better approach something.
This is the most common form of memory within agents or chatbot-based experiences that leverage language models because essentially, what it's going to be is keeping track of all of the different messages that have gone back and forth between both the user and the AI assistant. As each message is generated, it's going to be added to this list of messages and then reprocessed through to generate some additional messages. Working memory then really functions as the active computational interface here, coordinating all of the information flows between all of our other memory systems that we will implement in a bit, while maintaining precise state awareness of the current interaction.
To put a little bit of a framework around what remembering and learning from working memory could be considered, remembering would literally just be having direct access to that recent contextual data and the different action-result pairs. This is just going to be the continuous re-ingestion of the entire conversation back through your language model to understand, in the short term, how the conversation has been actually flowing. Learning from short-term memory then is just going to be the dynamic integration of new messages into the active context, updating our stateful representation of our entire conversation, which will then be able to maintain the temporal co-expense across our entire interaction.
This is more of a real-time learning process that's going to be different from the more persistent mechanisms that we're going to talk about in a second here. As the conversation is guided and we understand what has happened prior in the immediate conversation, we can then better guide our system towards whatever end goal we have.
To put this into code, we'll be using a little bit of LangChain. First, we're going to instantiate our language model, which in this case, we're going to be using GPT-4. Then, essentially, what we are going to do here is have a very simple system prompt, which is just going to be, "You're a helpful AI assistant. Answer the user's query succinctly in one sentence." We'll start then the storage for our historical message history. This is going to be our active working memory, and right here is just going to be a list of back-and-forth messages. So we'll put our system prompt here at the beginning, and then what we're going to do is get a user's message, append it to the list, so then it'll be the system prompt plus the user's message.
Then we'll pass all of those messages to the language model. Once the language model creates a response, we will add that response back to our list of messages, thus extending out our full working memory. This will continue to cycle, increasing the actual length of the working memory as our conversation flows. Interacting with the system, if I just say "hello," we should get a regular response back. But then if I ask, "What's my name?" it will say, "I do not have access to that information." Well, I can say, "My name is Adam," to which it'll say, "It is nice to meet you." Then I can say, "What is my name?" to which it will be able to say, "Your name is Adam."
Looking at the full back-and-forth history, we can see that there were a total of nine messages. We started with our original system message, and then when I said "hello," it responded. As these messages are being generated, they're continuously being fed back into the language model when I type in a new message. So when I said, "What's my name?" this message, along with all else, was returned back to the LLM, and it responded with this. That's why when I asked finally at the end, after revealing my name, what my name is, the entire working memory was re-ingested to generate this final response, to which it knows you said your name is Adam.
As I mentioned, this kind of keeping track of all of the messages and then re-ingesting them to provide the immediate context across our conversation is the most common form of memory that you're going to see that's pretty much baked into every single chat-based experience when it comes to language models.
With the idea of working memory out of the way, we can level this up one more time with episodic memory, or essentially being able to recall over a series of prior working memories. More explicitly, we are going to say that episodic memory is a historical collection of prior experiences, which are called episodes in this case. These can take a few different forms; they can be both the literal recollection of how something actually happened, but then also some of the non-explicit stated takeaways. When encountering a specific situation, you might actually recall the similar, related, or exactly the same events that you have been in, as well as their different outcomes. Recalling how these went really shapes the way that you approach new comparable experiences.
In the case of a language model agent or a language model chatbot, this would then include both the raw conversations or the working memory that it has participated in in the past, as well as some sort of understanding gained from those interactions. Remembering then here is going to be implemented through this concept called dynamic few-shot prompting, where what we're going to do is automatically provide similar successful examples as well as instructions as each message comes in to better guide the language model's response on subsequent similar queries.
An important note that I made here is that when we do actually recall similar experiences to ones that we are actively in, we also extract the different takeaways or learnings from those interactions. Thus, learning in episodic memory is then going to follow two sorts of processes. There's going to be the automatic storage of these complete conversations and what we've actually gone through already in our working memory, as well as an additional step of post-conversation or post-episode analysis.
So what we're going to do is then, after each working episode, we are going to take that and run it through a generation step to create these episodic memories, where we'll store the raw conversation, but then also we'll extract out some specific reflections, looking at what worked well during the conversation as well as what didn't work. The episodic memory here is really going to serve as our system's experiential foundation. This is going to allow it to adapt its behavior based on these accumulated conversations that are stored in the episodic memory, as well as being able to maintain access to different proven interaction patterns and the associated learnings that it was able to recall from them.
Implementing episodic memory is where we start to actually put together a continuously learning and continuously improving system, so that it's going to learn not just from the individual interactions but also be able to learn from the different patterns and insights derived across multiple conversations. Storing the historical messages is going to be easy; we can already just grab those as it's happening. But creating then the episodic memory is where we're going to be implementing this reflection chain. Essentially, what I'm going to be doing is taking in the memory and asking the language model to generate a few things about it. This is going to take the form of some context tags or a couple of keywords to help identify similar other conversations. There's going to be a short summary of how the conversation actually went, and then more explicitly, we want to ask it to look at what worked and also what to avoid across the conversation.
This is going to be implemented in this nice long prompt here, where essentially the main core of it is going to say, "You are analyzing conversations about research papers to create memories that will help guide future interactions. Your task is to extract key elements that would be most helpful when encountering similar academic discussions in the future." This is a little bit of a precursor to what we'll be doing with some of the other memory implementations, but the main course of it is that it is going to review the conversation and create a memory reflection, where it will return back all of these previously defined categories. The rest of these are just going to be few-shot examples for all of them and then a placeholder for the conversation, which what we're going to do is take advantage of LangChain's chaining capability to create a simple chain that will pass in the prompt to the language model and then parse it into a JSON object as a bit of a pre-processing step.
What I also have here is a function that will format the conversation, and by formatting the conversation, I essentially just mean putting together the conversation, removing all the extraneous data, as well as removing the overarching system prompt, since the system prompt here is going to be the main brain, per se, that we are going to be recalling all of our memories and updating as each message comes in.
If we show very quickly what running this short conversation through our reflection chain gives, we can see that we got the context tags of personal information, name recognition. The conversation summary is that we successfully recalled the user's provided name, saying that directly referencing the user's previously stated name worked, and nothing quite to avoid from this very simple conversation.
Now we have the ability to take our working memory and convert it then into episodic memory. So now that we have a way to actually create the episodic memories, we need also some place to store them for our long-term memory, as well as being able to remember or recall them. To do this, I'll be using vector databases with semantic recall as our memory kind of remembering structure. More specifically, I have an instance of Weaviate that I am running. The actual vector database that you use here doesn't matter too much; you could use your preferred one. I just wanted to check out Weaviate for this video, so I'm not going to go too much into actually setting up the database, but I have an attached Docker Compose file for running Weaviate within a Docker container very simply.
Then what I'm going to do is create a collection within Weaviate called episodic memory, where it's going to store the conversation. This is going to be our raw conversation, which represents our prior episodic working memory, as well as all of the generated episodic memory with the context tags, conversation summary, and what to avoid pieces here. We'll also have then a function for remembering or adding an episodic memory to our database, where essentially we're going to pass in our messages, which is going to be our working memory. Again, we are going to format those into a nice format with our helper function here, and then we are going to call our reflection chain to create the episodic memory, load the episodic collection, and then insert it into our database.
Remembering then is going to be simply running retrieval across our vector database. We will load the episodic memory collection, take in our query, which is going to take the form of a message in a bit here, and then I'm actually going to do a hybrid search, which is going to be a mixture of semantic and BM25 keyword retrieval. But this is all just going to return the top result, which should hopefully and likely be the most similar conversation to what this current message that's being sent is.
If we try this out, we only have one entry, our prior entry here, but we can see that through recalling, with talking about my name, we can see that we got the what worked, the conversation summary, the context tag, and then the full conversation here, as well as the NA from our what to avoid, which it didn't have anything to avoid. So now we have a way to actually convert working memory into episodic memories, as well as remember them by both loading them and recalling them from our vector database here.
Now what we need to do is combine it all together and actually apply our memory or episodic memory recall into our conversation flow. To do this, what we'll be doing is actually dynamically modifying the system prompt across the conversation by dynamically inserting in the latest conversation, including the last three conversations that we've recalled, as well as keeping a running list of all of what's worked and what to avoid that's been retrieved across our episodic memories. This should allow us to get the immediate context of the most relevant episodic memory to our message, as well as just keeping a running tab of some of the prior ones that we've recalled.
Our new conversational flow will look a little bit like this: as a user inputs a message, what's going to happen is it's going to ping our database for the episodic memory, retrieve that, and dynamically insert it into our system prompt. Then it's going to rebuild the entire working memory, updating the system prompt, attaching back all of our prior working memory of our prior interactions and then the new user message to generate. Finally, it will pass it all to the language model and generate an AI message back that is then going to loop continuously, with each new user message getting a new memory, changing the system prompt, reconstructing the working memory, generating a message, and so on and so forth.
When we exit out of the conversation, what we're going to do is convert the current working memory into an episodic memory, so that we'll just be using the add episodic memory function that we defined earlier. Checking this out, if I just say "hello," we'll see that it'll respond with "hello." Then if I ask it something that we haven't actually discussed before, like "What's my favorite food?" it should say it doesn't have that information. But I can say, "My favorite food is chocolate lava cakes," definitely up there. But then I can also say, "What's my name?" and it says, "You said your name is Adam," which means that it's able to accurately retrieve back those episodic memories and everything that we know from that.
When I asked, "What's my name?" it likely got the conversation where we talked about my name, and it now has the context to remember that it's Adam. So when I exit out of this, we are going to convert and store our memory into episodic memory. Looking into our message history, we can see then the current system message. We see the "You're a helpful AI assistant," and then we can see the current conversation match, which is going to be one of our prior conversations where we talked about my name. We also have the different prior conversations here that it's going to attach, all of the different times that we talked about my name, which I ran through this a couple of times.
Then we can see that it has started to add in what has worked well from recollecting on these episodic memories, so it's attached directly stating and then querying the user's name, directly referencing the user's previously stated name, all of these good things. Then we say to use these memories as context, and that is how when I ask for "What's my favorite name?" it's able to say that my name is Adam. Now that we have this message in our episodic memory, what we should be able to do is ask, "What's my favorite food?" and it should be able now to find and recall from our prior conversations that my favorite food is chocolate lava cakes.
If I just exit out of this without saving it, we can see that looking back into our memory, we can see that it recalled the current message, the current conversation where we actually talked about chocolate lava cakes, and then used it to answer my final question. Now we have a way to actually recall prior episodes and the prior conversations that we had, simulating our episodic memory, as well as creating these different reflections so that we can have some sort of takeaway and learnings from the prior different interactions that we've had. This should now effectively, for our example here, simulate our episodic memory.
Now what we can do is implement our third kind of memory into the system, which is going to be semantic memory. Semantic memory here is really going to represent more our structured knowledge of different facts, concepts, and their relationships. This is going to be more so the "what we know" rather than "what we remember experiencing." Semantic memory then is going to be kind of our understanding of facts and accumulated world knowledge. Grounding in some sort of factual understanding when it comes to language models is really going to manifest itself in kind of traditional retrieval-augmented generation, where what we're going to have is a database of factual knowledge that is dynamically retrieved.
Approaching something like learning in semantic memory is going to be a little bit more abstract. It's literally going to be expanding or refining whatever your current knowledge-based setup is in your database, adding or updating or somewhat transforming the different information on whatever topic you have in there. Remembering would be actually the retrieval step of this and being able to synthesize all of the relevant retrieved information from this knowledge base to provide those accurate and contextual responses. It's a little bit weird to think of just your traditional retrieval-augmented generation flow as being a memory system, but really that's a little bit of what we're trying to simulate with it.
We have our grounded factual knowledge, and we want to be able to instill that factual knowledge to contextualize the outputs of a language model. So what we try and do is recall or retrieve the most relevant parts of those facts to ground our responses. In the end, what we'll show is that we will combine semantic knowledge with our current conversation context, which takes the form of the working memory, as well as past similar interactions, which is going to be our episodic memory. That should help us to then provide comprehensive, accurate, and contextually appropriate responses.
The system will not only know what it's talking about by using the semantic memory, but it can also relate it to the current conversation through working memory, as well as through past experiences in episodic memory. For our semantic knowledge, then, what we'll be using is the Koala paper, "Cognitive Architectures for Language Agents," that a lot of this video is actually inspired by. What we'll be doing is going through chunking, embedding this also into our Weaviate instance. Time and time again, people have shown how to both load, chunk, and embed documents, so I won't go over all of the details. But if you're interested in some of these actual custom chunkers that Chroma DB put out, I did create a whole video all about chunking and some of the optimizations that you can do, which will pop up on the screen right now.
For the most part here, I'm just going to be using a recursive character length chunker with a chunk size of 800 characters, splitting out my Koala paper into chunks and then embedding it into a different collection in the vector database. This one is literally just going to have a single property, which is going to contain the text chunk. We'll, of course, then have a memory function, which is going to be very much just a simple retrieval function where we load the database, do our semantic BM25 hybrid retrieval, and then combine all of the different chunks into one total string.
If we just load this function and then query it with "What are the four kinds of memory?" what we'll see are all of these chunks. We have a 15-chunk limit, and it's just going to format this nicely into one large string here. Incorporating our semantic memory back into our working memory is going to look a little bit like this. Essentially, what we're going to be doing is doing the regular recall, creating this chunk string here, and then attaching it to a quick prompt that just says, "If needed, use this grounded context to factually answer the next question. Let me know if you do not have enough information or context to answer the question," where we will insert this chunk here.
Then I'm actually going to be returning a human message, so I'll get into that in a second when we go over the full diagram. The output of this, if we use the same "What are the four kinds of memory?" query, becomes then this human message format that is ready to be added on to the working memory and passed to the language model. Now, when a user sends a message, it's going to do the flow that we were just talking about with episodic memory of retrieving the most relevant conversation, as well as the learnings, attaching that all to the system prompt, and then inserting it into the working memory.
But also what's going to happen is the user message is going to go to the semantic memory, retrieve all of the relevant grounded factual context that we have here, which is going to be formatted as a message, and then what that's going to do is be added on top right before this final user message. Essentially, now we're going to have both the system prompt be changed, as well as adding in an additional message along with our existing user message to provide the semantic context. This is going to happen with every single message, so all of the messages are going to affect the system prompt and create a new semantic memory message, and then reconstruct the working memory.
We're not going to actually save all of these semantic memory messages; we're going to be removing them and then replacing them as each individual user message comes in, so as to not bog down the entire system with so much context being retrieved all at once. Let's give this a shot now. We can just say "hello," and it will just say "hello." But also what you can see now is that it said, "How can I assist you today, Adam?" So already we can see the memory coming into play.
Let me ask it, "What are the four kinds of memory?" to which it might take a second longer to create a longer message, which it'll tell me: working, procedural, semantic, and episodic. So that is perfect. Let me also ask, "What's my favorite food?" Let's see if it really is able to dig into the memory. "My favorite food is chocolate lava cakes." Awesome!
Exiting this conversation will then save it back to our episodic memory. So now we have all of this. We can then see if we format the conversations that this is kind of the full conversation, removing out the contextual semantic message that we dynamically insert, as well as the system prompt. This is just going to be the raw back-and-forth conversation. But then we can also check out what the context message looks like. This is going to be the last message that was actually inserted in between me saying, "What is my favorite food?" This was all there, but it was taken out when we formatted the conversation, so we can see all of the chunks that were retrieved there.
Now if we go back, and since that conversation is now in our episodic memory, we can ask something like, "What have you told me about memory before?" It should be able to look through and say, "From our previous conversations, it's mentioned that memory, particularly in the context of language agents, has all of these," and some good stuff like that. Then I can ask about "What are some concepts of learning with agents?" and it will continue to be able to ground its responses with the semantic information and add in some incredible stuff there.
Now we have both the episodic memory, semantic memory, and working memory all working together. The last and final memory system that we will implement and discuss is then going to be procedural memory. Procedural memory is a little bit different from our working, semantic, and episodic memory since what it really does is it covers more how we actually remember to perform different tasks or follow familiar routines. For example, something like riding a bike or typing on a keyboard are all things that we unconsciously just remember how to do. So it's really the "how to do things" type of memory.
This is very distinct from our factual semantic knowledge or our specific experiences, and what it allows us to do as humans is it enables us to execute complex sequences of actions without the conscious recall of each individual step. This is kind of the most abstract of memory to think about in terms of language model systems and agents because, at its core, procedural memory is better represented as literally the language model's underlying weights, as well as the code that the system follows.
This is really going to be these learned patterns that are stored right within the language model as weights and the code base that actually orchestrates all of the memory interactions and really shapes our system behavior. Learning in this sense is going to be a little bit more strange because learning would more explicitly be for procedural memory, actually fine-tuning or training your language model or making some sort of update to the system's core code. Naturally orchestrating automatic refinement of your literal program or the code that you've written by a language model or allowing a language model to be fine-tuned can be either time-consuming, complex, costly, or not very reliable.
So we'll be implementing a kind of smaller, low-scale version that's not true procedural memory in and of the sense here that is following the Koala framework, but still has some of the main core ideas. Do let it be known that it's not impossible. Some of the AI researchers over at Salesforce actually published this "RetrFormer" paper here, where they do actually do iterative training of a language model to learn from, reflect on, and guide further responses of a separate language model. Essentially, their memory is implemented within this retrospective language model that they're calling.
What the retrospective language model is going to do is it's going to internalize all of those memories and understandings into its weights so that it can better guide and provide further instructions into the main language model. I'm going to take the idea of the reflection response and putting it into the prompt here from RetrFormer, but not actually do the model training. As for the other kind of procedural memory of actually having and editing the code, a different paper, "Voyager," here, which more amusingly is actually an embodied agent that is able to operate within the world of Minecraft, does take this idea a little bit further, where what it allows it to do is actually create functions and edit functions to do things like combat zombies or mine logs.
This is literally going to be code generated by a language model that becomes part of the system itself and then can be edited, reflected on, or refined. While it's not impossible to implement some more of the advanced procedural memory updating systems here, it can be a little bit more advanced and tricky. What we're going to do instead is we are going to define persistent instructions, and the hope here is going to be that these instructions are going to be learned from all of our prior interactions and will be attached within our system prompt, updated as a running ongoing persistent list to further guide the conversations that we have.
This is going to extend the original prompt with its episodic memory to now include some sort of procedural instructions. Essentially, what we're going to do is what we did with our prior episodic system prompt: we kept a running list of what worked and what to avoid. After every conversation, what we're going to do is run that through a reflection step of the current takeaways, which is going to be our existing procedural memory, the what worked well and what to avoid feedback, and then have the language model generate or maintain an ongoing list of 10 procedures and 10 takeaways that it should follow every single time.
For this simplified example, our procedural memory is just going to be stored as a text file that we'll read and write to. This is going to be persistent and maintained across all of our conversations. I've already gone ahead and generated, with some prior examples, this list of 10. These are going to be these guiding general procedural instructions that are going to be maintained and then updated as new things are learned and internalized.
What we'll do, as I mentioned, is we're going to extend our original episodic prompt to now include procedural memory here. So it's just going to be everything that we've done the same, but now for every single prompt, our current system prompt is going to include the existing procedural memory at the end. That's not going to change at all and will be part of every single system prompt.
For the complete system, what we'll have is the user message still go and retrieve any relevant conversations, as well as the what worked well and what didn't, enrich the system prompt with all of those. It will then also go to the semantic memory and grab all of the facts and grounded concepts, then create the semantic memory message. Our procedural memory will be loaded and also attached to the system prompt. Both the semantic memory, the system prompt, the new user message, and all of the old messages will then be reconstructed to create the active working memory, which will, of course, be passed to the language model to generate its final response.
Now, when we interact with the whole system, I can say "hello," and it'll do all of this and say, "Hello, Adam. How can I assist you today?" So it already knows my name, which is phenomenal. Then I can ask, "What's my favorite food?" to which it will say, "Your favorite food is chocolate lava cakes." Then let me say, "I hate emojis. Don't ever use them." So it'll say, "Got it. I'll avoid using emojis in our conversations." Perfect!
I can still ask questions from our semantic knowledge base and say something maybe like, "How do memory systems interact with agents?" We can see what it responds to that very vague question. Well, there we go. It says, "The working memory, the different long-term memories, and then interaction with internal and external actions." So that's great. It has its prior memory, and I've given it some instruction here.
Once we exit now, it is going to load this into our episodic memory, but also it is going to update our procedural memory. We already had this procedural memory that we were using, but let us real quickly delete that and then go back to our procedural memory here. We can see that it has been updated. Now we can have persistent guiding instructions that guide and show how the language model should act, which will be refined over time as it gets better and better at what it does.
If we look at our conversation here, then what we can see is just our regular conversation and how it flowed. Then we can check out what the system prompt currently looks like. We can see that it has some of the prior conversations, as well as the current conversation on here, and then some of our additional 10 guidelines for interaction, which is going to be our procedural memory now.
We also left out the semantic memory message, which we can also load here and check out. We can see that it was able to pull all of these chunks still and provide that as context to the language model. So now we have all four kinds of memory working all together. There are many different kinds of approaches to memory, not just the way that I implemented it up here. I've provided some examples at the bottom of this repository, including some of the ones that we were just talking about with Voyager and RetrFormer.
So definitely check out some of the additional ways that different memory systems are starting to be researched and implemented. One paper that I can definitely recommend is this one, "A Survey on the Memory Mechanisms of Large Language Model-Based Agents," which actually compiles together a lot of different examples and all sorts of different papers and whatnot to reference and look at for all different kinds of memory systems. If you're interested in any specific ones or see some cool ones or just want a general overview, I would definitely recommend looking through this survey paper here.
As an additional note, I've ported this entire setup over to LangGraph as well, so if you're interested in seeing the LGraph implementation of something like this, I have this available in the GitHub repo that I'll have linked in the description below.
To summarize and discuss a little bit about memory and agentic systems, really at the end of it, memory systems enable us to move beyond language models as simple input-output models into agents that can really operate with forms of persistent understanding and learning. Each one of these memory types serves a distinct cognitive purpose. To break down all four real quick:
- Working memory is going to be your immediate cognitive workspace that will keep track of and contextualize everything that's happening right now. For language models specifically, this is going to directly combat the stateless nature by maintaining active context and also incorporating all of the other memory systems into the ongoing conversations.
- Episodic memory is going to cover our historical experiences and their associated learnings, not just storing the past events but also the ability to reflect on and learn from them through applying our memory of similar episodes to new experiences. This allows language models to extract the meaningful patterns and insights from experiences that we've gone through and use them in the future.
- We then also have semantic memory, which is going to be our pure knowledge representation, separate from specific experiences. Language models have knowledge baked into their weights, but semantic memory is going to provide the explicit, queryable facts that can ground responses and allow for the dynamic knowledge integration into our agentic systems.
- Finally, procedural memory is going to be both the model's weights as well as the explicit code that's orchestrating the whole system. This is going to shape how all of the other memory systems are actually used and how the agent executes its different behaviors. Unlike the other memory types, changes here are actually fundamentally going to alter how the agent functions at a system level.
Together, working memory is going to actively manipulate the current context, retrieving relevant experiences from episodic memory and grounding in semantic knowledge, all guided by our procedural patterns of memory. Each type then builds on each other to enable increasingly sophisticated cognitive capabilities within language model system designs, taking your language model agent from just being something that doesn't know much, doesn't learn, or can't fully contextualize the situation that it's in, up to the next level.
With all that being said, I hope you were able to get a fun overview of and maybe start thinking about how to apply these different memory systems and apply different cognitive architectures back to your language model systems. Being able to implement these things can really improve your performance by actually being able to give it the contextualization that more simple systems tend to lack.
If you liked the video and want to support the channel, consider subscribing and leaving a like on the video. If you have any questions or comments or further discussions about memory, leave them in the comments below, and I hope to see you next time. Thank you, and have a great day!