Transcription
Are we good on the mic? Yeah, there we go. I hear it now. Cool. And I don't I don't quite see my slides. If we could switch to um to this guy, that'd be great.
So, yeah. So, like Nia said, my name is Will. I'm a product manager at Neoraj. Uh I work on uh AI innovation team. So we spend a [clears throat] lot of effort making sure that NearJ uh remains the best graph database for AI use cases as well and so that's what my team focuses on. Um you can find the slides uh graph.com CG meetup contextgraph meetup uh or the QR code there.
So there's been a lot of like great discussion around context graphs. Um this this really like bubbled up and uh resurfaced I think after uh foundation capital put out this this this post about you know context graphs are uh trillion dollar opportunity uh for AI. Um there's a really good context graph paper from a couple years ago. um you know and there's been a lot of uh a lot of discussion about what's needed for context graphs the the challenges um and and a lot of good discussion around it. I think what what's common in all of the discussion around the context graph topic is it's all about this idea of the missing why right so that if we have I don't know an an agent you know recommends making a credit approval decision we need to understand like why like what is all the the data that goes into that decision and and this isn't just about uh AI agents right like um fundamentally a context graph is a knowledge graph that contains all of the information necessary to make decisions throughout the organization, right? Um and so we may keep these uh in a audit log uh which shows you know okay this is the you know line by line we rejected this transaction but why like what are the the specific pieces of the I don't know policy that that went into this this may be stored across lots of different systems that we need to sort of combine together and query across in one uh in one knowledge graph and that to me is is fundamentally what this idea of context graph is is all about.
Um so let's look at at maybe an example um in the financial services world. How many people here work in in financial services? I'm just curious. A few. Okay. Not not many. Yeah. This is this is uh San Francisco. So you know makes sense.
Um so we think of like okay what's like the data model that we might need for a context graph for the financial services world? Well, we're going to have people like we're going to have people that that have accounts at at our bank. There's going to be transactions. These are uh these are the events, right? So transactions are are what happened. Entities, the the people, the organization. Um these are, you know, the the things and then the context, the why. We're modeling these as like decisions, policies, uh the these sorts of things.
So let's take a look at an example. This is um this is online. You can you can run this context graph demo versel.app. Um we'll take a look at it. Uh so this is a a demo app that I put together because I I saw all this great discussion about context graph this and context graph that and you know we could do this, we could do that. That'd be really neat. I'm like yeah okay cool like let's try to take some something that looks a little bit real and and see what we can uh what we can come up with here. So this is the data model. Um we have this data it loaded in in Neoraj very similar to the um slide I just showed but a bit more complex right so our decisions they you know apply a policy but decisions also serve as a precedent right so we have uh relationships that connect our decision nodes um as well and and we also have some additional information like um alerts support tickets um this sort of thing so just a little bit richer data model and so we can think of this as let me zoom in as much as I can here.
So, think of this as like what uh I don't know, an analyst at a bank might use this sort of tool uh to respond to customer requests that that sort of thing. So, this is um this is a context graph agent. So, we're saying, hey, you know, we have a credit limit request from our customer Jessica Norris. She's requesting a $25,000 limit. And we can see here if we inspect the the details of our agent, uh our agent has lots of tools. These are all tools that we've defined uh how to interact with our context graph and of course it has a a system prompt. You are a helpful agent uh that that knows a lot about financial services and so on. And then we have the tool calls, right? So tool calls are the way that an agent interacts with and understands its environment. And we can see here all the tool calls that this agent makes to fetch the data to fetch the the context that it needs to make this recommendation. and and we render all the the pieces of that in a graph, right? So we can uh we can kind of see the different people, decisions, policies that were implemented here. We can like doubleclick and and traverse out through the graph to get more information for our analyst. We can also look through the uh causal chains of decisions and so on that that led up to this. And ultimately we get a we get a recommendation uh from the agent. Um in this case the recommendation is uh conditional approval and says hey do you want to record this decision? Great yes record this in the context graph and this will uh update our graph adding another decision node uh that will then serve as precedent and and so on going forward.
Um so again you can feel free to to play around with this online there. The the code is open source. There's a um a blog post that explains how it works as well.
Um now if we look at at you know what's actually going on here, how is the agent, you know, tracing this this decision history to make this recommendation for it. How is it using this context graph? Well, the first thing it does is it it searches for the customer, right? and and then uh looking for the context around that customer by traversing the graph and and finds oh well you know this customer um had a a previous fraud flag. Uh we can then use uh hybrid search so combined uh vector and graph search to find the most relevant uh precedents and and ultimately make our recommendation. This uh idea of hybrid search combining graph and vector by the way I think is is really important uh and really powerful.
Um Neo Forj uh has tooling called graph data science which allows us to run graph algorithms like uh centralities page ranks or here we're talking about graph embeddings in this case the the fast RP algorithm which is which is a way of generating embedding uh not just on like the content of text right so if you're familiar with uh text embeddings and you know maybe storing those as vectors and doing vector similarity research, right? Um that helps us to find you know in this case uh decisions and and and policies that you know textually like semantically are are similar but what about the the structure and that's where things like graph embeddings come in. We can actually look at the structure of a account's relationship uh to the way it maybe interacts with transactions, other accounts, these sorts of things. The fraud patterns that we see, these can also um be used to generate embeddings and then we can use uh similar vector search functionality um that we're familiar with to find the most relevant data u for our context. So there's a little bit more about what's going on in in the context graph demo. some of the other graph algorithms that are using it that are used some of the tools um sort of a resource for later there.
So that that's all fine and good. Like we saw we saw like a nice agent uh chat. We saw how it can help us make decisions and apply policies. Like that's great, but how do we actually build one of those, right? I think this is one of the the bigger challenges of of making use of a context graph is how to actually capture these reasoning traces.
Um and so at Near Forj, uh, we've built a NearJ agent memory Python package, um that allows you to add graph-based agent memory, uh to your agent and we have integrations pretty much any any agent framework, but I want to talk about kind of how we think of the the relationship between memory and a context graph.
So, nearj agent memory um this is a open source Python package that um you you can install uh and leverage again lots of different uh framework integrations. We'll talk a bit about entity extraction because I think this is um this is an important piece of uh the agent memory package.
So, there are sort of three highlevel abstractions of the type of memory uh that we're working with here and I think they're all needed for the context graph. Right? So going back to this slide, like context graphs need short-term memory, long-term memory, and this reasoning or decision traces, right? Like the the decision traces and and the auditing like okay, that that's obvious why we need that. Maybe less obvious why we need short-term and long-term memory as well.
So short-term memory, this is, you know, conversation, session state, like working memory. Think of it like this. Um, when we're using the NearFJ agent memory package and we save a message, this kicks off a background entity extraction job that that's kind of like moving us from short-term to long-term memory. Uh, and so we're looking for all of the entities, all of the connections between entities, the facts. Um, we're extracting that from unstructured message data and, uh, loading that into the graph. Now there there are several other um agent memory systems that have this concept of entity extraction to construct the graph.
Um one thing that that I realized or we realized when when we were building this it's really important is not just using LLM based entity extraction. Um this is very very slow and very expensive to send every message to the LLM and say hey what are the entities how are they connected? Um and so instead we have this uh this pipeline approach where we uh start with some of the um you know more uh probabilistic statistical methods like named entity recognition the the things that are available in like the spacey um Python package if you're if you're familiar with that. Then we use uh things like gliner 2 which is a local model um that is a a pretty small model that we can run on CPU um for specifically fine-tuned for extraction um and uh entity extraction and also relationship extraction which is quite nice. Then we only need to fall back to the LLM. Um so that's that's built into this package. I think that that was quite important for handling you know large amounts of of conversations.
So we talked about this entity extraction process. Um one thing that's important in extraction in in addition to like not just relying on the LLMs to do that for for cost and efficiency but also the data model that we use for induction is really important. It's really important that the memory that we are saving to the graph is relevant for our domain. Right? Like if we're if we're financial services, we should be uh making sure that we're saving financial services related data and not um you know unrelated things like I don't know where someone kid went on vacation or I I don't know whatever's in there that might not be relevant, right? And so by default we use this poll uh people organization location event plus object. So pole plus o model. Um, if you're familiar with poll, it's used a lot in uh like uh crime investigations and and and this sort of thing. Um, so this this is just the default model that we use though. So you have the ability to apply your own domain model. Um, and those are the things that are then extracted out like this. This I think is also really important for uh for agent memory.
And the the third piece here is the reasoning memory abstraction. I think reasoning memory is one of the more important pieces for building context graphs. And I think it's this piece that we actually don't see a lot of support for in um a lot of the AI agent memory frameworks that are out there. When we're talking about reasoning memory, this is uh you may hear terms like procedural memory is another one that that's used here. Uh so you can think of this as kind of like procedural memory with with a little bit more to it. So this is recording like the tool call traces that we make. Um these reasoning decisions uh are are stored in the graph as well. Right? So short-term long-term reasoning memory. These are the the APIs that we have available in near agent memory for not just constructing your context graph but also like searching it finding the most relevant uh context assembling that uh for your agent.
So when you combine these all together, short-term, long-term reasoning, memory, this is all one connected graph, right? So short-term memory are conversations, entities extracted from those conversations and then reasoning traces, tool call, reasoning steps, those are connected to the messages and the entities that are retrieved for the context. Right? So now that we have uh our memory in one graph system, we can query across uh data from all these systems.
Here's what this looks like. This, uh, an example of hitting the the three different types, right? Storing a short-term uh, conversation, searching long-term memory, and then, uh, recording a reasoning trace.
So, there's another example that I'm I'm going to touch on just quickly here. So, this is um this is another demo app that we built. It's online. Um, the code is open source. How many people are familiar with with Lenny's podcast? It's like a popular tech product kind of podcast. Um, so we took the the transcripts from Lenny's podcast, ran that through the Neo Forj agent memory as if it were a like a agent conversation and sort of constructed a a context graph from that. And and so we can do things like ask uh you know what are all the locations mentioned? So in in the Brian Chesky episode, Brian Tesky is uh co-founder of Airbnb and so we can um you know we automatically geocode and then enrich the uh locations that are extracted right so um Bolinus is mentioned I don't know Brian Chesy owns property there I I don't know I didn't listen to the episode and but then it's enriched by uh data from Wikipedia right in uh Risy the Rhode Island School of Design where Brian Chesy like went went to school right and So um I I guess I I guess what I'm just trying to point out here is like there are there are other uh types of information in the graph as well that we extract not just uh like textual data right like here we're working with geospatial data um and this concept of enrichment um I think is is also really important um cool so I'll I'll skip maybe going through more uh examples of that to talk about um just kind of the the architecture of the Linux memory example.
Um, you know, we go through this entity extraction to context graph construction and then we're able to create agents that are able to interact with uh with that context graph. Um one thing that that that might be interesting uh let's take a look. Uh so this Linux memory app so this is running this is running publicly. we can see like you know all the um all like previous questions that folks have asked. This is what folks are asking um publicly out there, right? And um what we do is we record and write back to the graph the reasoning memory from everyone's interaction with this agent. So I have this running in um NearJ Aura. This is Neoj's cloud uh service. So if I go I think it's this one. Um and if we open up Neo forj browser. So this is the like the query workbench um for Neo forj. And so we can see here like we can see the entities that were extracted um in the episode and uh kind of query that way. But I want to look at let's look at our uh reasoning traces that we've recorded. So here's um here's a reasoning trace that has uh two steps. So each step is um is a tool call and we're recording um you know the the tool call result. We're recording uh like the number of tokens used the the response and so on. Um and this becomes the basis for the reasoning memory uh and understanding the traces uh that our uh that our agent is taking. Cool.
So anyway, a little look at at what that looks like in uh in the graph. We've been um doing some integrations with lots of the different agent frameworks. Um this is a look at some of the cloud vendors with agent frameworks that we've built um integrations with the nearj agent memory package um with and these agent frameworks are like Google ADK um ads strands Microsoft agent framework they all have their own sort of like memory abstractions and so um you know we wire those up to nearj agent memory and then we publish a specific integration uh for those packages um and make sure we support their their models and um running on things like vert ex if if we're looking at ADK uh or bedrock if we're talking about AWS.
Also some other other functionality in some of the releases like MCP open telemetry support support opic as well if anyone's using that.
Another interesting demo app I just want to leave this as a resource. There's um an agent swarm version of this financial services demo. So if you want to see what it looks like to have a bunch of different agents with different personas. Maybe we have a anti-money moneyaundering agent, a compliance agent, a customer service agent, and they all have a shared memory layer. So, I thought it was really interesting to see what does this look like if we give, you know, not just a single agent, but across um, you know, all of our agents are both contributing to and understanding uh, fetching context from a shared NEFJ agent memory layer. Um, so that that one's quite interesting if you want to to check that out. just a link um of some some resources.
How many folks are registered for Nodes AI in April? Okay, that's that is like 10%. That is not nearly enough. Everyone needs to scan the QR code, register for this right now. Nodes AI. So So Nodes is Nefj's online um conference, the NearFJ online developer education summit. We do this in the fall, but we've just now started an AI version of this. Um so this is really fun. Um we're going to do workshops leading up to this as well. So definitely register um for nodes AI. We'll be giving lots of talks about context graphs um and you know lots of folks from the community sharing what they're working on um as well.
Cool. So that's all that I have. Um you can get the slides with the QR code or that link. We're going to save questions for the end. Uh all the speakers uh plus Emil who's in the audience somewhere uh is going to join us for a panel discussion at the end. So we'll save questions for uh for that. So that's it for me. Thank you very much. [applause] Thank you. Thank you so much, Will.