Transcription
Hello community. So great that you are back. Yes, we have a new transformer and I want to show you how to build your own hyperraph structure. So welcome to my channel discovery.
We have a look at the latest DI research paper and in about 10 minutes time you will understand why I tell you go beyond rack. Stop retrieving your text chunks based on a vector similarity and start building hyperraphs of your thousand documents. Retrieve the topologies.
So but let's start at the beginning. You know LLM and graph, this is just great. Now we have augmenting our LLMs with graph based on logical knowledge representation which facilitates here the compositional reasoning across all the beautiful knowledge domain and it just works great. And we also have it for here in context learning of structured knowledge graph in context for ICL. The model can then precisely delineate between the interconceptual relationship and kind of engage in a form of reasoning that maybe parallels you the human creative scientific thought process. Until we discover well this traditional pairwise knowledge graph structure is ill-suited for the pure scientific reasoning as it cannot adequately capture higher order interaction around multiple entities and this is something that we need in science. So this is not if you just want to have something here in I don't know social media, this is really if you go in deeper reasoning structures.
So simple solution. Let's extend our pair-wise graph to higher order mathematical networks. Come on. And here we have it today. This is our main study January 8, 2026. Isabelle Stewart here, MIT Cambridge and Marcus of course here. Beautiful high order knowledge representation for agentic scientific reasoning. And you know what the beauty is? Not only this study, it is here this paper but it is also here the GitHub repo I'm going to show you now. Wherever you look, even biological material intelligence, you have now mathematical framework spanning graph theory, category theory. We are moving heavily into physics, into mathematics because we want here to build the next generation of artificial intelligence. And you find it everywhere: higher order intelligence, multi-entity in the social network, in a structural hierarchy here of your materials. You can go here to a swarm behavior, neural processing, you go here compositional hierarchy of chemical structures, biochemistry, wherever you look, a graph is not enough anymore.
So therefore, hey, come on, we introduce hyperraph as a more natural representation here since they can now capture relationship among multiple entities simultaneously. So MIT reason analyzed here the reasoning capabilities of LLM applied to hyperraphs and they evaluate how effectively they retrieve and infer information from high order relational representation. And I'm not going to tell you a secret, it just works great. It is interesting. Let's reformulate this. So therefore, by representing now multi-entity statement as a native higher order interaction rather than just collapsing them into pair-wise edge, the hyperraph supports the global corpus analysis enabling finally a discovery of recurring higher order motives, densely interconnected communities and emergent mechanistic patterns that are directly encoded in the literature. But our simplification of a simple graph network, we have chosen the wrong tool for the job.
So we perform our inference on the constructed hyperraph with of course, we have to go multi-agent now where only one agent oh has direct access to the hyperraph and the remaining agent specialized here in a particular domain specific reasoning. I'm going to show you in a second. Okay, if you enjoyed it and said okay, I just wanted to have a basic idea, this was end of level one. Welcome to level two. The failure of the diadic reduction. Why? They argue now written here the argumentation of information theory in graph topology. They argue that the dominant paradigm of knowledge graph and this is the resource description framework here, the RDF triple is mathematically insufficient for a higher specific representation of more complex tasks because you know in standard knowledge graph you have an n-array relationship. Chemical synthesis in Wolfram for reactants, one catalyst and one product is projected into a pair-wise graph. And the most common projection is that the click expansion where every node is connected to every other node in the set. But in a click, the graph implies that the reactant A interacts with product B independently of the catalyst C. Now, you know this is scientifically incorrect. This is false because the interaction is irreducible. A only relates to B in the presence of C and we cannot map this here to a normal graph structure. What we have now is a topological distortion. The click expansion artificially inflates the clustering coefficient and the transitivity creating now suddenly dense cluster that are just artifacts of a projection, not the features in the underlying real world in the real science.
So therefore, we have a solution. The hyperraph. The hyperraph solution H. Here's the definition. What is a hyperraph? A hyperraph H consists of a set of nodes, simply nodes, and a set of hyper edges E, where each hyper edge is a subset here of V. So each node must appear in at least one hyper edge. Now normally hyper edges are not weighted. Duplicate hyper edges are not considered and each hyper edge contains at least two nodes, at least. But of course, a hyper edge in a hyperraph can connect to any number of nodes. Size of a hyper edge is defined as the number of the nodes it contains. Now there are now a lot of informal diadic projection for hyperraphs. Hypergraphs can additionally be projected into non-standard but interpretable forms of network representation that are maybe suited here for LLMs. So this is typically observed LLMs task with extracting knowledge triplet due to the efficiency of the token usage unless explicitly instructed to do so otherwise. So I just want to tell you there are three forms of informal diadic projection you should know about: the collapsed, the cyclic implicit and the chain implicit graph representation. If you're into hyperraphs, you know what I mean. If not, have a look at the original paper.
What is also beautiful? We can build nested hyperraphs. We can increase the complexity. Nested hyperraph generalizes the classic notion of a classic notion of a hyperraph by permitting now hyper edges to contain other hyper edges. You see exactly what's happening here. This structure introduce a hierarchy among the hyper edges where an outer hyper edge may contain one or more inner hyper edges and you have multiple levels of nesting. Beautiful. Now guess what those nested hyperraphs where do we find them? We find them in natural language processing. What a coincidence.
So you and I are interested how to build a hyperraph. You have 10,000 documents. How you do this? Well, this is simple. Yeah, because MIT showed us here. Listen, this is the LLM guided hyperraph construction with an incremental merging. And you see we have our hyperraph with our nodes, with our hyper edges, our document level subgraphs, our tuples, our cosine similarity between the embeddings. Oh yes, we still have them. A similarity graph, an equivalence class, a representative node here, not node mapping functions and you have here particular recipe. But you know what? I will give you the code for this so you don't, yeah, if you want to have a deep dive. But yeah, it is operational. And MIT applied this to a corpus of more than 1,000 manuscripts on bio-composited scaffolds and they tell us, yeah, we built a hyper, a global hyperraph of 160k nodes and 320k hyper edges revealing a scale-free topology and everything is beautiful.
But what I want to show you is the interplay that they found of the agent that works best. So multi-agent reasoning system. We have a user. How does a concept X relate to a concept Y? Now you have at first here a graph agent. And the graph agent locates these entities in a global hyperraph representation and then extracting an induced subgraph representing here, let's say, the shortest relational structure or if you have a BFS search here to extract the shortest path subgraph between those. So if you have the global graph, the path finding and the first step is clear. And then you have an engineer. And this is now intelligent. And the job of the engineer is now the agent engineer to answer how X relates to Y. I will use here the subgraph and then you have a path finding mechanism. You have a starting node, you have an end node here. And if you have a hyperraph here, you see there are multiple paths available for you to analyze. And in the end, you have a third agent. This is your hypothesizer. And it says, give me here the insight from the engineering agent. I will hypothesize now a test function.
So I want to show you this example because you got immediately a feeling what this is. So me, the user says, hey, can a cerium oxide mechanically relate to PCL? The graph agent extracts here the keyword. It found some matched nodes in the embeddings and then the graph says, hey, please consider the following relationship of the knowledge related to the questioner and make a response. And you get all the information. And now the engineer takes this. And now this is the answer of the engineer. As you can see, the engineer makes sense of this. And then we have a hypothesizer. The hypothesizer says, hey, to further explore the mechanistic relationship between cerium oxide and PCL, an experiment could be designed to synthesize and characterize a composite material consisting of cerium oxide nanopart nanop particles embedded here within the PCL nanofibers. And then the hypothesizer has now some ideas how to build this given here the information from the hyperraph. Now, the information from the hyperraph is so much more dense, so much more information than a single two-dimensional flat graph that we have now. So you see this is something where we have here the higher order relation that we need in science are suddenly available for us for the LLM.
Of course, it depends on the quality of the LLM. If you go into this paper generator 8, you have here, uh, chapter 4, ontological hyperraph corpus construction and the manuscript to the hyperraph algorithm. Normally, we wouldn't jump now into the algorithm, but you know what? They have such a beautiful. I already experienced here, uh, GitHub. Here you have it. Hyperraph reasoning. Here you have all the files. The graph reasoning, the setup here, you have the environment configuration. You have install here, Llama CPP Python, install the hyperraph reasoning package here. And of course, here, generate the hyperraph and the hyperraph embeddings here from the markdown files. Here, run the hyperraph agent. And everything is available for you. So I think just jump in, have some fun, take your documents and explore if a hyperraph is something that you need for your job. Do you have a complexity level that really needs hyperraphs? Are you working in science? Then maybe you will enjoy here this MIT resource. It's Apache 2.
Of course, what I would wish for, if it would not already Christmas been passed. Unfortunately, the current model is static. No, it does not account for a temporal evolution of the hyper edges. So, if you have an idea, oh, you want to write a PhD. Hey, we're looking all for a temporal hyperraph extension because this would really be necessary for a robust proof maintenance here. If we have here the temporal dimensions included. What are the insights? It's a beautiful study. It's a much more beautiful repo that you can play around with insights. And here we are again. Beyond drag. Stop retrieving text chunks based on a vector similarity or similarity. Start building your hyperraphs for your documents. If you work in science, if you have higher order connectivities, this really opens up here a complete new level of reasoning process. Retrieve topology.
The topology are really important because if you use the graph topology now as a syntax checker for the logic, and this means if an agent proposes a step that doesn't correspond to a valid hyper edge intersection, you can just block it. So this means that it's now really the case that the topology validates the reasoning chain of your LLM. So you do have some hallucination control with this particular hypers structure. And if you have an outlook, what I would wish for is the future, a graph native transformer architecture where the attention mechanism is masked now by the incidence metric of the domain's hyperraph. This is maybe the next, another next generation AI architecture, a blueprint that we need because, you know, hyperraph is defined by incidence matrix here. And if we can do an intelligent masking, we could go for hyperraph native transformer architectures which are just going to be amazing. 2026, we have so many ideas how to optimize further AI architectures.
I hope you enjoyed it. I hope you had a little bit of fun this Sunday. Hey, why not subscribe? Become a member here of my YouTube channel. Anyway, I hope to see you soon in my next video.