GPT is Not The Future of AI: NEW AI Topology

Hello community. So great that you are back. I think GPT is not the future of AI. Let me explain.

Imagine we have our classical GPT, our decoder-only transformer architecture, and you know this is more or less just a next-token prediction. I'm currently working here on a quantum optical experiment, and I thought this is the same. We just predict the next token, the next token. But we are here really into a linear structure that is unbelievable. And our probability that we calculate here in an autoregressive way is we have a time minus one, and then we predict it for the time t. Here the state, here the probability, we have a probability distribution, and this is how the next token is selected. And this is just a linear sequence, which is okay, this is nice, but can we do better? Can we find other AI models? What about this one? What about we don't start here with some linear things? What about we start here with a complexity that is just amazing, and then we have some new machine, some newer engine, and then we just have here the perfectation, a new AI architecture that you will know because you will have seen this already on my channel. So, discovery, we have a look at the latest research papers. Let's start now.

You know, we have here with our typical GPT a decoder-only structure in the transformer architecture. No. And for the past few years here, our LLMs have all been decoder-only. We predict the next token. Beautiful. But you know, we have core limitations with GPT with a decoder-only structure. And let's call it a tunnel vision. And this is also why I wanted here to build this with Nanobanana Pro here to show you this is a linear tunnel, and we have only the next token. So we do have blind spots in the decoder-only AI capabilities, specifically how AI models read and see before they generate the next token, before they speak. So this is a fundamental flaw of a decoder-only architecture, and this is causal by design. So every token can only attend to the previous token. It can never see the full picture at once. It builds here the picture piece by piece, a sequence. If we talk about vision, so next-token prediction is excellent for generation, for writing, but it is really suboptimal for understanding, for reading, for perceiving complex data. But when a model needs to analyze a complex input like a long document, a decoder is forced to process it linearly. It cannot look ahead to understand the context of the next words. It cannot look to the end of the sentence and have here all, I don't know, 10,000 tokens here in a self-attention. Nothing at all. These are the limitations by GPT, our Generative Pre-trained Transformer, and you know GPT from the company OpenAI.

Now, there is a beautiful and easy solution because what we are missing is a bidirectional attention mechanism. And you know we have a T5 encoder where every token can see every other token here, past and future, simultaneously. So this allows now for a much richer and a holistic representation of the input before any generation, before the next-token prediction will even begin. So this is why we want the encoder block, the encoder part of the transformer back, because this encoder has some beautiful benefits. Have a look at this.

So here on the left-hand side here, it shows you the blind spot of our current GPT systems. No, let's say we have here an abstract space, a lower triangular matrix, and look at the oldest space that is here a black box, even for the GPT system. This is here the next tokens. Now imagine we have now from a next-token prediction to an understanding of the global state. So if we switch here to bidirectional encoders who analyze the complete content of 500 pages immediately, it is not a step-by-step prediction here. It has a fully connected dense hypercube. This is the difference between the black space that we have here with the classical GPT system, and here if we would have an additional encoder, we would have a complete understanding of the input before the system even starts to work. And imagine we have now a three-dimensional space. Now here, can you imagine we go from a three-dimensional space to a five-dimensional space to a 512-dimensional space? Can you imagine how immense, how huge this pitch-black void it is here for a GPT system? Can you imagine how much data is hidden here in the dark, unavailable to a GPT system? Compare this to a T5 topology. So you see exactly, we're going to go from a causal masking restriction to a full-rank bidirectional attention mechanism because we want here that this information here in the black, pitch-black void becomes available to the next generation of AI models, and GPT is not the way to go there. T5, a T5 structure reintroduces our dedicated encoder, and there are three specific benefits I think if you compare this to our classical GPT system.

Benefit A is you have superior long-context exploration. Benefit B is true multimodality, and benefit C is you have efficiency via the cross-attention mechanism and not only here the self-attention mechanism from the, if you want, decoder part of our transformer architecture. So let's do this. Let's bring it back.

Benefit A in detail. You know that the encoder compresses the world state, the document, or the image, whatever you have, into a stable representation, let's call it H, that the decoder now can query repeatedly. In contrast now, the decoder-only model must maintain the entire raw history in its active working memory key-value cache at every step, which becomes unstable and computationally expensive as the context grows.

Benefit B. The encoder can treat the image token with full visibility, allowing it to build a complete visual understanding before the text generation of the LLM even starts. So this means we have the complete visual information of, let's say, our image available before the reasoning process in the AI starts. Compare this to a GPT model where often you have to serialize the image into some sort of sentences, forcing a linear interpretation on a two-dimensional spatial object. This is really a massive limitation here of our mathematical space that we operate in.

And finally, yes, we have cross-attention, not only self-attention. Yes, combine them. Come on. T5 can attend here to the input, to the encoder with full visibility, and the output to the decoder with causal masking. So this separation that we have now addresses you exactly the reading process versus the generating or the writing process. So beautiful, you know, when the complexity increases, old trick in AI, you reduce the complexity in multiple tasks. So we have now a reading task and a writing task, and not everything is done at the same step. Compare this to the classical GPT model looking at a prompt and looking at what you just wrote at the same operation with our self-attention mechanism.

So we are missing out. Let's be a little bit more specific. What are we missing here if we only, or if we would rely on a GPT system in the future? Any ideas? What would you say? What is the most important part here? Let's have a look at the system here. Here you see here some benchmarks, and these are from December 20, 2025. And you see here the average win rate. If you let here all the current LLMs and VLMs fight against each other, this is here from LM Arena. The benchmark for textual causal reasoning logic performance on text. You see here the best model is here Gemini 3 Pro. It wins in 60% of all the cases. Then Gemini 2.5 Pro, Gemini 3 Flash. Then we have Grok, Grok, Claude Opus 4. Then, then we have GPT 5.2. Then we have Opus 4.5 Sonnet 4.5 Claude 4.5 and then we have GPT 5.2 too high here, really here in one of the last places. And you would say, yeah, okay, but we don't know against what system those systems were competing against now. And you're absolutely right. So let's look at the ELO rating. No, let's look at the performance at the absolute performance with a certain confidence interval. And you have here the same, the rating here, higher is better. So we start again with Gemini 3 Pro, then we have Grok 4.1 Sync, then we have Gemini 3 Flash, and then we have Claude Opus, Swap Drive Opus here from the non-syncing version, Grok 4.1. And you see those are the best models here, as rated here by the AI community and several thousands and tens of thousands of individual votes by the AI community on the ELO rating. And interestingly, you see the latest of the, the, yeah, if you want, the most prominent GPT systems like GPT 5.2, 2 is rather here, not really here in the top 5%. And then GPT 5.2 High, voted by the community, is here. Now, you will see here that the confidence interval, especially by GPT 5.2 High, is quite huge. Why? Because I think there was about 6,000 or 7,000 votes only. So we're already statistically a little bit significant already. But you know, it could move up or down a little notch here, though it's not really fixed yet. We have to go to 20,000 votes here by individual tester from the AI community who says, okay, those are the best models. But currently, at December 20th, 2025, those are the best models. And you see here clearly here a performance and GPT 5.2, 2, the very latest GPT models, you see exactly where they are. What does it mean? Is this a systemic performance or is it just that something is happening? Let's say that the company OpenAI is preparing here for their 1 trillion IPO and are not really here activating here all their resources here for improvement here of the technical side of the systems. Now, I have to tell you, I also have two videos here on GPT 5.2 and also on GPT 5.2 too high. And honestly, if you look at both videos, you understand here. Well, let's say my impression is absolutely in line with the impression of the AI research community that those models are rather not in the top spot of the high-performance AI models currently.

Great. Let's come back to science. But relying only on this GPT, or let's call it decoder-only architectures, we are missing here the deep perception of the intelligence, if you want. So we have built models that are excellent at talking, but are structurally handicapped at listening or reading. So they're like a person who starts speaking the moment you begin a sentence, predicting your next word rather than waiting to hear your full thought. Compare this to a T5 model. This is a return to listen. First, you encode the information, and then when you have all the information, then you start to speak. Then you have the decoder.

Now, T5 is familiar. If you're a subscriber of my channel, you know here two, three years ago, in detail, I went about accelerating the fine-tuning of T5, a classical T5, a professional fine-tuning of a Flan T5. We also had a look here at exactly here at T5 here on a free Colab. We went this in a combination here of a Flan T5 XXL with a Vision Transformer. And in general, you know, if we have our transformer architecture and we only use here the decoder part, the GPT part, we are missing out. If you only use here, for example, the BERT structure, no, the encoder part, we would be missing out. And the beauty of T5 is it is a complete transformer system as predicted and patented by Google here years ago. A complete transformer is back. Yeah, if you want to see this, about three years ago, I already compared here a Flan T5 to a GPT-3 system about coding. Is it better than a GPT? And today, more or less, we're asking you the same question. If you compare to a GPT 5.2 High model. Yeah, of course, we already had three years ago. I showed you here the transformer T5 in JAX and in Flax, how we could code this, not only here in the PyTorch model. All this information is already there. But this is now the new paper of today, December 16, 2025. Google DeepMind comes out now with a brand new idea at T5. Gemma 2 tells us seeing, reading, and understanding longer. And you might say, okay, it's a nice paper. It's not so important. Why should I care about this? No, it's close to Christmas. Come on, I can forget about it. Don't. This is an amazing paper. It shows you the future of AI. Why? Because it combines here the T5 architecture based on a, the Gemma model, and Gemma 3. And you might say, but hey, we already have Gemma 3, so what is Gemma 2 doing here? Well, just look at the difference here. The performance data, pre-training, post-training, never mind, wherever we look, in blue, we have Gemma 3, the classical decoder-only transformer, and in red, we have this new model of today that we're going to talk about today, T5 Gemma 2. Look at the performance. Look at multimodal. The classical Gemma 3 has no multimodal ability. Look at the ability here of a 270 million model. Then we have a 1 billion model. Then we have a 4 billion model. And you already see something. The bigger we get, the better the classical performance of a GPT system becomes. So this means if we scale up the size of a GPT model, GPT becomes more powerful. This was the idea. We have to scale up to bigger and bigger GPT models. We have to have more and more data centers globally because the model will scale up and scale up. But what Google in this paper is interested in, what about we go the other way around? We make it smaller. You can run a 1 billion AI model on your normal GPU, on your gaming PC. Imagine only a 270 million free-trainable parameter AI. This is, you can run it maybe on your iPhone. So you see exactly where Google is going. Not scaling up, but making the model smaller. But look at the intelligence. So much more intelligent. Either you go for X multilingual, you go for reasoning, you go for long context understanding, or you go for multimodal understanding. So Google is showing us here in this innocent paper, hey, wait, there's another direction. Let's make the model smaller for edge devices, for handsets, for whatever. And let's make them more intelligent because Google tells us here, GPT is not the way to go into the future.

So this is now my idea now. So we have two options for our future. We have this hyperscale data center centralized, or you can have a distributed edge intelligence. You can have a new AI architecture that is only the size of your model. Is only a 1 billion free-trainable parameter model. But this fits, let's say, on your gaming PC, on your GPU that you already have, or maybe you have to, I don't know, 16 GB, 24 GB of VRAM. But you know, small models, but a complete different architecture and a much better performance. Now, this is now the fight that is currently going on. Should we make some, let's make it a little bit more dynamic now. Should we have here some hyperscale data center centralized monoliths with an extreme energy consumption, with cooling, with water, or should we have an intelligent network? You decide. But Google tells us, you know what, maybe we are using the wrong AI tools for the job to be done. Let's have a look at this now.

You know, if you look at Gemma 3, the technical report, this is here from end of March 2025. I hope you are familiar with this. This is here a typical, if you want, decoder-only port. And you see download here on Hugging Face. Yeah, by the way, it's an open model. So you just have to agree here to a particular Gemma license. Okay, you're not going to compete with Google as a startup. I think, okay, let's say we agree to this. The downloads here from the community last month was close to 1.6 million downloads. So this is a 1 billion Gemma 3 model that the community is really responsive to. I knew this new T5 Gemma 2 that we've had, 270 million, is now available for two, three days, but it's not really, as I see here with the downloads, that the community really is understanding what is this beautiful new, the beauty in this new model.

Think of Gemma 3 as the parent, the older partner. It's a classic decoder-only model like a GPT-4, 5, or a Llama model. It was trained from scratch, massive computer resources. The primary cognitive mode is improvised speaking. I predict the next token, generate some, I don't know, poem. Think of T5 Gemma 2 as now a specialist offspring of Gemma 3. Gemma 3. The reason I took a pre-trained brain of Gemma 3, performed a, um, cloning of some particular layers, architectural modification, and retrained now the whole system to function as an encoder-decoder structure, as a T5 structure. And its primary cognitive mode is now listen, then digest, and then you speak. I make it real simple. So designed to read massive amount of context, images, or text using a dedicated encoder before it produces a single word of an output of a reply. Completely different here to the decoder-only Gemma 3, the classical GPT structure.

So Google now evolved here Gemma 3 into a T5 Gemma 2 because they both have a different AI topology. Gemma 3 is just a stack of decoder blocks. Input tokens, output tokens are processed by the same stack. The attention mask is strictly triangular. Token 50 can never see token 51. I told you it has a big black void. With T5, Gemma 2, you have an encoder and a decoder because it's a T5 structure. So in the encoder, you have a stack of blocks initialized on Gemma 3, but the attention mask is now removed. So the token one can see, let's say, up to token 100,000, whatever you have in your document. So it produces now a high-level memory bank. The decoder has now a second stack of blocks to generate text. It uses now a complete new form of attention. I will explain this in a moment. They call it a merged attention. It is more or less a crossover. No, not a crossover. You have the self-attention and a cross-attention. And if you combine these two attention modes into a new attention form, they call it a merged attention for a T5 Gemma 2. Beautiful. So it looks more or less both at its own past outputs X and the encoder memory bank H simultaneously. So isn't this beautiful? It can see the future and the past and it can then start to argue. This is especially beautiful for the vision integration for multimodality because if you have noticed the Gemma 3, the smaller models, they are text-only because at 270 million or just in a 1 billion variant of Gemma 3, the visual complexity, these models are too small. But guess what? With a T5 Gemma 2, we can now build a cyclic vision encoder, a vision transformer into even those small 270 million models. So therefore, we do have a T5 Gemma 2, 270 million to 270 million because both the encoder and the decoder have 270 million. And by routing now the visual tokens into the bidirectional encoder, they upgrade the text-only Gemma 3 weights into a multimodal system. So suddenly we add, if you want, some eyes to a Gemma 3, and now we have a T5 Gemma 2 with a 270 million model that can now see the images. And you might say, how is this possible from the, from the embedding side? No, this is the beauty here. They made a simplification. Let me show you.

So here I try to show you the difference. No, this is here the parent. This is our Gemma 3. This is a decoder-only. This is a classical GPT system. No, this is like you have also a ChatGPT. This is the architecture, causal attention. Beautiful. And now the evolution, if you want, here is going back here to the T5 architecture. Remember when GPT 3.5, I think, was came out, two, three years ago, it changed everything. It stopped the evolution of T5 because Google had suddenly to focus here to compete with OpenAI and was not able to further develop its own idea of a T5. Now, in my simple mind, with Google Gemini 3 Pro, they say, okay, we are back here as a player here. Now we can continue our original idea of a T5 of an encoder and a decoder structure to further improve the performance of the next generation of AI models. And you see also our blue decoder-only. Yes, the same. But before they have now their encoder. So a T, classical T5 structure with image patches coming in, and you have a merged attention, you have a, a self-attention and a cross-attention. Now into a higher complexity. If you look at the mask, this is especially interesting. Now, in a decoder-only, like in Gemma 3, you have only the causal attention mask. No, you have an input sequence that comes in, your particular mask. Beautifully predicts this. Great. With T5 Gemma 2, you have a hybrid encoder-decoder structure. Now you have here in the encoder a bidirectional attention mask, and in the decoder, your causal attention mask. And this becomes now interesting how you build here your attention bridge in a combined model. Great. Here we are, merged attention. Let's have a look at, if you want, a simplified mathematical unification. I got here a comment. Hey, in your last video, you explained something in mathematics that I even understood. So I will try to do the same. It is the simplest you can imagine. Yeah.

So here we are in a standard encoder-decoder transformer. And original T5 or BERT, a decoder block contains two distinct attention sub-layers that run sequentially. You have a masked self-attention and a cross-attention. With the masked self-attention, yes, you have your query, your key, and your value for the decoder, and they more or less query its own past. For the cross-attention, you will have the same here for the end-decoder and encoder. So you are querying now the encoder outputs, of course. So therefore, what is it now? The design introduces separate learnable weights for both layers, increasing here the memory bandwidth usage and the parameter count. But if you want, the innovation of T5 Gemma 2 is not the following. They fuse these two operations into a single joint attention module. So this forces now our AI model to decide in a single softmax step whether to attend to the internal memory or the external perception. This is more or less the encoder context. If you look at this from a mathematical point of view, a tensor operation. So we have here particular tensor input, M is the decoder sequence length, and N is here the encoder sequence length. Oh, and H is missing. Sorry. But instead of the separate projection, the T5 Gemma 2 constructs now the keys and the values by, guess what, simply concatenating the encoder and the decoder states along the temporal dimension, as you see here. It just concatenates them. My goodness, such a simple operation. And the queries are derived solely from the decoder input X, as the encoder does not query the decoder. So you see, simple. What is really beautiful, you have to have a deep dive into the joint masking operation on this new architecture. If you want, a magic quotation mark lies here in the masking matrix that we have to define because think about it, it must enforce two different jobs now, two different physical laws simultaneously within the same matrix multiplication. No, at first, we have to have the causal, um, layer, the autoregressive law. First M columns, the decoder part, the mask is a lower triangular. The token t cannot see the token t+1, the classical case. But then we have, hopefully beautifully, the bidirectional or the non-causal for the subsequent N columns, the encoder part. The mask is suddenly fully visible. So the token t can see all the N encoder tokens. It can see here the complete image. And the attention scores are computed here with a formula, you know, just note here, here we have our masks. So you see a simple idea. If you know the solution as provided by Google to understand what is going on here. So it is standard T5 again to compare this. The model always performs self-attention and then always performs a cross-attention. Now notice here with the T5 Gemma 2, the softmax normalizes across both X and H. It creates here a learned competition. At any given step, the model can choose to ignore either the encoder entirely and focus poorly on the generation dynamics, or vice versa. Of course, you have to train the system. You have to build it. You have to train it. And Google gives us some indication how they build it.

Now, for the training, it seems to be a simple case. If we talk about edge devices, notes, what is also the, the benefit here instead to a classical T5 system? This T5 Gemma 2 removes the entire set of the projection matrices required for the cross-attention block. So we have to reduce the mathematical complexity and the computation time significantly. Let's look at this.

Okay, this, this is here. I tried, I tried. So we have here our image. Beautiful. If we have now a vision transformer here, you know, it just reads here a single segmentation, beautiful, and everything is now here a visual token that goes now into our encoder, not in here the classical decoder, in our encoder system where, if you want, this is a system to intelligence. If you go like the connotation, no, a vision transformer system. Beautiful. And then you have this beautiful cross-attention bridge over to our classical, if you want, GPT system. This is now here where the reasoning is happening. And then the last zone, the generation is here. Whatever tokens you generate, normally they are discrete in this typical case. But we also have to talk about diffusion model continuous denoising structure where you get then a beautiful dragon out here of a statistical noise. So this is the complexity we're going to talk about. And you see here in the background here, in this light gray, this was the old, the current GPT system, the current decoder-only structure. So you see compared to this, we are building now this to increase the performance of the AI system of the future.

So now, how we built this, how Google built it, I mean, we, since the T5 Gemma 2 is adapted from a decoder-only parent Gemma 3, it is already good at the generation. So how do you do now the, if you want, tuning and the training? Now, of course, Google developed already a UL2 mixture, carefully tuned to preserve the generative ability while injecting here the bidirectional understanding capabilities that Gemma 3, for example, lacked. And we do this here with a particular denoising. We have here, I will explain this in a moment, an R-denoiser, and we have a prefix LM. And if you do this in an intelligent way, you come up with a beautiful T5 Gemma 2. So this curriculum shows us that we can build a bridge that allows a mutation from the decoder-only to the encoder-decoder architecture to really succeed and have the performance we're looking for. Yeah, I told you, you will too. This is the paper you have to read. This is here from end of February 2023. So quite an old paper, but now we use it again. Of course, guess what? It's from Google Brain, "Unifying Language Learning Paradigms." Here you have here this mixture of denoisers that I talked about. So we will have in, in theory, three. We will use all of three of them. So we have an extreme denoiser, and we call it an XD denoiser. Then we have an R-denoiser, a short span and a low corruption denoiser. And then we have an SD denoiser, sequential denoising, a prefix model languaging that I showed you. And the mixture is important, how we weight here the intensity of this denoiser. And then we have here the learning paradigms beautifully. You will have here a deep understanding in this paper on UL2. And here you see here, especially what is in, in detail, in an example. Now, what is an R-denoising doing? What is an SD-noising doing? What is the extreme denoising now doing here? What is it masking the tokens here for the prediction tasks? Okay, highly recommend this paper. In very short, let's say you have read the paper now, you understand what I call here the R-denoise at the regular span corruption or the editor mode. A short span of text to mask out here. There are particular settings. Forget about it. The idea is the goal is to teach the model local precision and syntax error or denoiser. Then we have the sequential denoiser. The extreme corruption. This is, if you want, the write mode. What is the task? Massive chunks of text are masked out here. Sometimes the entire second half of a document. No, not 500 pages, but if you have one page, you really have here masked out the second half of the page. Some particular settings, never mind, 50% of the text is hidden. And the goal is that it forces you, the AI model, to generate long, coherent paragraphs. Beautiful. And the most important, if, if you want, is the prefix LM. This is the completion mode. So give me a particular prefix of section A. Predict the rest of the document, is a section B, without any corruption tokens in the target. So this turns here effectively the encoder-decoder more or less into a standard language model. The encoder reads the prompt, and the decoder continues it. But now we have a much better, complete understanding of all the input data.

So now we're realizing that reading this means encoding, and writing this means decoding, are, if you want, also compared to the human neurological different tasks requiring different topologies. The old GPT approach was, well, we have currently in our GPT 5.2 systems, forced the same neural weights to both. So this is efficient for the training, but suboptimal for specialized tasks. So therefore, Google's idea was, and now they continue with the T5 approach, specialization, expert system. The encoder acts like a system two processor, deeply analyzing the prompt, looking for contradictions, and compressing here the logic. And then we have the second part, the decoder. The decoder acts like a system one processor, taking that compressed logic and fluently articulating it into text that humans prefer. So you see, this T5 approach is much more interesting.

Let's talk about the vision problem. The multimodal integration. I told you the T5 Gemma 2 has a multimodality even on the 270M model. Connecting an image to a text model is quite awkward. Here in the classical GPT architecture, you usually just shove the image tokens to the left of the text and hope the causal attention figures it somehow out. This is why we called it a black box. Now, in T5 Gemma 2, the separation is much more precise. The encoder now handles the noisy raw perception, the input patch, the image patches, and the input text, and it acts as a perception cortex. If you think about the human analog, the decoder stays pure as a language center, only receiving the refined high-level signals via the cross-attention bridge that I showed you in my image. So you see again, higher complexity, we reduce it, we specialize, we have a clear task for the encoder and a clear task here for the decoder. Beautiful.

So this is now, if you want, kind of a summary. This is why we want a T5 AI architecture. We want it back. We wanted Google to continue because, yeah, the dynamic and the interrupt by OpenAI was great. But now we see that we hit a wall with the GPT architecture. We need something more intelligent, especially for open-source models, especially for smaller models we can run locally at home. Now, I see we, I mean, in particular, I am unhappy with GPT systems because we have pushed it to do things it was not designed to do for a deep analysis of massive documents. No RAG, typically. If you want to chat or write a poem, continue to use GPT, a decoder-only transformer architecture. This is beautiful. No, this is creative, gorgeous. But if you want to read a 500-page manual and find the contradiction in the clause 4.2 to paragraph 8.9, then you use another AI. Then you should use a T5 architecture, an encoder-decoder, because the encoder sees the whole manual document at once, and then the decoder extracts here just the answer. And this is so much better, especially for RAG systems. So the idea is not to have GPT with external RAG, but bring RAG into, if you want, the architecture itself in a T5 architecture. So much nicer, so much more information because you remember with RAG, what we do here with the GPT, you know, we go to a database, we search for a particular keywords or semantic structure or cosine similarity, then we have a re-ranking, and then we do it again, and then we just bring back pieces of the information that might be helpful to answer the human query. So we have, if you want, a disintegrated sequence of some facts that might be helpful. But here with a T5, the encoder sees all the 500 pages. It ingests all the 500 pages, and now the decoder can select exactly what information, what knowledge is essential, in what complexity level to answer my human query.

So a unified AI architecture landscape might look next year already like this one you see here in this light gray. This is our classical GPT decoder-only. And maybe we move from this little thing here to this monster here because then we have, if you have an image, or if you have some huge text document. Let's stay with the image. We have visual tokens. We have a vision transformer that separates us here through different lenses. This should be lenses. Okay. The visual tokens. The visual tokens directly go into the encoder system. We have a bidirectional network. We have our cross-attention bridge over to our decoder structure, classical one. And then we have the output here. Discrete tokens, probability distribution, everything that you know. But based on a complete and homogeneous understanding here of the complete content of the image, and not just a linearization where we go line by line and element by element and we sync step by step. No, we have the complete knowledge, the complete data in our encoder, encoded.

Yeah, I was talking here. We should use, we should have a short reflection on diffusion because I got some questions from my viewer. Hey, how does the diffusion fit into this into the future of AI? So very short here, my my crash course here, scientific distinction between a T5 Gemma 2, a Vision Transformer, and a diffusion model. Now, a Vision Transformer, you know, as I told you here, in T5 Gemma 2, we have a Vision Transformer. This is just the eyes who are now able to see the environment, and we have visual tokens that go into the encoder. So actually, this means a T5 Gemma 2 has or contains a Vision Transformer inside of it. Of course, the eyes. Now, in general, Vision Transformers do not generate text. Not to translate pixels into vectors. This is a mathematical concept. Full stop. A Vision Transformer chops an image into patches and treats each patch, if you want, as a word, if you compare to an LLM, and runs those visual words through an encoder like the T5 encoder. The relation to T5 Gemma 2 in the paper, you ought to use here a Cyclip Vision Transformer, but use any other Vision Transformer that you like, which is kind of a specific type of Vision Transformer. Looks at the image, converts it into 256 visual tokens. If you have a high complex 4K, 8K image, you know exactly what to do, how to increase the visual token amount. So our T5 Gemma 2, our LLM, takes now these 256 visual tokens and reads them to answer your question. Of course, they do this here in a vector space. So if you want, a Vision Transformer is purely an encoder. It is a sensor. It is the eyes of your AI system. They cannot speak. A Vision Transformer can't speak. It can only see and report it here to a language model.

Now, what is now a diffusion model? As I told you, the LLM of the T5 Gemma 2 is discrete language, is made up of discrete blocks. Cat, dogs. There's no word halfway between cat and dog. No, the model predicts the probability of a specific discrete token. The goal is to be logic, have here the reasoning capabilities, and a perfect grammar. Diffusion is completely different. Diffusion is now continuous. Images are made up of continuous gradients of color. Diffusion models learn to manipulate you the pixel values on a continuous spectrum. It doesn't predict the next pixel. It starts with a static noise, like the snow you see on your TV when you have no, um, input, and mathematically simply subtracts the noise step by step until the limit emerges. Goal is here, focus on texture, geometry, aesthetics. And so I have dozens of videos on diffusion models, how they started three, four years ago, how they evolved over the next two years. Okay, what I want to give you here is also, I got a question about Nanobano Pro. How are they really? We have not yet a clear technical paper by Google explaining everything about Nanobano Pro in detail. So we are here a little bit here trying to understand what's going on. In my understanding, they are composed of two massive neural networks wired together via the cross-attention bridges. Part A is the brain. Our Gemini 3 model here does not output an image. It outputs a semantic blueprint, a sequence of high-dimensional vectors. Vectors are or matrices are what we need to multiply. This is AI. And Part B is now a diff, a diffusion transformer. Now, a diffusion transformer has also attention heads, just like a large language model. But instead of attending here to previous words, it attends now, of course, the Gemini 3 semantic blueprint here, the Part A, the brain. But now this is highly interwoven. It is not just a single sequential procedure. In my understanding, again, we have no official documentation that I can point my finger and say, this is exactly how it is done. This is only my understanding, and my understanding can be wrong here. These three sentences can be wrong. If I say, draw a cat, I imagine that Gemini 3 tells, okay, the internal monologue of the AI, the user wants that I draw a cat. So I will generate a particular sort of a cat. I will use a soft lightning here for the image. The cat will be sitting on a rock here. So I define the environment of the cat. I will define a style, a photorealistic style, whatsoever. And then it just maps this to particular embeddings, to a representation in a vector space. Then the connection here, the embeddings are now piped into the cross-attention layers of our diffusion transformer. And the diffusion head, if you want, the hand that paints now here this image, what does the diffusion head do? It takes the noise. It looks at this particular embedding structure. It sculpts the pixel until it has here the image out of the noise generated beautifully. Now, you might know here the legacy form. This was a U-Net with convolutional neural networks. U-Net, it was called because it has a beautiful U-shape. I have four videos on U-Net in three, three, four years ago. Now, you know the problem with U-Nets is they are real hard to scale. They have a local inductive bias. And I think one, two years ago, the whole community moved from a U-Net architecture to a diffusion transformer architecture. So now everything is a transformer, more or less. No. So we moved from convolution to patches. And I just want to have here the complexity of this diffusion transformer here in this structure again, generated by Google, Nanobano Pro. Now, the beauty is that our transformer has some beautiful scaling laws that are native to this. So much easier to scale. So global self-attention, priification, you know, noisy images, you understand here. This is here. If you hear U-Net, now you know this was the legacy model. Now the new one is a diffusion transformer. If you want it broken down to a single sentence, T5 Gemma 2 uses here an encoder to turn the pixels into a logic. And Nanobano Pro uses here an LLM to turn the logic into pixels. We immediately see how Google is building it, a complete AI environment. Why can a diffusion model not simply replace a T5 Gemma 2? It has no symbolic reasoning. Yeah, they don't have attention heads capable of linking cause A to an effect B across 100 pages of text. Here, a very simple explanation. The flow direction here for a T5 Gemma 2 is from pixel to text. None of our Pro is from text to pixels. Clay provision transformer or diffusion heads for bank the pixels. Yes, beautiful.

Let's come to an end overview. I want to give you here a real overview of all the components and answer all your questions received here in the last weeks. So what we have, we have different architectures in AI. We have a Vision Transformer, we have an encoder structure, we have a decoder structure, and we have diffusion transformers. So how are they connected? Think about here. Yeah, you can choose for this architect, just particular components like CIG as a Vision Transformer or whatever you lose, or here Flux, or whatever you have for your preferred diffusion engine. What is it? A Vision Transformer is just the eyes of a system. No, an encoder, especially a T5 encoder, is just, if you want, the reading brain, the understanding. Takes text and visual vectors, analyzes them, and builds here a mental model. The decoder here, classically, this is here our GPT system, decoder-only transformer architecture, is here the mouth. It produces here an output token by token, word by word. So it takes here the mental model generated here by the encoder. It articulates the answer in a human language like a system one, looking, and then, if you want, if you then have here the intelligence and it's spoken out, and you know exactly what you do, then you have also, if you want, a hand that paints now here on the instruction here, something. And here you need a diffusion model, not presented here in this paper by Google, but I just wanted here to respond to your questions here on my channel. What are the architectures and how can I combine them and what are they? So more or less, this is it. And this is also the role here in the T5 Gemma 2 architecture. Beautiful.

So I showed you, we go to smaller, local, open-source models. Thank you, Google, for providing this. You find this model open-source here on Hugging Face. You can download, you can play with it. You can test here really out that you see, uh, Gemma 3 has here in a 270 million, no multimodal capability at all. But if you go to a T5 Gemma 2, wow, what an improvement. So smaller, more intelligent, more capable. Okay, I think this is it. The end of Part One. And yeah, and I thought if I show you here where Google is today at the end of December 2025, let's make an outlook. What is the new technology you can expect in 2026? So next, we will open up and complete new paths understanding today's video. What are the paths into the future? And I will build here on one of my last videos here, that video, "AI Must Break Physics to Function." Based on those insights, we will generate now a complete new understanding. We will integrate now the new T5 AI architecture into the newer symbolic AI. You will see that there's a beautiful synergy. We will have a look how we can integrate a physics engine here for the causal reasoning process. And finally, we will talk about how the research on world models can be coherent into the development of the next T5 systems. Okay.

So you see, this was it for today. Again, unified architecture landscape. And now we, next video, we ask how we build here world models here in this architecture where we have a beautiful image here, a Vision Transformer, uh, visual tokens that go to an encoder, have the bidirectional network, have the complete context, then have here our cross-attention bridge over here to our general decoder architecture, and have here finally the output, exactly what we are looking for. And you know, you can then also add here, if you want here, a diffusion transformer, not have just words, but generate from those words here a beautiful image. And this is a diffusion process. And we also have here a short explanation in this video how this diffusion process is happening. But as I told you, I have multiple videos on diffusion explaining what it is. And all of this now here is, at least from Google's side, the new idea for the AI. Compare this to this little gray one here, the decoder-only. The GPT that we have. This is also our classical ChatGPT system. And the limitations we currently face more and more with this GPT system. And now you understand why at the beginning of this video I had this idea, this hypothesis. GPT is not the future of AI. Because look what we can achieve if we build a more intelligent AI, if you want, architecture and infrastructure for our next AI models. And of course, we will have the task to integrate here physics engine, an effective world model into this. But more about this in my next video.

Okay, I hope you had a little bit of fun with this video. Maybe it was informative. Maybe you got some brand new ideas. Anyway, would be great if you subscribe. Maybe become a member of my channel. But anyway, I hope to see you in my next video.