Transcription
Google has published a paper that directly caused three stocks to crash, and the reason for this is called Turboquant. The simplest explanation is as follows: Think, for example, of Zipfiles, which compress large files so that you can even send them via email. Turboquant does something like this for AI models, but not for the entire model, but for the KV Cache, which is essentially the short-term memory of ChatGPT, Cloud, or Gemini. You can imagine it like sticky notes that ChatGPT carries along during a conversation, and the stack of sticky notes naturally gets bigger the longer the chat becomes. And Turboquant can summarize this stack of sticky notes extremely skillfully. Turboquant only needs every sixth sticky note for this, is eight times faster, and this without any loss of information. Sounds impossible somehow, but it gets even crazier. By the way, I'm Max, I'm doing a PhD in Math, and I explain AI to you as simply as I can. Feel free to follow me if this kind of thing interests you. Every Large Language Model can use this technique immediately. So ChatGPT, Gemini, Cloud can use it directly without having to be retrained. And this is not only exciting for AI Labs, but especially also for Apple, because this is a way to compress AI models so that they might even fit on an iPhone. A corresponding deal between Google and Apple already exists. Briefly, two sentences for nerds. Turboquant randomly and orthogonally rotates the Key and Value vectors in the Attention Cache in such a way that the information of these vectors is distributed more evenly across the coordinates, and thus each vector entry can be represented with a few quantized bits. And for the calculation of the attention, the remainder error in the scalar products is additionally corrected so that it is not distorted. And feel free to follow me for more AI content.