Transcription
Just last week, Elon Musk said he was going to open source Grok. If you're not familiar, Grok (spelled G-R-O-K) is X.A.I.'s large language model.
Why did he do it? Well, spite is a powerful motivator, and he is putting a ton of pressure on OpenAI. Now, he just sued them a couple weeks ago. I just made a big video about it. The gist is, OpenAI is not open, and Elon wanted to prove a point.
Right now, it is a 314 billion parameter mixture of experts model. This is all new information that we did not know. 318 GB. It has eight experts with two active, the same kind of architecture that Mixtral uses. 86 billion active parameters. Apache 2.0 license, which is wonderful. And they actually supplied the code. So you get the code, you get the weights, everything.