Transcription
Hello. Today we will test 18 of the latest neural networks in code writing. Today is August 2025, and you can see the list of neural networks. These are the latest neural networks. Opus 4.1, GPT5, GMI 2.5 Pro, Open Source Player 4, Qn 3, a whole line of neural networks, GLM 4.5, Kika 2, and R1. The latest version is 0528. Let's see how well these neural networks write code. We will test using my prompt, which you saw in previous videos. This is the task of writing your own application, an analog of ChatGPT for desktop. This is what the prompt looks like. The only thing here is that the name of the neural network is written in the file name for logs and in the title. I have already done all the work off-camera. And for each neural network, the code has already been generated by this prompt. And now we will launch it together with you and fill in the table. The neural networks were taken from web development rankings. For example, this is the WebDev Arena on the LM Arena website. This is what the rating looks like in their opinion. The second benchmark is SWE Benchmark Software Engineering. And here they are compared in the task of writing code. What is written in this prompt? It is necessary to create a desktop application with two screens. The first screen is the login screen. You need to enter your OpenAI key there, and the application should verify it on the OpenAI website. And the second is the Application Screen, where you can chat with any OpenAI model. A standard dialogue is programmed. All the code needs to be written in one file, give it a title, and strictly use these versions of the libraries. They are also specified in requirements, so that the neural network writes code for a specific version of the libraries, and their calls depend on this. And there will be several stages on how we will test. First, whether the application launched at all, meaning the libraries were imported correctly, a window appeared, and, in general, you can enter the key further. Next, key verification. In case of successful verification, transition to the chat screen. Then we test the dialogue. How well the assistant responds or doesn't respond. And the prompt also specifies support for streaming, like in ChatGPT, or in other chats. We write a question and receive an answer token by token. And for each neural network, we will fill in this table and draw a conclusion about which neural networks are better to use for writing code in this case in Python. And in this case, for one stage, with one prompt, a whole application needs to be written. This is exactly what we will compare. Well, let's start with the neural network. Opus 4. Opus 4.1 is Opus 4.1. So, it launched. And let's immediately launch Opus 4. Opus 4 is the previous version. As we can see, the applications have launched. The title is correctly specified. There is an input field. In both. There are the necessary verification and save buttons. Well, let's enter the key. This is the OpenAI key. You don't have to retype this key. I will delete it immediately after recording. It was simply created for recording this video. So, let's check. So, the key is valid. The key is valid. Both neural networks correctly verified the key and said it was valid. Let's save it now. It was saved successfully. It was saved successfully. So. And this is what the chat dialogue looks like. Well, let's ask each of them some question. So, Opus 4 crashed. Opus 4.1 also crashed. Okay, as we can see, the errors are very similar, and both neural networks have already failed with this. There's some peculiarity with the library versions. Well, yes, by the way, the code is highlighted. And, in general, these neural networks did not cope with the task of writing. Now you need to write a new prompt for the neural network to fix the exception. And let's fill in the table. So, both neural networks started correctly. The key was verified correctly, the chat screen appeared and it was correct, but the dialogue no longer worked. And streaming, accordingly, we could not check either. Next, let's move on to the next neural network. This is Sunnet 4. This is a cheaper neural network from Opus 4. And let's see how it coped. So, the application launched. By the way, the design is slightly different. You can see that there is a large title, there are indents on the sides. So, the design is slightly different. We paste the key. So, the key is valid. Interestingly, it didn't appear in a separate window, but in a separate area below the buttons. It was saved correctly. We also got the chat screen. So, a few indents. And this is the same error. So, it seems that the neural networks have similar datasets on which they were trained, only the size differs, and the errors are very similar. Actually, I use OBUS 4 constantly. Now I'm switching to OBUS 4.1, but I can't say yet, I just don't have enough experience to know how good it is. I will share my opinion with you later. So, Sunnet 4. Everything is the same. And the dialogue did not happen. So, it was supposed to be done with one prompt. Next, neural networks from OpenAI. Recently, a new neural network GPT5 was released. It has several versions. Well, let's compare them. This is the GPT5 version in the ChatGPT interface. This is what OpenAI provides to free users. I don't have a paid subscription at the moment. And let's see how well this neural network will work in chat mode for a free user. GPT5 is a neural network called via API, called through the OpenAI Router website. GPT5 mini is a smaller version of this neural network. And recently, OpenAI also released two interesting open-source neural networks. This is the first open-source neural network after GPT2. And they have been released. I tested them. They work quite well for their size. The size is 20 billion parameters and 120 billion parameters. And they can be run locally. Well, let's launch them. So, this is GPT5. This is GPT5 via API. So, let's close this one. So, let's close this one too. So, this is what the interface looks like. Well, honestly, I like this less because the alignment is somehow strange. Here, it's also unclear how it is with Anthropic. I like the design more. So, let's paste the key. So, both neural networks coped. Let's save it. So, GPT5 coped with this stage. And let's ask a question. So, it wasn't sent via Enter. It was sent here. Oh, well, as you saw, GPT5 worked. And streaming was supported, but it crashed overall. So, and here, here we encountered an error, very similar to what happened with the neural networks from Anthropic. In this place, I think it's related to the library versions. So, the neural networks are trained on different versions and by default use some other version, not the one specified in the prompt and in requirements. So, what will we put? Here everything launched. Chat dialogue crashed here. Crashed. So, and the version from ChatGPT. Here we had streaming, but it crashed. Let's put a plus-minus. So, now. No. Well, just like this, so that we remember that streaming seemed to be supported. We saw that the line was printed, but then the application crashed. So, okay, let's move on. GPT 5 mini. Let's launch it. It also launched. Well, by the way, GPT5 Mini looks prettier. Let's paste the key, let's check. So, well, GPT5 Mini seems to have crashed with some error and didn't cope. No message appeared. Valid, invalid key. It was saved, nevertheless. Well, here we can put a minus for it. So, let's ask. Well, and GPT5 mini also crashed. So, let's mark it here. So, it didn't work, but it worked with flaws. That is, there was no message, the key was simply verified, some message was given in the log, but nothing was shown to the user. Next, let's test. OpenAI open neural networks. So, let's start with the small neural network. It launched. Let's start with the large neural network. It also launched. So, let's check. So, key verification worked for both open neural networks. Oh, the open neural network wrote something. Let's. Well, in general, the application is practically working. That is, the chat is working. The only thing is that there is streaming, but there is no chat history. That is, I expected that like in ChatGPT, messages would appear and the entire history would be visible. In general, at the moment, this is the best solution. There are no errors in the logs. So, let's look at the small neural network. The small neural network crashed. Also in a similar place. So, the verification passed. So, chat dialogue is conditional, streaming is present. At the moment, OpenAI neural networks work best. That is, it completely wrote a working project. The only thing is that the dialogue is not saved. Let's continue testing. Mini 2.5 Pro from Google and Grok 4. So, Gini and Grok 4. So, Grok 4 gave this error and immediately closed. So, the application is not working. Let's test the neural network from Google. So, the key is valid. Well, by the way, the design is interesting. Centered alignment looks worse than OpenAI. Let's keep watching. So, it wasn't sent via Enter. Well, okay. Oh, in general, you saw that it worked with streaming. So, neither Enter nor Ctrl Enter work, only the Send button. Well, in general, at the moment, the neural network from Google looks best. That is, all the functionality works. There are some formatting issues here. This was not in the prompt, but it can be fixed. The main thing is that the core functionality works here. Let's ask something else. Ah, well, it's interesting here that the penultimate message is updated. Well, yes, the last message from the assistant is updated. In general, the chat history is saved, but with minor glitches. So, let's fill it in. Grok crashed without even launching. Gmi min, it all worked. Chat dialogue worked, but with flaws. And streaming is present. And the sponsor of today's video is the company Teta. This is a website where the main neural networks are collected. You can see them on the screen. And it provides an interface similar to ChatGPT. You can ask a question. In particular, you see here the prompt for Gigachat. Gigachat 2 Max. And here is the answer I copied from here. That is, the Gigachat code was obtained from the Charunet website. I want to note that this company supports Russian cards, and it will suit many Russian companies that cannot pay, for example, for Open Router, Anthropic, ChatGPT, or some other foreign services. Here you can pay for them and use all these neural networks. At the moment, you can see that Gigachat and all neural networks are free. You can use them freely. New models are also added promptly. For example, GPT5, Opus 4.1, Open Source neural networks, Open AI, Kenk 4, and so on. There is also a multi-chat interface. This is an interface with multiple neural networks. You can select the desired neural networks from the available ones here and compare how they work. You enter some prompt, and it is sent to several neural networks at once, and you can compare the answers from different neural networks very clearly. Well, in general, this is a very convenient service for comparing neural networks. You can test them and choose the desired neural network. So, let's move on to the code generated by Gigachat 2 Max. It is immediately clear that the syntax is already highlighted, meaning the libraries are called incorrectly. And the code didn't even launch. In general, I periodically test applications from Sber, but they write code poorly. So, Gigachat did not launch. Next will be three neural networks from Quen. These are very good neural networks. of large size. And let's see how they coped with this task. This is what the windows look like. They all launched. It should be clarified that this is the latest version of Quen 3 235B, which was released recently. It is released in two variants: Instruct, i.e., without reasoning, and Fine-tuning with reasoning. Encoder is a specialized neural network, I think, with over 400 billion weights, focused on programming. Let's see how it works in the programming task. Here I made a typo in the prompt, and accordingly, the title is slightly different. Keven 3, not 3. In general, all programs launched. Let's paste the key. Let's start with this neural network, with instruct. So, something crashed. In general, this is a neural network without reasoning. It was interesting to see how it would work. So, thinking correctly verified the key, and Recoder also correctly verified the key. So, the button is inactive here. So, there are no errors here, but let's close. Well, let's see how they cope with the chat. So, it was sent via Enter. It wasn't sent here. So, fine-tuning uses a different model, accordingly, accordingly, it doesn't work. That is, a request is sent to OpenAI for Infc using LM, and the Quin 325B neural network is set. Logically, it's not there on the OpenAI website. Recoder coped. It works. So, well, here the new answer is written in the same line, which is relatively inconvenient, but in principle, it's correct. And now it started rewriting. Uh-huh. Well, in general, the solution is approximately working. Let's ask a long question. Well, in general, it coped. Yes, but it also answers in Russian. That is, everything that is the entire chat history is fed into the model. And, in general, it works like this. Everything is fine, let's close. Fine-tuning coped here. So, check. So, now the encoder coped with everything. So, instruct, what was there with us? Yes, it didn't cope with verification. So, and fine-tuning, the chat screen appeared, but the dialogue didn't work because a different model was used. Well, in general, the Encoder coped with this task. The only thing is that the chat dialogue had minor problems. Let's also make a note. So, and here, probably, it's not a small model, right? Here's a typo, it should be like this. That is, the large model coped with this task, the small one did not. But at the moment, the open-source OpenAI coped. GMI 2.5 Pro and Quent Recoder. Well, let's move on to the last batch of neural networks, GLM 4.5. This is another Chinese company, another Chinese neural network that was recently released and shows good metrics. Neural network with 1 trillion parameters. Stop. Mika 2 with a trillion parameters. JLM, I think, with 400 billion parameters. JLM IR with over 100 billion parameters. So, approximately the same size as GPT 120B. Let's start with GM 4, by. So, it launched. J M4 with AR also launched. Well, let's test. So, the large neural network. Let's check the key. Server is overloaded. This OpenAI interface error is unclear. Let's try again. Ah, valid, right? So, here the answer was written correctly. That is, the request to OpenAI went through, but the OpenAI website returned an answer, and the program correctly showed the error that the API is currently overloaded. On the next attempt, everything worked correctly. The key is valid. Let's save it. So, it worked here. So, it worked. Here it also worked. So, actually, I like JM 4.5 the most right now. That is, the history works correctly here. The question from the user and the assistant's answer are highlighted differently. The IR interface is a bit simpler, but in general, it also works. It works. It's very convenient that it's highlighted immediately, and the answer comes here. Well, this is a good result, and a new company from China wrote a fully functional application with one prompt. This is a very good result. Here for GM 4.5. I think here it's fully like this. That is, the text is still written correctly, and these two neural networks worked well. And let's move on to the last ones - these are huge neural networks. Kimika 2 with a trillion parameters and Xik with over 670 billion parameters. Let's see how they coped with the task. Mika 2 is a Chinese neural network with a trillion parameters, and Psyk. The latest reasoning Psyk. This is what the generated program name looks like. Let's paste the key. The key is valid. The key is valid. Let's save it. And let's look at the dialogue mode. So, Psyk coped. So, why isn't it responding? Ah, it responded. Well, here it's not entirely correct, that is, the assistant is inserted after each paragraph, but in general, streaming works, the dialogue works. So, 2 made the same mistake as the OpenAI and Ontropic neural networks. at the chat stage. So, 2 coped, coped. Chat dialogue no, streaming no. And Psyk coped completely. Well, this is what the testing turned out to be. Honestly, before recording the video, I thought the result would be slightly different. It's strange that the neural networks from Grok worked poorly. Paid neural networks from Open E also worked poorly. It's surprising that the open-source OpenAI neural network coped with the task. Grok 4 did not cope. Sber, like Grok 4, also did not launch. In general, what are the conclusions? There are neural networks that can write code well. At the moment, what I liked was Gigapsyk, GM, both large and small neural networks. Quencodder showed itself well from the first prompt. Open-source neural networks from OpenAI. In principle, you can run them and code locally. And, in general, you can completely abandon some paid services. So, what can be run locally? You can run these two neural networks with 120 billion parameters. Running NRCer is difficult, it's 400 billion parameters. For this, a server is already needed, that is, even to run it in RAM. JM 4.5 can be run, it's a 100 billion parameter neural network. And Psyk locally on a laptop or desktop hardware is difficult to run. And, naturally, with quantization, it will work worse. In general, these are the conclusions. Neural networks are getting better and better. This is already the fourth comparison of top neural networks. And it is clear that neural networks are appearing that can write a significant piece of program with one prompt. Moreover, you can improve them further, feed them exceptions so that it fixes them. And, in general, development can go quite well. It's great that neural networks with a small number of weights, around 100 billion, are appearing. These are JM 4.5 A and GPT OSS with 120 billion parameters. In principle, they can already be run on a laptop. If you take a top laptop with a 5090, install up to 100 GB of RAM, and in principle, on a laptop, albeit a powerful one, you can run top, well, not top, but quite powerful open-source neural networks and use them for programming. My name is Vitaly Kuliev. Write here in the comments on YouTube what was right or wrong in this testing. In my opinion, it is quite objective. It covers the task of developing an application from scratch. Further, you can use these same neural networks for code correction, either in semi-manual mode through plugins like Proxy II, or in automatic mode through special software like Cursor, QnCoder, and others. Well, write questions, what you are interested in testing, what new neural networks have been released. In general, this is what the testing turned out to be. Also, in the Telegram channel, there is an active discussion on the use of neural networks, on their launch, so in the description here under the video, there will be a link to my Telegram channel. Join, and there you can very quickly discuss some important issues related to neural networks. My name is Vitaly Kuleev. Goodbye everyone. See you in the next videos.