Transcription
I've tested ChatGPT's deep research, and it's terrifyingly good. Okay, let's check it out.
So, on the 2nd of February, OpenAI introduced Deep Research, and it tells you, blah blah blah, what it does, um, how to use it. We'll go through that, but how it works. Deep Research was trained using end-to-end reinforcement. It goes backtracking, reacting to real-time information where necessary, and it's doing really, really well with AI tests.
This one's really funny to me because it's humanity's last exam. If you go here, this is the sort of questions it asked. Like this one here is a representation of Roman inscription. Originally, it's terrifyingly hard. Um, and so I don't know why I'm saying terrifyingly. I'm terrified. Um, you got this one: Classics, ecology. Look at this. Computer science. Let G be a graph, an edge indicator of G. These are incredibly tough questions, but you can see that it's actually doing really well. 26.6% accuracy as opposed to, you know, like what's the best one before that? 13, which was the uh OpenAI 03 mini. So, it is getting so, so smart. But I wanted to know, can it do its smarts for research and academia? Let's check it out. And uh, yeah, it's it's so, so very good.
All right, so I just want to set the kind of the scene for you. This is what ChatGPT used to look like. So here I've got a simple prompt. I am writing a literature review, and I'd like for you to conduct a literature review on OPV devices. Use peer-reviewed literature and adhere to academic standards of writing.
So, its response is kind of, you know, what we've expected from ChatGPT, and it's pretty good. You know, like this is what we were used to. Um, it's relatively short. It's got all of the like little tails like "in conclusion." Um, in terms of references, references are there, but you know, this has got Wikipedia. How many has it got? It's got like one, two, three, four, like five or six references. You can click on the sources here and see them. So, uh, yeah, you know, you can kind of see that it's done its best, but it's not really doing great. So, I want to know how much better is Deep Research compared to that.
So, when you first sort of like go to ChatGPT, you've now got a new button, which is not there at the moment. If I refresh the page, ah, there it is. I paid that hard-earned internet money to get this button, Deep Research. And so you put in your prompt, prompt, prompt, prompt, prompt, prompt. And then you click this Deep Research. This button cost me $200 US to use, but uh, I think it's rolling out to sort of like plus users and premium users and institutional users later. So you don't maybe have to pay $200 when you're watching this video, but at the moment, that's how much it costs. But I'm willing to take the monetary sacrifice for you. So let's have a look what happens.
So, you put it in there and then you click go. So, this is what I asked it. Um, I'm writing a literature review. It's exactly the same prompt as before. And this is sort of like that first little step that was a little bit interesting. It said, "I'd be happy to conduct a literature review." Oh, thank you very much, Deep Research. Um, "to tailor the research effectively, could you please clarify the following?" And it's got scope, time frame, key topics, preferred sources, format.
So, I copied and pasted those questions into the uh message uh box. I was going to swallow and burp at the same time. That's interesting, isn't it? And so here we've got scope. I wanted materials, efficiency improvements. Key topics. I want it from like the past five years. Um, and preferred sources. Prioritize best information. I didn't say literature. Um, format. Structure the literature review into themes such as materials. And then we've got length. I wanted 10,000 words. I'm going big. I'm going big with this one. I wanted to push it to its limits. And then it says here, "I will conduct a comprehensive literature review." I thanks very much for keeping me informed of your progress, mate. And then it says, "I will notify you once done."
And so it went away and it was doing all this stuff. It took uh, in this case, 10 minutes and it found 40 sources. And I'm absolutely sort of like blown away by what it found and also how it sort of like characterized the information and put it into an easy-to-read format. And this is what it's all about. This output here: Introduction, OPV devices, solar cells, blah blah blah. And like it starts well, it's really sort of in detail. We've got materials. Um, if you scroll down, you know, we got subsections: Material stability and morphology control, history of OPV technology development, um, and uh, this best performing OPV devices. So, these things are I told it like I wanted you to do these things. It didn't put history first, which is what I would kind of expect it to do, but nonetheless, it followed my sort of instructions to the key, followed my instructions to the whatever, very accurately. Um, and so if you want to provide some sort of structure for your literature review, just make sure you sort of like set that structure out before you get it to go off and do its things.
So, it's structured how I asked it, and you've got all of these different uh references. So, if you click here, the next web startup. So, this is not peer-reviewed literature, but it is like, um, you know, online articles, but it does link to where they're actually sort of like talking about the thing that they're using the reference for, which I really like. Um, and then we can go back to this one. We've also got ResearchGate. So, we've got real peer-reviewed references in there. Um, I feel like it could have done a little bit better with selecting, you know, for academic standards, but nonetheless, you know, you can go through this yourself and make sort of like sense of the articles, which ones to use, which ones not to use, but it is still in there. Look, it's read all of this and it knows it's getting it from this bit of information here, and that's what I really like.
So, in terms of its output, in terms of its depth, in terms of the references it gives you, I cannot fault it, really. This is the best AI literature review from ChatGPT that I have ever, ever seen. And uh, it's only going to get better. If you're entering a new field, if you're entering a new sort of like project, doing one of these upfront will save you hours and hours and hours of writing, researching, anything you can do now online where you want deep research, this can do for you. And I think that's going to really, really upset some people in academia because it's now got easier than ever to get expert information sort of like referenced well in just 10 minutes. That took me during my PhD days of researching, reading, synthesizing information. Now it's just done for you. So, in terms of, you know, creating a literature review, creating a review paper, you can now get all of that information in a fraction of the time that you used to. Just incredible.
Um, I'll go through my limitations in a minute. I I I think like there are some things that for an academic purpose this doesn't do too well. But let's test something else. Do you like my new t-shirt, by the way? Sewed that up this weekend. It's nice, bright, lovely. Mmm, love it. Fits well. Custom made. All right then.
So, the next thing that I want to do was write an introduction to a peer-reviewed paper. So, not a literature review where you go out and find loads of everything. It's a very sort of like sort of condensed uh literature sort of search over a very specific topic for a peer-reviewed paper. We're talking 300 words max for something like this. So, I gave it this prompt: "Write me an introduction section to a peer-reviewed paper about solar concentration mirrors." That was one of my projects as well. Um, and advancements in the past five years. It's going to be at the beginning of a paper that shows environmental stability of mirrors with various adhesion layers to a glass substrate.
And once again, it comes back to me and says, "What do you want actually to know, mate?" And I'm like, "I want to know this." So, it wanted to know the type of solar concentration. It knows to ask these questions. What? What? That's insane. What? Okay, that's the better intonation for that, isn't it? Like, what? All right. So, now we've got a type of solar concentration mirrors being studied. So, I was doing Helios. The primary environmental stressors being examined. I could answer that. Any specific adhesion layers and the intended focus. So, I sort of like copied and pasted that in and then sort of said, "Type of solar concentration mirrors with the Helioat. Primary environmental stresses being examined with temperature fluctuations and mechanical wear." So, all of this stuff, you know, it knows to ask, and it's now sort of like using that to go and research and do its deep research.
This took 4 minutes. It found 24 sources. If you click here, it will take you through all of the stuff that it did, which is just incredible. So, you can go through and uh, you know, see what thinking it did, really. And uh, yeah, it's just a really, really great way of uh starting your research these days. H oh, I'm I'm jealous. I'm jealous of you for being able to use this. Now, if you've got $200 at the moment, it'll be cheaper in the future, I think. Um, and then 24 sources. You can click here and you can get your sources. Oh, nice. Look at that. You can export those sources. I think you would have to go and, yeah, sort of like put them in manually somewhere. We'll talk about limitations in a minute, but they're at least all there for you.
And the output, let's get rid of that bonk. And the output here is okay. It's not what I actually asked for it. I don't think it knows what the beginning of a peer-reviewed paper really looks like, but that's fine. Um, it gave me too much, but I'd much rather have too much than too little at this point. So, it's got the 24 um references, so I could use those. And then I actually asked it, "Make it shorter and suitable for the introduction paper of a peer-reviewed article." I meant intro introduction part, but it understood, um, about 300 words. And this is then what I could use at the beginning of a peer-reviewed paper. Um, in the introduction section, it didn't actually give me any references for this bit, but I can use the references that it gave me before. So, I think that's okay. It's not ideal that it didn't put the references in here.
But in terms of output, "Helioat mirrors play a pivotal role in concentrate uh concentrating solar power systems by reflecting blah blah blah. One king development is..." It's just perfect. It. Oh god, I'm having a breakdown. Oh, this is better than I would write as a first draft. This is better than most people would write. This is better than my supervisors could write as a first draft. And I'm happy for you. All those hours I wasted. Now with technology, it's just so easy. All right. This is good. This is This is good. Good. This is positive. Positive for you. Sad for past Andy. Um, but yes, it does this so, so, so very well. And uh, you know, for $200 bucks, if your supervisor is paying for it at the moment, I don't know why you wouldn't get this for your lab because it will really, really accelerate all of the writing that you're going to do, and all of the important writing that furthers careers is going to get easier and easier.
But there are some limitations. All right. The limitations that I can see for an academic purpose, let's take this literature for example. Um, you cannot actually uh output the cited sources in a way that you would want to use them. It doesn't output it in like LaTeX, for example. So you can put it into Overleaf. It doesn't uh export it into like BibTeX. It doesn't export it to Zotero. That is where I think, you know, specific companies will be able to swoop in at the moment and be like, "Here's the output, and also we've done all the referencing for you as well." Something like Thesis AI is great because you can export it out to um Overleaf, and then you got complete control over all of the referencing, just like you had written the paper. That's where this kind of like struggles. Um, and also just in terms of, I think some of the word choices, you know, it just sounds still a little bit a little bit AI.
But overall, in terms of how great this output is, how referenced it is, how you could probably just use this as it is, I think the journals are going to have to be updating their um submission guidelines and their AI guidelines because this is just so easy to do. Now that people will be doing this, and they'll be fighting a losing battle if they try to stop people just doing this. You know, clearly you need to go through and you need to read it and you need to check it and you need to make sure it's not hallucinating. That's going to take far less time than going through all of the literature and coming up with your own kind of like uh, you know, group of themes and that sort of stuff. It's all done for you. This is the future of research if it will only sort of like, you know, perfect those last little things like being able to export the references to a reference manager, being able to copy and paste it across to Word, Overleaf, or something like that. I think this will just be, I mean, I I hesitantly use this in real life. I think it's a game-changer.
If you like this video, go check out this one where I talk about the 10 ways that NotebookLM can make academia easy and dare I say it, fun. Go check it out.