Transcription
AI has promised us easy literature reviews, but not all literature reviews are created equally. I've tested the six best tools and here are the results.
So, I tested Scyace, Thesis AI, Answer this, Chad, GPT, Gemini, and Manis. I've tested these in the past and they've been the ones that have stood out to me the most and that's why I'm doing them in this video.
I tested a number of things. So, first of all, number of references. I want in a literature review as many references that relate to my topic as possible. The more the better in my opinion.
Now, length not necessarily important, but we do want to make sure that it generates enough content that actually is meaningful for the sort of literature review we want to do. There's no point just having a couple paragraphs. Length does matter, but obviously we don't want it to be so long that it's just boring and repetitive.
Another thing is readability out of five. So, as I read it, does it sound like something I would want to submit or something that I would be happy putting forward as my own work?
And then we've got exportable. Now, it's very, very important to me that these tools generate something that's exportable and therefore editable. Now, a lot of tools just say, "Oh, here it is, but I want to then put it into Word. I want to put it into latte. I want to put it into the things that actually allow me to work with that output."
And then we've also got AI detection, as in can I pass it off as my own? Not something you should do, but starting with something that doesn't have AI detectability or has a zero or very low percent means that you're not going to get stung down the line when you changed it a lot, but it's still working with that AI output.
Okay, then. So, let's just go through each column. I think that's probably the best. Let's take a quick look at the outputs first. So, here we have size space. It's very, very simple. I put in the same prompt for everything. It's this one here: "Write an academic literature review on self-healing nano composite transparent electrodes in a formal scholarly tone. Synthesize the key theories, methods, and debates in the field rather than summarizing individual papers." And this is important for something like chat GPT where sometimes they'll just sort of like assume you want the easy answer. I want it to go into detail and do a little bit of analysis. "Critically evaluate the areas of agreement, disagreement, note strengths and limitations of existing research. Clearly identify gaps or unresolved questions. Write for a graduate level academic audience and avoid inventing citations. We don't want any hallucinations, do we? And state uncertainty where needed." So we don't want it just to say yes, these are the facts. We have that hedging in academia that is so very important. Let's see if we can capture a little bit of that.
So this was the output of size. Oh yeah, it was good. It's long. We'll talk about the length in a minute. So it has lots of output. It has uh references. It has loads of things that we would expect. It doesn't have the multimedia that some other ones have, but overall it's long, it's detailed, and sci-pace has got an excellent database behind it to pull that information from.
Then we have thesis AI. Here's thesis AI. Once again, I've put in exactly the same prompt and I got this output. So, all I've been testing it, you I've been highlighting things, but um you can see I've got a content which is fully kind of linked, which is really nice, and it is dense. It is long. It is full of everything that I would want.
This is answer this. Answer this. Once again, I've got that same prompt and it's much shorter. It's the shortest of all of them and it's only got six references. Hm. I'm not too sure about that. And then down here, you can see that we've got only where is it? It's down here. Only one or two references for this entire paragraph. So yeah, and also this term here, "flexural endurance." Is that a word? Not sure. I've never seen it before.
Anyway, then we've got chat GPT. We'll go through that. We got Google and also Manis. Manis AI is a sort of really nice agentic AI that I was very impressed with a few months ago. And yeah, I think this sort of stands up to uh rigorous testing. So here it is. Here's everything that it generated. Not the longest response, but uh here we go.
Here are the results. So now we want to compare it with a little summary. Let's go number of references. So I'm going to highlight this. And then I'm going to say make it dark. There we are. So SciSpace is 28, Thesis AI is 13, we got six for Answer This. We got ChatGPT 16, Gemini is 36, and Manis AI is 19. So, if you're looking for a literature review that gives you a load of references, I'd be selecting Gemini or SciSpace. SciSpace probably is my go-to at the moment for... Oh, I'm going to sneeze. [clears throat] Oh, I felt that one building ages ago.
All right, then. So, SciSpace would be my go-to if I really wanted an academic AI tool, but Gemini as a general large language model does give me the most information, which I absolutely love. And then you can see Answer This is the worst, only gave us six. So I really like this. Go to Google Gemini and you can see all of the stuff that it's generated. The one thing I love about gener um, you know, the output from from Gemini is the fact you do get these tables as well. It does synthesize the results and uh, yeah, it is a really great way of sort of like getting a snapshot. Uh, it's free to run as far as I can tell for the most part. And uh, overall, it's, yeah, a really great something. And you can also go on and sort of like create other things from this output, which you can do with a lot of the other tools. But in terms of length, yes, SciSpace and Gemini, they're the best at the moment.
Then length, right? Length. Does size or length matter? Yeah, it does. I don't want something that's mega, mega short because this is a literature review. If I'm using it in the real world, I want to sort of see what sort of themes are pulled out, what sort of order things are presented. I also want to see what kind of references it digs up from underneath so that I can talk about it in my own words. Remember, typically you're using these sorts of outputs in academia as a foundation, as a basis, as a first touch point to an output that you would generate yourself.
So, here we've got length. Let's have a look to see what it is. And you can see that uh Thesis AI is the longest by far. It's a girthy boy. And uh it that is the sort of length I would expect, 23,000 words for a thesis uh literature review and introduction. So they really stand up to the name, which is Thesis AI. Um, and I think I'd aim for about 20,000 words for each chapter of my thesis. And that is right there. It does take a while to generate, but look, you're getting a lot of content while you're waiting. Um, next we've got SciSpace up here and then down the bottom, Answer This. So you can see Answer This really didn't try its hardest to fill out that information.
So let's have a look at something like Thesis AI. Thesis AI, this was the output and you can see it is long. It is girthy. It is dense. So overall, we've got an abstract, we've got an introduction, and it is very, very dense. We have got sort of like a little bit of um extra formatting in terms of a uh was that a a list? A not a bulleted list, a numbered list uh there, but that's about as far as it goes. There's no tables. There's just lots and lots of words, but it is well referenced. It does sort of like reference multiple things per uh paragraph, per section. It's not just relying on one thing. And so that is something I really like. You know, you wouldn't submit this, but you could use it to have a read of a particular research field before you start doing your own literature review.
Um, and then let's have a look back here. Answer This is the shortest one. Very, very short and only six references. So, this was by far the weakest uh that uh I kind of found. I even asked for it to like max out the number of uh citations. I wanted no less than 20 and it only found six. Who really didn't follow my prompts or my settings um as well as others did. Um, and then everything in between. So Manis uh, you know, is relatively short. Gemini is next. And then ChatGPT decided that it would give me nearly 5,000 words. So Manis AI, you know, it does sort of like look good. It does have these tables, which is excellent, and uh, you know, it has given me 19 references. So, it's a middle range kind of uh literature review. Um, if I was to, you know, sort of like mark it against the other ones, which we are doing, by the way, in this, aren't we? Um, and then we've got, uh, yeah, all of this stuff.
So, readability. Readability. Hm. We want something that actually sounds good to read. A lot of these AI tools just use randomass words that don't mean anything or try to sound too smart in a way that is obviously AI. So, I was looking for something that I would be happy to submit almost as my own work. Um, and that means that it uses the appropriate language. It doesn't go too crazy. And here are the results. Bonk.
So, overall, I think Thesis AI actually gets the academic language readability score the best because if you read, you know, it is a little bit um a little bit wordy, but it doesn't use anything crazy that I wouldn't use. You know, "A central advantage of safe healing configurations in this domain lies in their capacity to exploit the thermal mechanical stimula inherent to wearables." So, yeah, look, even that sentence is a little bit too long. It's a little bit too many words in there. It's a bit wordy, but nonetheless, you can take that and understand it. Um, unlike other things which I said down here, which was the Answer This one, which was interesting because it said uh "flexural endurance." I'm going to Google that. Is that even a word? Flexural relates to bending or curving, described akin to... Yeah. Okay. Flexural strength. So, it is a real word. Clearly, I'm the idiot in this one, but it's not a word that I would use. Um, and therefore, it's uh, you know, a little bit too wordy, a little bit too uh, you know, thesaurusy, I guess, for the amount of words that I know and that I would see regularly in the uh, research field. I think that's the one thing. All of these AI tools, they lack the nuanced understanding of each individual academic area and the sorts of languages they use. And so the words are just sort of like unnecessarily uh long and big and like someone's just flicked through a thesaurus and gone, "Great, that one's a better word." And that's why a lot of them just haven't got a really great um readability score. I think Thesis AI is the best. There's pretty much all of them in the middle do an okay job, and I think Answer This, because it was so short and because it really leans into um like the words that I wouldn't use and that the the field doesn't use um, I find that, yeah, it was just kind of like the the least uh readable out of all of them.
And so now exportable. Here we go. Exportability is so very important because you want to actually work with the information that is generated. Typically, you want to put it into your own um, like document editor. It could be Word, it could be LaTeX, it could be markdown language. But ultimately, these are the results. So, if I go here and click reset, this was what I had. So, overall, I think the best two are Answer This and Thesis AI. And that's because with Thesis AI, not only do you get a PDF output, but you also get it as a number of different exports. We got export to Overleaf, which is so important. It's the only one here that allows you to do that. Overleaf is a LaTeX um sort of like what you see is what you get editor, and it is really great because you are in control of absolutely everything, unlike some of these other outputs. Obviously, PDF is just PDF. It's not editable in the same way that the other formats are, but download Word. We've also got Word that we can work with. So that is really, really good. So over or exportable? Yes, Thesis AI does this. And adds to this was also really great because we can turn this into a notebook. We can say add to a default notebook, and then you get an output like this, and you can export it in a range of different formats. We got export to PDF, export to DOCX, export to LaTeX, export to Markdown. So all of these are absolutely perfect and how you actually would want to work with this output at the end.
And then one of the worst ones was ChatGPT all the way down here because I couldn't work out how to get this information out of here. ChatGPT does a good enough job. But uh, how do I get that? I'm not quite sure to be honest. I can share it. Okay, great. I can share it. Probably a link somewhere. Oh, it's taking ages now. Wish I'd never clicked that for this video recording. But you just cannot kind of find... Okay. Copy link. X. LinkedIn. Reddit. Yeah. Okay. Post that to Reddit. >> [laughter] >> They would hate that so much. All right. Um, but yeah, I just can't really work out a way to download. I can branch it. So, overall, you get the information. You could have to copy and paste it manually across to something like Word, but yeah, really not useful in this context.
And then also we've got Manis AI, similar issues. Gemini. But uh, SciSpace was kind of concerning because I really want this, you know, out. I really need it. But then I click download, download as markdown, download as PDF, and then I have to pay. All right, I know you want our money, but I would really like just to like save this in something other than markdown or PDF, such as Word or LaTeX, or even out to Overleaf, but that's something that really is lacking with SciSpace at the moment, which is such a shame because it really is doing so much sort of good work on the rest of the other things.
Now, AI detection. Hm. The last thing I wanted to know was AI detection. And we won't spend much time here because as you'll see, they were all detected by AI and it was 100% confident that they were generated with AI. This is the cat and mouse game of AI generation versus AI detection. Um, nonetheless, we wouldn't obviously be able to submit this to anything like Turn It In originality, uh, zero GPT. So, I used originality just because I had credits for it. But um, yeah, if it can't pass this, it's very unlikely to pass other tests. Um, so here are all of the things and the outcome of all of this testing is really, I think you've got a choice between SciSpace and Thesis AI. If you want lots and lots of references and you want a reasonably long literature review that reads relatively well, is kind of exportable if you pay, then SciSpace probably is your best bet at the moment. But I really like Thesis AI for literature reviews. I know it doesn't have all of the extra information that you would expect, like tables, even diagrams. I know I spoke to the CEO, he's working on that at the moment, but uh, ultimately this would be my go-to base for something I was working on. I wasn't impressed in this instance with the number of references it found. I could also add in my own references with Thesis AI, but this really gave me the best of everything. And Thesis AI gave me the number of references. I like the way it read. It was exportable into a range of formats. Then it is completely editable and uh, you know, like everything else, it didn't pass AI detection. But uh, yeah, down here Manis, ChatGPT, and Gemini, I think they're also good options. I probably stay away from ChatGPT for academic writing if you want to, you know, work with it afterwards because you just couldn't export it. But Gemini does really good. It does sort of like export it to Word, as does Manis AI. Um, so you can, you know, download it in a range of different ways, markdown, PDF, DocX. Obviously, we wouldn't expect them to have LaTeX in a general tool like this, but it would be really nice to see that in the future.
So, there we have it, ladies, gentlemen, and everyone in between. We have got SciSpace or Thesis AI really as the winners at the moment. Let me know in the comments what you think. I'd love to know. If you want to know how to do a literature review using AI tools, go check out this video where I talk about using AI for all of the different steps of a literature review. I think you'll love it.