How Google Defines Originality in the Era of AI-Driven Search - Koray GUBUR - Pavel Klimakov

Hello everyone. Here we are with Pavl Kilico, one of the latest legends in the semantic SEO field. And not only the semantics, also beyond the semantics, or overall holistic. He is one of the top leaders in the holistic community for over 40,000 people. And he performed a great speech in the SEO Master Summit Saigon in 2025. The speech was longer than my speeches, to be honest. And the audience was also laughing at that, but the speech was also a kind of academic level. And I believe it carries actually a really good amount of value to explain how AI systems actually understand, let's say, the copied content or originality signals.

I also performed a kind of private speech earlier for Chargeflow's events, and I directed there also. I stated that Google algorithm or the engineers, every year or every two years, they try to actually focus on one of the different types of criteria to differentiate quality sources from the non-quality ones. Let's say, around 2019-2021, momentum, which means publication speed and update speed, was really helpful for us. Even if you publish 3,000 documents a day, you were just increasing in the rankings. And then, thanks to the AI, they started to understand that these are AI sources, and they started to give manual penalties. And lately, it looks like they focus on originality a lot. That's one of the reasons that actually we see forum websites are increasing the rankings a lot, or websites with a function or with a brand. When we say function, we mean actually SaaS websites, e-commerce sites, or forum sites. They have a function. But when it comes to content sites, they don't have any function. That's just a wall with a text and image. And it's not that much different than actually the wall behind me.

So, in this context, today we are actually processing the importance of originality and how Google actually and technically can understand originality of a content, either page level, either sentence level, paragraph level, or maybe visual or visual level, or HTML level, or layout and design level as well. And we are trying to, we will be trying to understand how they can actually filter your website if you are using AI-heavy websites in this case. At the same time, we are trying to, we are trying to actually demonstrate his speech on Saigon. I believe it is already open to the SEO Master Summit members. And many thanks to the audience for organizing it. And maybe they will also allow it, allow us to add it to our topic of the course later too, or share it to our Facebook group or Telegram group, etc. We are going to see that how generous he is. And after this long introduction, welcome Pavl.

Thank you very much. That is a record-breaking long speech. Thank you. I believe you are right now in Estonia. Yes, correct. And you are one of the, let's say, very famous SEOs from Estonia, I guess, right? There are not that much people in the SEO field in Estonia. I believe the country with low population, we can say, right? Yeah. Well, if you put it like that, then definitely it's not that big of a country in the first place. But then also, I think the amount of people in SEO, it's really also not, not that high amount. Yeah, I can imagine that. And I believe you were living in Mai like for the last four, five months, and you usually come to Europe when it is summertime. That's has been my plan for the last three years. I just tried to escape the cold winters of Eastern Europe. And this year, I was in both Thailand and Vietnam. Three months and three months.

I believe you also help some SaaS companies for consulting in the semantic guest field. You also help some e-commerce websites or different types of industry websites as well. How does it work for you, or how do you help them? Can you explain a bit what you do? So, in SaaS cases, uh, usually SaaS providers, they try to evolve or and adapt to the search engine. So, they're looking for different types of evaluation systems or new metrics that they can include into the software. And in some cases, uh, it's a little bit, let's say, challenging for them to adapt to the actual current scheme of the search engines. That's why they need a little bit of, let's say, guidance, instructions in terms of, uh, what to prioritize, how to score different things, or how to actually essentially put everything into the code so it can function properly. So, these are essentially, it's like a tying in the SER and the evaluation of the SER and the search engine to the actuality of the algorithms.

Okay, I understand. And things that actually usually help to do AI SaaS companies or SEO SaaS companies in the context of again, maybe semantics or natural language processing, and also originality or detection of the application or automated content. Am I right? Yeah, of course. This is like, you know, everyone's excited about AI. Everyone is trying to produce content with AI because that seems like a very logical sort of solution to use AI. But the reality is, there's a lot of, let's say, pitfalls or small traps that are actually quite unknown to our industry in general. So that's kind of what mostly my speech in Saigon was about.

Okay. So, in this case, uh, I will just ask an honest question. Do you think that SEO SaaS owners or AI SaaS owners, do they know that what they are dealing with? In 99% of the time, it's no. I can, I can say this. I noticed this pattern, and this is not, let's say, an attack or any kind of negative comment towards anyone. But usually, SaaS, if it starts for, like, a personal goal or a personal vision, in the initial phase, it's really good because usually the owner uses the SaaS by himself or by themselves. So, at that moment, it is really efficient, and it works quite well. After that, when they realize, "I can sell it." Their goal changes from having a good product towards, let's monetize it and basically get the most money. So the focus now is on marketing and selling. And then it goes like, "Okay, let's add this shiny dashboard. Let's add this new cool feature that all the competitors have." And basically, now it becomes this average tool, exactly like everyone else has, just with different branding and a few different, you know, terms. Just a logo is different, but, uh, everything else is actually the same. I believe there is a really good tool inflation in the SEO field right now. And the worst part is that I believe there wasn't any other moment or age in the SEO that Google's search features are so advanced, and most of the time, SEO tools even don't know they are there. I mean, let's say you are helping some celebrities, and there are so many specific features just for celebrities. Or just, let's say, you are trying to track actually some other type of industry features, like, let's say, forums and discussions, and you want to compare Stack Overflow to Reddit for some angles, let's say for dates or the amount of activity in the threads, and how they are triggering actually appearance there, or how the cannibalization between different subreddits actually works. So if you want to analyze such a thing, you have to spend so much time in a creative way there. But there is like zero sites for any of these fields right now. And they're all actually, again, come back to the topic of originality.

Lately, I created a new concept, like exact match subreddit, like exact match domain. And I am, I am creating, not even buying, creating lots of exact match subreddits, like allergy, or e-commerce Australia, or let's say, even condom, to be honest, or let's say, I don't know, many other things, like weapon, etc. Any, any word in the English dictionary can be a subreddit, basically. And I was saying something similar in my Saigon speech in 2023 or 2024, basically like, take the Oxford dictionary and add the word AI to the end and buy all the domains. It was the EMD strategy. Now I am doing the same for Reddit, technically. But I can tell that that application and originality system between, between Google rankings and Reddit, they have a kind of different structure there. For example, if you add two similar, similar pages to a website, they cannibalize. If you do the same for a forum, they don't cannibalize. Instead, they consolidate and they come together, and they even rank better. So, the forum SEO is a little bit different there, I can say. But technically, the main point is there is like zero SaaS tool out there for tracking these things, and it's like we're in a dark age, to be honest. And it also bothers me sometimes.

It is very, very true. And I can say I heard you like a year ago, you were mentioning how search engines separate queries into like factual or opinion-based queries. Yes. And based on that, we see different features on the SERPs. And that's like a such a simple and easy thing to integrate into almost like any kind of tool or even, let's say, plugin or anything. And that's just completely missed, basically, by that. It's such a critical element, actually, for producing the proper content. It even will again affect the original angle, too. If it is about experience, for example, originality wouldn't be that level of importance because it's just coming from casual language. The, the engrams or the predicates or the sentence structures that they look for. It doesn't require any expertise there. So it can be very casual, very short, very unstructured. When it comes to the factual part, it has to be very structured, very unique, and accurate, most important, I can say.

And by the way, uh, just to add to this, I kept thinking for many years together with lots of other awesome holistic SEO members and thought leaders, what does it actually mean to have human efforts? What is that thing actually? And to be very honest, let's say we can, you know, really prompt LLMs and work on a like very sophisticated design to produce high-quality articles wherever it's factual. But if you find these extra ways to adapt to like personal experience, when the language starts from like, "I did this," or "In my, like, I experienced that," it's much harder to actually do it correctly. Plus, if you add originality on top of these things, it becomes like a complete madness. Uh, so that's much, much measurement method of these things don't exist for inside any tool either, too. Like originality. I mean, when we say original, we don't just mean plagiarism, okay? It's, it's beyond that, actually. I mean, when you look at to the actually the concept of stylometry, basically, it is about understanding who has written what. The concept came from actually a discussion in the history of the United States Constitution because apparently, 13 pages of the US Constitution, nobody knows who has written it. And Alexander, I guess, Alexander Hamilton claims that he has written it. And there is one more guy, Thomas Jefferson, I guess, he claims that he has written it. And then they actually check the style of writing. They get both of their articles and they just run a kind of algorithm to classify their, let's say, word, for instance, word distances. And they try to understand statistically, these 13 pages are closer to either Alexander or either Thomas. And they were able to actually understand who has actually written it. Apparently, it was Alexander Hamilton based on that structure. So, based on all these, let's say, author signatures, they can actually understand who actually has written an article. I believe years ago, I have written a simple case for it, importance of author or authorship, uh, to explain. It was a long document that explains that. But I believe today it becomes even more important.

And another thing is about the categorization of sites. Let's say we have an e-commerce site, and we have, let's say, a regular content site. Even if you use AI without human effort, sometimes Google might tolerate actually an e-commerce site while they don't tolerate a content site at all. They directly remove that one because the other one has a function. So it, it actually brings many things into the angle there. But let me ask you a question about e-commerce. Let's say you want to use AI for creating better product descriptions for 40,000 products. Uh, and you also want to provide originality in this case. How would you do that? By the way, for the audience, uh, we didn't prepare these questions. Pavl doesn't know what I'm going to ask. Even I don't know what I'm going to ask. So, this is just a question. I keep always my interviews natural. And now we listen to Pavl.

So, originality in the sense that we want to produce some kind of unique angles or perspectives. I thought, let's say, uh, a few things where it can be a pretty solid solution. I, I'll share both of them. So, one of them, let's say, you have an existing database, uh, which, which means it's just a list of these products with very generic information, like, let's say, price, size, things like that. So, there is not that much, let's say, attribute coverage there. One of the ways is you can prompt or process all of your products through different AI systems to try to understand what other additional attributes you can add or integrate into your listing that will help show different types of angles or different types of things related to these products. Then comes the next, let's say, tricky part, because you'll still most likely have to collect that information somewhere, which will be again somewhere else on the web. And here can come this point of, let's say, there could be valuable information stored that can be retrieved, which then you can integrate into your knowledge base, making it richer and, let's say, more unique. But the point is that, um, essentially the valuable information can be stored and be retrievable, but it is not a part of, let's say, highly authoritative sites that are essentially highly prioritized for all the rankings. So, it's a kind of information that's kind of can be scraped, that is, let's say, safe to use or reuse.

Okay. Another, another one, uh, just to mention slightly, uh, something that I'm paying attention to a lot right now is the Google's AlphaFold, which is a multi-agent system. It is really used to, let's say, improve different types of codes and different types of theories and things like that. But the most interesting part there is that essentially they're pushing the AI to innovate and say something completely new. And then they're trying to filter out something that, let's say, would make sense or would be provable that it's, it's not some random nonsense type of fluff. So that's a fascinating part. The only thing is, they, of course, didn't open source the code and the whole procedure. But, uh, some of the concepts are pretty clear in terms of how they're doing that. So I think it's a very interesting thing. It still needs a lot of, let's say, work and research and attempts. But in theory, you can rely on AI's creativity as long as you can clean all the nonsense from the, like, you know, remove the bad stuff, keep the actual new innovative things. So, very interesting, actually.

So, basically, it's a kind of new thing, I guess, AI creativity. So, AI actually now can create new opinions that are not created before, even by the human civilization, you say? Yeah. So, it's like a new level of originality. Okay. I believe mathematically it is possible, but that living part might require maybe quantum computers. I'm not sure because mathematically, it is so vast amount of possibilities for a computer to bring two irrelevant things together and understand it is not relevant, and bring another, then bring another. It's a bit hard, I can say.

That's a great point because Google has another research paper where they essentially try to create a kind of like writing agent, and they try to improve the existing text just to make it, let's say, better, more readable, just different types of criteria that they can find. So, the criteria that AI proposes, they still need to be filtered out. And what they find in that research paper is it can give you a list of good criteria, or in general, a list that includes good criteria, but all systems struggle really badly to filter out good versus bad. That's why there is lots of like loops and chains and lots of things, and basically, the success rate is maybe like 10%. But the point is, even lower than that in my opinion, because, uh, I believe it is for promoting a standard. It would be lower than even one in my opinion, because as, as I said, that there are so many vast combinations of possibilities there, making it run successfully requires you to perform a predetermined seconds filtering because there, this is basically a seconds or a kind of path exploration model exists there. They are trying to understand which path of the words make the most sense. And it means that you need a second filtering system to understand, okay, this predicate with this noun, or this context with this context are also new and also logical. So, the logical section is actually easy, but also making it new, which is something not existing in the database, that is, uh, very hard. And I believe quantum computers can do that. I agree with it. But with the current CPUs and GPUs, uh, it will be really in my heart. But if it is happening, you can solve all the mysteries on the universe, which is completely. Well, there are, let's say, promising breakthroughs. Something that I remember from a few days ago, this AlphaFold, the Google system, there was some kind of mathematical solution that required, I believe, like 49 calculations, and that was a mathematical problem that was basically solved like 53 years ago or like, you know, long time ago, and nobody could create a better solution. The system found a better solution and it reduced the amount of calculations from 49 to 48. So, you know, it's not groundbreaking, let's say, but the matter of fact that essentially humanity gave up on this problem and just used. There's one, one key point there, I believe, because as I said, they, they filter the subject because they focus the system into one main thing only, and they run it only for that small thing. If you have a generalized model, that model will be bigger than maybe the universe, like. So, maybe for specific questions, specific problems, yeah, it might be happening. But automating it for SEO purposes, I guess another decade, we might need to wait for now. But it is a very good, good thing. I mean, even in my, in my some of cases too, I always that actually, yes, LLMs can actually, let's say, outrank many people for originality too. But when it comes to outranking entire humanity, it's very hard because it is coming from our collective memory and collective database that we gave it to. So, it's very, it's important, like actually inventing fire, in my opinion. It's really breakthrough.

Yeah, it is. Uh, and right now, they're really focused mostly on mathematics and code production. That's really the biggest chapter right now with reinforcement learning to improve all of the AI models. The point there is that if you need to, let's say, give a reward or punish the model for making a certain decision, you need to be able first to verify if it's a valid decision or not. So, you can easily verify mathematics and code. You can, like, run it and test it, or you can know the correct answer. But when it comes to things like creativity, or logic, or reasoning, or like, you know, philosophy, or some other types of things, we don't know what's the actual right thing, or how to systematically verify that this is right. Okay, of course, we can use training data, but that's super limited. That doesn't something else.

One day you have shared me a very good research, maybe you remember. It was also using reinforcement learning for filtering AI-created words and human written words. It was a filtering original content from AI content, word by word, not even sentence by sentence. Could you also explain it a little bit to show the capacity of search engines today for detecting the AI-created content? Yeah. So, this is actually the paper from Microsoft, and it is part of the presentation as well. Um, a really important one. Essentially, the story there is that, uh, Microsoft is doing a kind of research. They're trying to understand if it's possible to detect AI-written content in any kind of way. And it's a, it seemed a little bit like a desperate study. Essentially, they hire a hundred random, like freelancers. They give them different texts and say, "Well, tell us if this is AI or not." And what happens is that there's five people out of those 100 whose accuracy for predicting if it's AI or not is like, you know, like 99% or 98.5%. It's like super accurate. So, all of these researchers, they are like amazed and at the same time shocked. So, they start to, you know, like annoy these people, like, how do they do it? And how, who are you basically? How did you figure this out? And what happens there is that these people essentially, they use AI for writing on a daily basis. So, they're very familiar with the kind of, like, patterns that AI produces. And so, the next step in the research, they start to extract all of the patterns from all of these writers, and then they put them down. And so, the fun part is that is actually mentioned in that paper as well. That's where, uh, we see things like, let's say, certain types of words and phrases are very common for AI to kind of default to when it tries to explain a certain concept.

Okay, I will just interrupt one thing there to make the audience also recognize. So, people usually, some people at least, they think that, okay, these are just researches, but they don't, they miss the fact that actually Microsoft spends millions to make these type of researches and also then integrate them to their own system. So, why do you think that they, they did this research? They just wake up one morning and during the breakfast, they say that, "Hey, let's have fun while doing this research. Let's gather 60 people who uses AI every day and write a research just for writing." Is this the angle, or do they have another angle there commercially? There is, if you look into any of these research papers, you realize how many other, uh, papers they reference, and you realize that there's a whole hidden industry where lots of scientists and students and researchers try to push the limits of, like, figuring what's possible. Because, and by the way, the starting sentence for all of these papers is something like this: "AI has become so powerful that we cannot differentiate the writing, which is a growing problem for the web, for the, like, students that write essays and so on and so forth, for politicians, for everyone." By the way, there's a huge another thing that people really forget, it's the copyright industry. Because how do you prove who is the real author in that case? What if you, like, you know, copy someone's, like, writing style, write your own book, write it better than the original author? What's the way to prove it? And like, who actually wrote that book? Um, so anyway, um, it's a massive industry, and there's really lots of, uh, work happening there. It's just, you know, there is no hype YouTube videos about it. So, it seems like it doesn't exist, but it's, it's really, really massive, actually.

So, let's say Microsoft gets better at detecting AI-created content compared to Google. I am trying to just understanding how this helps Microsoft make more money, either through search engine, either through other products. Maybe they will be selling some software to the United States America government, clean all the fake authors from the book industry, I don't know. Or maybe they will be selling it to the courts to detect who is the real author of the book to protect the copyrights. It would make billions, I know it. But I'm trying to ask it to you so that you can explain to the audience what type of monetary value this thing has compared to Microsoft, and what type of software can be sold and where, with what purpose? Sure. Of course, the, let's say, the detector one, that's the easiest one, and there's a like insanely massive demand for that. Uh, there were a few research papers that were called something like DetectGPT or something like that, and they gained like 100,000 users overnight, essentially, just because there is lots of professors and educators in the world who work with students and with essays, and they need a solution. So, whenever that gets released, it's like spread immediately like a virus in the, like, academic world. So, that's one of the obvious ones. Another one is also job interviews. They also write fake answers in job interviews by using AI. But you continue. Yes. Yeah. By the way, there was a startup I saw, there was a guy that made a tool that allows, helps you to pass the job interviews, and the guy was kicked out of the university or like banned or something like that. But that software made him, I think, a couple million. So, he's fine with that. Yes.

So, the thing is regarding, let's say, the monetary values, actually the biggest problem is not even, let's say, selling it as a solution to someone else. It's more like protecting your own systems from increasing the costs as the web is growing, and you're essentially forced to crawl all kinds of contents and then evaluate and process them to save money, to not evaluate AI-produced stuff that you essentially don't want to allow to go into the indexes. That's a like massive money-saving solution, or that would be the result of that. Um, and that's what actually I believe all search engines are mostly interested in, because they're kind of like vulnerable and open to spam, essentially. And I also think there's also, you said it like two years ago, the engineers are spammers, or the, the line between white hat and black hat is being erased, because it's not even clear what's going on. Because you really think about it, I think we don't have that real data, but think how many people saw ChatGPT and how well it writes, and they thought, "Oh, I'm going to make my own blog." And they don't even know what is SEO or how they're going to like, you know, traffic. They just want to post these things because they really like these things. And now Google is going to go and crawl all of that, and Microsoft is going to crawl all of that.

So, another part here, um, there is obviously an AI race, AI arms race that is happening right now, and we can see through lots of research papers as well that essentially training AI on existing AI data is not that good of a idea or not good of a solution. Mhm. This is maybe a little bit, let's say, like an opinion, because people actually prove different theories. And for example, DeepSeek was trained fully on AI-generated content, which is a very good model. So, you know, there are, let's say, different perspectives there. But let's say if it, if it stays true that you don't want to use AI-produced content for further training. Okay. Uh, if you don't want, if you don't want to use AI for further training, then it's incredibly helpful for you to, let's say, collect the new training data, then clean it from the AI-generated pieces, and retain only the original human-produced contents. So, that's a huge advantage to, let's say, cleanliness of the data and the actual, let's say, efficiency of training in these models. But as I said, it's a little bit questionable moment because people prove different, different things.

I believe this answer will be creating a loop of problem because if you make AI so good in a way that can write like humans, but then all the AI models after a point will be writing like that, then humans will be recognizing it one more time, then. And the base model will need a better microscope to realize. Let's say today, as humans, we are able to understand maybe every AI word in one sentence. But maybe models will be so good, we will be saying, "This is AI because of just one word in three different pages." And we, we will say that, "No one would use the word 'foster' for example," or I don't know, there were many other things like "enhancing" or, yeah, "del." Yes. So, you will see the word "del" just one time, maybe across 20 pages. We will get so sensitive in the future. We'll say, "Okay, this should be AI. At least a little AI dropped into the document there." But I believe from, from some angles too, from at least the competition between the search engines for this AI detection, first, it's coming from selling this software to the many other places, either for people who want to use it for bad purposes, or also for good purposes, too. The second thing is that imagine that there are two different searches. One is full of AI content. The other one has, let's say, some human involvement in that angle with, in terms of, let's say, visuals, layout, or the sentences on the pages. I believe people would prefer the second one rather than just seeing some automated text. Because the first one, full AI-created content, I wouldn't be able to trust that.

I mean, you are, I will be giving my body to you for, let's say, three hours, and you will be just cutting me and you will be just taking something outside of my body. And I would like you to actually type by sitting in front of the laptop and I want you to just type some stuff rather than just actually clicking ChatGPT. It wouldn't give me any trust to be honest. Then anyone actually can be looking like a dentist or someone else. But would you trust someone like that? I mean, if you want to give your mouth open for two hours and it is just drilling inside, and then he will tell, "Hey, ChatGPT was saying it will shouldn't work." It's, it's risky in my opinion. And that's why I believe Google notes, it was something that they brought first, uh, because they were trying to save themselves from this AI touch as much as possible, giving their voice right to humans. They wanted humans to add notes for Google results, which was something that actually Google never, never gives up to be honest, because earlier they also had two things. One of them was Google Google Site Wiki, the other one was Google Answers. These are from early 2010s. Google Site Wiki was very similar to the Google Notes. Basically, you were opening a sidebar, when I say a sidebar, and you were seeing who has written what about that specific result. And this was early 2010s, and during that time, Matt Cutts still was in Google, and he was also mentioning that authors will have authority. And it was the area that in 2012, we even have a have a patent names actually, topical authority, but just for authors, not websites. And then the Google Answers, it was the forum actually for only the Googlers or Google users. And we also have seen Google Groups, which is still active, actually. So, I believe they never give up from this social network creation purpose. They keep failing on that. But I believe this time, by repurposing Reddit, they just kept it safe. And also they gave a very clear message. If you don't write as a human, we will be bringing human forums to talk them. And even if they give wrong answers, they don't care, they are humans. That's the number one rule. Yeah.

You know, you just said that SER can be split into just AI-generated and something that has human involvement. I believe what we have already is a kind of version or model of that because if you imagine that all factual content already right now is written by AI, then whenever people, and let's say all people in the world are aware of that, it's there's no difference. If you're asking from ChatGPT for some kind of answers, or if you're going to go and find these results on Google, it's both were written by essentially the same super-smart AI. Then what would be the actual unique kind of angle or something that human would want to reach or find? It's exactly communication with other people and finding what other people experience or what they went through, a kind of forum discussion. You know what works, what doesn't, because ultimately that's like the most valuable thing that AI truly cannot produce since it doesn't exist in the real world yet. Wait until robotics come out. Agree.

I believe also, uh, from the AI invasion of the SERs or AI invasion of the web, since these documents in terms of vocabulary, they will be so close to each other after a point, they all will be clustered together by Google. And Google always actually, when they cluster documents based on their vocabulary similarity, they usually choose a representative to represent the entire cluster. Means that all the cluster loses rankings, but their representative increases in that case. And when when it happens, usually the high authority source, which is usually either an e-commerce or either a SaaS or either a very major publisher, they usually go higher in the rankings. And the smaller ones, since they are not able to be unique enough to separate themselves from the cluster, they all just compressed and go down. This is also called sometimes link inversion because if the cluster goes bigger, usually the representative even goes higher in the rankings because it is first cluster to cluster comparison. That's why when usually we create a topical map, we also calculate this, and we try to cover everything based on different clusters and their average vocabulary and differences too. But I guess the originality again comes in handy. For as a small publisher or small brand, you will need to provide originality earlier to take the attention of Google. If you create this very similar, one more content that is already existing on the SER by using the same AI system with the same questions, like, "What are the symptoms of cancer?" It won't work that much, actually, I can say. I mean, they already have documents to rank for that. You are just one another page on the web. Yeah, exactly.

By the way, regarding the, um, vocabulary or the dictionary that's being used, I saw a very interesting research paper which was from Google, and they study essentially or they prove that when people have a kind of conversation with ChatGPT, and there's a real desire to accomplish something, a lot of people start to adapt and repeat the vocabulary of the AI. Remember something you shared about the Markovian state in the search engines? It's like a, it's like a highly similar situation. So, the point is, a lot of people would they write content manually, but they use AI to format it or edit or tweak it, and that can introduce certain vocabulary features as people rewrite it or, let's say, adjust further. You know, there's really there can be kind of like blurriness or a merging element between human brain terms of diction. It's actually a kind of loop because people search in the way that they read, and in the way that they read, they also write, and the way that they write, they also again search one more time. Yeah. So, it changes the entire loop in that way. I was calling a kind of bit loop, I can say, because they change your vocabulary as well, and your vocabulary change affects your queries, your queries affect again ranking documents. They also affect how you write one more time. It is always going like that.

Another thing is, I believe AI Overviews will be forcing people to also be more original and included by humans because maybe writing today, writing more is easier, but the main problem is that everyone is writing more, which means the quality thresholds are higher than ever, relevance thresholds are higher than ever. Which is, even if you use AI, you'll be spending more time. The other day I was on Twitter, and I have seen somebody was saying that, "Everything is automated today, but I am working 15 hours for doing less." And I mean, it's not, it didn't help anyone to read since everyone is using it right now. So, everything is automated, but everything takes more time right now. And it also, another, let's say, paradox that we are in right now, I can say.

Yeah, and I think it promises, like, as AI improves, there is almost like a promise for more usage for it. A lot of, uh, these CEOs of LLM production companies mentioned that essentially, as the product becomes better, there's more usage for it. And essentially, you know, it burns more power, and it's like, whole shifts the way society operates with these things. So, what to think about this, uh, for AI Overviews? Sometimes I am doing this personally, for example, if you go to the SER right now, open your VPN and choose the United States or change your Google settings and do this right now while listening. Go Google and United States. Just search for this: "How does a hosting work?" Then, "How does a hosting function?" Then, "What does a hosting do?" If you search these three things, they actually mean the same thing technically, but they will be giving different AI Overviews for you because of just vocabulary differences. If you also start to add adverbs and adjectives, like, let's say, "What does essentially a hosting do?" Even these will be changing your answers in the AI Overviews and also their linked documents in that area. So, in terms of originality, would you suggest people to, because when we create content briefs to construct a semantic content network, we usually try to cover all the vocabulary, all the entities, all the attributes with a proper mathematical calculation to understand the relevance ties between all these concepts or sequences or paths. So, would you suggest someone to actually include all these synonymizations or synonym groups in their answers? But if they do that, they might again use AI, something like in the first sentence, "Explain how does it work?" When it comes to the function part, for example, as you say that a hosting works through XYZ component. In the second sentence, let's say you say, "A hosting functions thanks to X and D." Would you like to change the attribute connections based on predicate slightly? The first one says, let's say, the main component. Second one might be saying, maybe essential components. Or do you try to measure any co-occurrence distance? For example, if I say "work" and three sentences later, I say "function," then maybe AI Overview doesn't choose me when it should be choosing me. Maybe they should be closer to each other because there are three types of distance on a page: pixel distance, letter distance, and byte or HTML distance. So, how close I should be keeping them to each other? Do you think it is possible to train humans to think about all these possibilities, or is it possible to create this level of detailed content briefs to train them and to work on that? Because for AI Overviews, it is certain that vocabulary richness and length of the question really is important because for AI Overviews, people ask very long-form questions.

We, I actually explained this like two and a half years ago. There was a subreddit called "Explain It To Me Like I'm Five." Both Google and Meta, they were using subreddit LF5 to train their language models to understand how humans talk and to answer long-form questions. And Google actually made a move and they directly bind Reddit to themselves. And they even use a firehose to actually beat their algorithm with all the Reddit threads and the sentences there. So, my question here is that, how would you rank for AI Overviews with this rich possibility of predicates, adjectives, adverbs, nouns as well? What type of content briefs would you create, and would you use AI for it for providing this level of coverage? If you use AI, how you are also going to make it original as well?

So, regarding AI Overviews, something that I think is very, very clear that if you have long-tail queries or really, really long requests or questions, if you have closer similarity matching your answers towards that, that's something that's much easier to prioritize for all of these AI summarizers because there is essentially less work to do, less different sources to pull from to try to merge them to match that type of, to create that kind of like question-answer pair. But this is really like an annoying thing to try to optimize or, let's say, perfect because the amount of these long-tail queries can be like near infinite. And now if you think we're going to have to somehow match or integrate all of that into a single document, the single document is going to become like extremely long and massive. And that becomes a problem, at least in my understanding, that it's just way too much, and most of that document is not even going to be used. So, I do think, um, let me ask this as a parenthesis. I use this actually as an, as an opportunity, maybe people also can take the advantage of it. I sometimes use actually a section, imagine I create a specific section document, sometimes at the bottom, I add some sections only for Google extended or AI Overview user agents, and I use "data-nosnippet," which means that section won't be indexed, but only for AI, it is actually actually communicating. Do you think that this type of a trick actually would it work in the long term?

I really like this part, and there's a very interesting thing related to exactly this moment. I saw in different, in two different research papers which were testing, I believe, AI Overviews and definitely Perplexity and definitely ChatGPT search function. What they found there, even though, let's say, it's not AI Overviews, it's still same functionality, um, what they found there is that if you add certain additional, let's say, information into things like footers, it tends to be picked up by the actual summarizer. So, it kind of seems like your main content works as, let's say, the standard ranking element. Yeah. With that, as a bonus, comes the footer that can contain extra information in it, which then is still picked up, you know, by these summarizers, AI summarizers. And an interesting thing there is that, um, what they mentioned is essentially you don't even need to rank, let's say, at the top of page one. If you're just considered as one of these candidates, that's already good. And what's really crazy is that all of, uh, transformer-based models, there's a like a weird thing going on where the last bits that go into the context window or the like this context-aware, aware window, the last bit becomes the most important one. That's what gets assigned most attention and the most weight. So, it's almost like if you can barely get to the bottom of page one and inflate your footer, that's most likely what's going to produce the most amount of attention throughout these AI summarizers and like things like Perplexity and stuff like that.

There are two things there actually in my methodology in my framework. Usually, I use the last sentence to bind it to the next heading. That's why it always works like a smooth anchor between the previous part and the next part. So, that actually context can flow with a connection from previous to the next one. The second thing is actually in today's system, there are three types of or three layers of ranking. One of them is the regular web document ranking. We have passage ranking, which usually uses, used actually for features in the past or also people ask us questions. And today, we also have what I call as the slightly context ranking, but we can also call it actually as people say RA, or we can also call it maybe like say generated passage ranking in this time, like unification of these first two. So, when you're also right, when if you do that, your footer section very long, basically you will be first, you will be breaking the contextual vector. You also decreasing, you will be entirely imbalance or you will be breaking the balance between the contextual coverage and the flow as well. So, basically imagine you have a very big butt at the bottom of the page, which actually makes it harder to walk, even. In this case, yes, it would actually decrease your document ranking while increasing your rankings for the third part of the ranking, which is generated passage ranking. But in my system, I basically use data-nosnippet for this part, which is irrelevant to the web ranking, but still relevant to the AI or generated passage ranking in that angle. So, there are other tricks, uh, too, that I'm using sometimes on that angle. But I believe, uh, these three, I mean, finding a balance between these three, document ranking, passage ranking, and generated passage ranking, I believe this is directly the semantic SEO mastery today that we have, and I love it, to be honest. Because one good thing is, taking answers from AI Overview, it works fast. It's not like actually document ranking. Document ranking takes really long time, and you need to win coral place. But, uh, generated passage ranking, it works actually way faster as a web answer. The one major trick there is many people still in the SEO industry is said that nobody actually said it yet, but.

Most of the AI overview and feature snippet systems in Google, they are coming from Steven the Baker and Sawasan Wankatahari. Sasan Watahari directly is the name behind Jin 2.0 zero, and he is directly the person who patented and invented the generative summaries in search patent. Most of my algorithmic authorship rules actually coming from these two, and some beyond names as well, based on my experience too.

So, one major trick there is the three grams. Steven the Baker directly states that if the coverage between the generated or processed query for the audience is different than the keyword-processed query, if there is a three-gram coverage between the processed query and the passage, it increases the satisfaction in a really good way. That's why usually when I substructure a document, I try to go actually like with the three grams mainly before substructuring my questions, answer passages, and sections too.

And this slightly brings me one more thing. When we focus on originality, many people go with actually text, like, "How can I make this text more unique?" But how about the HTML structure, or the centerpiece annotation at the top, or how about the layout that you have? Can large language models also understand your layout component by component, for instance? What do you think about it?

Yeah, I think it all plays a role. You introduced the concept of, let's say, the cost of retrieval, and then the actual sort of the key zones that allow them to predict the quality or predict a certain metric for each document. Predictive ranking.

Yeah, exactly. Predictive information field. I was trying to remember that. Um, I think there could be lots of different things that can go into that specific algorithm, and it's not just about, you know, stuffing like more words or explaining more different things. They really can look at lots of different components. And I do think this can, in some weird ways, I just have, let's say, an feeling or an intuition that the way you structure your documents, the way you design them, that can be a reflection of on these again, human effort type of measurements. If they are able to detect, let's say, some criteria for human effort, plus they can see that everything is quality in a predictable way. Sure, why not?

By the way, related, I also seen this this year. Google has so many research papers where they essentially have a certain algorithm in place that's just calculating, calculating different types of, let's say, relevant scores or relatedness of different things and stuff like that. They take it out, they get the like base LLM, they put it in, and they try to see the performance. And they realize that for lots of these algorithms, just a base functioned LLM works way better. Then they use a kind of, I think it's called an encoding system, which is a essentially a part of these LLMs, which then get fine-tuned, and that works super fast with way higher accuracy. So if they can do that to understand the relevance, or let's say relatedness of certain things, and actually use it for the rankings themselves to extract, let's say, certain different more relevant documents, um, why wouldn't they do it for some kind of visual interpretation instead of, you know, loading the whole JavaScript and everything else? We see that these models are now multimodal, so they can see the image, they can analyze, evaluate it. So I believe there, there's it's very clear that they're trying to adapt and improve as many of these systems as they can. Uh, it is also clear that, um, it cannot replace every single possible, let's say, problem or micro-algorithm that are in place right now, but, um, well, lots of them can be improved, and I believe we'll see that in the next core updates.

Okay, that's nice to know. So I believe this also coming to the concept of query responsiveness. Like, you might be relevant, or you might be even more relevant, but are you also responsive to the actions that the user will be taking on the page? So that's why actually centerpiece annotation becomes important, because that actually means that it is the biggest HTML component at the above the fold area that explains the purpose of the page or what action you can actually perform there.

But this also brings me to the originality one more time. Let's say there are two different pages for credit card application. One of them has a slider. This is the H1, paragraph, slider to show maybe, I don't know, your budget or amount to spend. Then we have full name, then phone number, then let's say apply button. Then we have maybe some tabs for different types of credit cards: credit cards for teenagers or, I don't know, teachers, veterans, etc. And at the most bottom part, I'm just going, but let's say in the most bottom part, we have some FAQ, more related question sections to explain, let's say, how to apply, necessities, legal size.

The second document, let's say we don't have the slider. Let's assume this time we have checkboxes for explaining actually either you are employed or not, or you are, let's say, gender is not important for that, but let's say you are employed or not, your average age, you are married or not, let's say, and some other checkboxes. Then I have a dropdown, maybe to show actually your annual average income. Then I have the apply button, and the rest is the same. So when I have such a, let's say, slight difference in my centerpiece annotation at the... Would you just, just from your brain as a reflex, which one do you think is more relevant? That's the query relevance. Which one do you think is more responsive to perform specific actions? Basic query responsiveness. Third one is, which one is more original, do you think, and why?

The one that is more relevant would be the one that had didn't have the slider. I believe, uh, the one that is more responsive would be the one with the slider, the bigger one. Uh, and what was the original originality, right? Yes. That's most likely also going to be the one with the slider, because I, I believe you think that slider will be less common. So being less common increases the originality, or another factor. I just think that if we see, if we don't think of word content as just written text, but we can see it as everything that is visible on the page, including the things that you can interact with, and that slider can be, let's say, a part of the content. So in this case, um, just talking specifically about the slider, uh, if we go a little bit deeper into that, it allows you to like fine-tune and tweak and find like your specific type of answer that is perfectly adapted for yourself. So, um, I think, let's say from this perspective, this page is more accustomed to serving more interests of different user groups, which is why even, let's say, through the behavior of users, most likely it's just more interesting or more interactive, or it's going to provide a better type of answer.

Let me ask, like this, one more time. I like creative questions this way. So we got your answers for relevance, responsiveness, and originality. Three main criteria, let's say we have, and these are the three main criteria for ranking. And we also have three types of ranking, like web document ranking, passage ranking, and also the generated passage ranking. So we, we get this part, but I will ask one thing. Do you think that Google actually uses some of these HTML components like checkboxes or sliders to also verbalize them with LLMs? For example, in a slider, if you have, if I have numbers between 5,000 and 100,000, do they verbalize it like, "I earn five, six, seven, eight until the 100,000k" to the end of the slider to get all the relevance for all these possible numbers by using HTML or by using the checkboxes? What do you think about it?

I think it is possible. But if I refer back to the things you've been mentioning, if a search engine can look for things specifically, the actions that can be taken on the page, let's say for just have text that's like, you know, read, learn, understand, but if you also have things like download, or let's say, um, study, or examine, or then let's say you have a kind of submit a form, things like that, I do think they can count the amount of actions per page. And where would these actions come from? It is, it has to come from a kind of interactive elements or things that are added into the page, which is again, is can be seen purely as a part of the content. And let's say to actually transform them into sentences, maybe not sentences, but could be again, like trigrams or just to kind of like variations for what can exist here, that could be possible.

Okay. So not directly maybe verbalizations, but maybe some annotational integration there can be happening, like a range between five and 100k, okay? For the tables, I know that they are verbalizing it. They have a language model even for that. That's why I usually go with tables for some segments. So let's change the industry. I'll be asking it for e-commerce this time. Let's say I have a, see, two different category pages. I take my like this. I need more space. Two different category pages I have here. And let's say the subject is CBD. Let's assume one of the most monetizable industries. And I have different types of CBD oils here and different CBD oils here. One of them has two different filters. Let's assume one of them is the site filter with less popular filters, other one also, the second filter is the, uh, horizontal with more popular ones. I have an H1 again, fit the same way. I have a paragraph, and I am using a "read more" button after the first sentence to not push the products further down. Popular filters here, less popular ones are here. And this is the product grid. And I am giving three products per row. And I'm showing only, let's say, 12 products in total in the first page of the category.

And in the second one, only difference is I just don't have these popular ones. I have only one thing here. All of them are here. And the popular ones, popular ones are at the top. Less popular filters are at the bottom. So I am not giving a popular secondary filter in a horizontal way. They are all vertical in one place. This is the grid again with the same level of product, product numbers. The only difference is just there is no horizontal filters here, and the products go higher on the page. So now I am asking this, I have the same amount of words. I have the same amount of tokens. Even the order of the words are the same. There is only one minor difference there. And may I ask again, which one do you think is more original, more responsive, and more relevant to the query?

The one that would be shorter, I assume. Can be more shorter means one, one filter right there, one filter. Is it because it takes the products higher on the page? That's what I assume. Okay. The for responsiveness, yes. Originality depends on the document, document statistics actually, because we don't know what other documents actually have there. And usually the originality will be coming from being uncommon, but it's not also about the layout, of course, it's also about what type of filters you are putting, uh, in that area too. In terms of relevance, uh, it might, it depends on how you structure these options, to be honest. Like, uh, if some of them, because it's about how the query will be processed. People when they search for CBD oil, Google actually first augments the query. That's the first step of query processing. CBD oil will be something like CBD oil, CBD oils, and then a long version of CBD. Then they will distribute a mathematical calculation for all these possibilities. Then they will be generating questions from all these possibilities. Then they will be using a verbalization for all these possibilities, like buy, understand, compare, examine, etc. And then you will need to structure these filters in a proper way, either by making them internal links, or either by making them like table of content by using hash values on the same page. So it depends on the query network, but technically for responsiveness, definitely you are right. If you take the main component higher, that's actually always better, because if you delay your answer or if you delay the engagement, it will decreasing, let's say, the responsiveness directly.

Shall I ask one more? If you want to bring some? Go ahead. Let me think. Tell me an industry. We can do it random actually for this main three main criteria. Let's do some dog shelter. Dog shelter. Do we sell the dog shelter, or are we the dog shelter? We are the dog shelter. We allow the pet owners to leave their dogs. Yeah. Forever or like? Of course not. When they go on vacation, they... Okay, keep asking. I will be assassinating them so that's so for such a thing. Okay, it depends on how we will be converting a little bit. But if the query is "dog shelter near me" or "dog shelter in X location," we have one location, right? Okay, so in this case, I will use an exact match domain definitely, whatever location I am in. And I will be opening multiple other exact match domains. And then in terms of these three main criteria, I believe again, I will be using a kinds of simple context paragraph, but I will be hiding the rest. It will be like "read more" directly. H1, context paragraph. I will be having a very short, let's say, form element to without increasing actually its height, one more time. And then I will be adding some certain type of one, one more time, certain type of, let's say, lexical variations of my query, because the phrase "dog" can be directly turned into the also "pet shelter," "dog shelter," and I can also add things like "big dog," "old dog," or "small dog," and dog breeds as well, according to mathematical structure that I can have there. I will try to distribute these things into the tabs, because when the scroll down section, if I put it to the further down, relevance will be lower. If they are all actually important, I'll usually, I usually use tabs to make them actually relevant to the specific part. After that, between these steps, again, I'll be putting these "call us" type of, let's say, the CTAs under to the or bottom of the every content that I have. And further down, I'll be going to do, let's say, a bit more engaging section, maybe to have some basically reviews, because I'm sure that they were searching also for the reviews directly there. And again, we're using tabs even for the reviews too, like reviews for food, reviews for, let's say, the shelter's comfort, or reviews for my experts or health section. And I would also try to create a topical map overall for dog health, dog diet, and I will be creating affiliates to the other industries so that I can also get back links from them, and I can also repurpose the site for making more money. One more time, uh, I believe also I will create some other donation pages too. I believe they will people will doing that. But in terms of responsiveness, there, having the form element and CTAs in every part will be one of the main things. I'll be getting all the synonyms of, let's say, I don't know what predicate will be perfect there, like "leave dog to the shelter" or "register dog to shelter," I don't know, "apply for dog shelter," whatever the predicates I have, whatever the dog breeds I have, or other attributes like size, color, etc., and whatever a dog needs to do, like barking or, let's say, the coat health, you know, I don't know that they need to be, you get my my point, I believe, yeah, all these predicates that needs to be done to a dog, on all the predicates a dog needs to do, I will need to be opening all these triple sections on that page with a proper layout and proper mathematical distribution. Then it means that you will be also original and responsive and relevant to the all angles if there is enough level query research demand for some some of these contexts. I will be opening basically an extra page and internal link to that part, I can say.

What about adding the price element? Would you add it? And where?

It depends directly. Let's say price plus tax plus maybe let's say, I don't know, are there countries where you have two currencies at the same time? Or maybe it could be like you pay, you can pay with Bitcoin. Let's say it's local currency plus Bitcoin. Yeah. Yeah. If I am, if I'm able to make the sales directly, or if it is like a dog hotel, let's say, if they are able to directly book the space for the dog in my place, like a hotel, in a hotel website, I would actually directly add the, let's say, the payment options for them. Because I believe they wouldn't put the small and big into the same section in the shelter. So I will be having some different navigational conversion points for every one of these places. And I believe they can directly make the payment there. So instead of the form, they can also have directly a payment, like choose a date, for example. We can turn the form into a date selector, and then it will be like, according to dog size, since it will be print, they will choose the type of the dog, like small, big. I guess the big one would cost more, takes more space. I don't know the industry, I just make guesses. And then the days, the size of the dog, and maybe aggressiveness of the dog, or diseases of the dog that you will not tell. All these things whenever you refine your sections, I will be increasing or decreasing the price accordingly. But this is a little bit about how we will be converting, because if I ask too many questions from the conversion page, they might not convert. Usually, it is better to keep it short to convert faster. So if it doesn't affect my costs on my end that much, I wouldn't ask these questions. But I would include these angles still in my informational or commercial midsection rather than the most top part, most top part. Even if I, even if I include them, I would add them with info icons. When you hover over or when you click, you can see the info. If you don't hover over, you don't see it, because I also need to convert them. I also need to make sales. If I add too many checkboxes, too many input areas, text areas, sliders, they will be confused, and they would be, they might not buy it. So it's better to actually, if somebody thinks about details, they can scroll down and handle the details there. Or I can even use two different CTAs, like "fill the form" or "just buy directly." Whichever converts better, I can bring one of them opened. If they want, they can click the other one and they can fill the form, and we can reach out them by asking questions as well. So you can basically variate your conversion path and also your, let's say, responsiveness, because if you have a form element in your HTML, if you have checkboxes, and if you have all these different verbalized versions of these conversion possibilities, it will also increase actually your relevance and close the gap between you and multiple other alternative competitors, because I don't think that they will be having all these options with all these angles.

What do you think about this? Just as an example, let's say the same scenario where a dog shelter, our landing page has a form. We know all of our competitors also have a form. We know all of other websites in every other city in the world, they also have a form on that landing page. But then we realize that whenever customers land here, they don't really just want to send their email or a phone number and that's it. They actually first they want to select a date in a kind of calendar, then let's say select size of the dog, and then submit the contacts. So because logically and you know your customer base, you realize they want a calendar together integrated with the form. Would you actually replace the form with a calendar?

I will do that, uh, because there are some also patterns about this. It was about the session based on category, I couldn't say cateru or category based, let's say search session duration. It was a kind of quality signal, because according to your page type, the session duration or click behaviors are changing. It affects how the click model will be run through your site, and they can actually see higher satisfaction, or they can try to test you for these type of, let's say, different behaviors as well. And this is actually something that many affiliate sites that they do as well. They try to imitate a local business in their homepage, so that actually they can avoid from helpful content updates or system, because they have a function, have a local business, like let's say photography studio, but rest of the site is actually about best lenses or best cameras, but the homepage is about "come to our store" or "take a booking" and also convert from there. And again, for credit cards too, the moment that you add a form, you increase your rankings. If you remove the form, you are just an affiliate. From Google's point of view, it's not easy to understand who is the real, who is not just imitating with a form. But technically, yes, I change the page to increase my responsiveness, and also in the long term, when the data is longer, or when when there is more data, it will take less time for Google algorithms to realize that the behaviors and click satisfaction here is different than the others. So you see, it can kind of lead to a kind of web where pages have more, let's say, parts of the funnel integrated into them, or just more interactive elements integrated into them, because if that's what's satisfying for the user, and that is the content, then it's no longer about, you know, but for organizational point of view, you can always actually make your form more understandable for Google algorithms too. You can verbalize it with info icons and extra tooltip text. You can always use a better HTML structure for the dates by using time HTML tag there. Or you can always make your HTML structure more, let's say, with semantic HTML and also less parent elements and less child elements. You can actually have a easier to parse HTML structure there. And from again point of view, you can combine your form with other things. For example, you can also have buttons like for two weeks or for one week or for one weekend. The others might not have these options, because when you search for dog shelter, Google already thinks about all these possible variations for tonight, for tomorrow, for this weekend, for next weekend. I can always being connected to the all these mathematical possibilities, while others are just giving a calendar for you. You can always make it enriched for higher relevance, responsiveness, and originality.

So there's an interesting part here because you can see that, let's say, landing pages variations of the processed query can be addressed through different functions like buttons and certain like tabs, features, sliders. But when it comes to, let's say, educational content or like pure content pages, you were asking about these, let's say, AI summaries and like long-tail terms. If there's is a creative way to try to signal those types of things, that it's it's a completely creative or a random idea, but something that can be in in, uh, so like centerpiece annotations. We see that in Steven the Baker's patterns, and also Google API league, we see other annotations like sentence boundary annotation, like section boundary annotation. It means that for Google, even when actually understanding where a sentence ends and where a sentence starts, even for that they need some certain annotations like punctuations or different signs there. In a similar way, in your educational content too, you can make some sections opinionated by starting your sentences like, "I test," "I think," "I experience." Some other sections can directly be from the angle of the manufacturer, researcher, student, consumer, or politician, or whatever angle you have there. You can actually ask questions like, "What are the latest researches for X?" "What are the latest statements for X?" "What are the latest, let's say, accidents for the products of X?" You can always add consumer angle, manufacturer angle, producer angle, or different type of angles. If you give the angle, it's not about the fact, it's about perspective. So that's why if you add more perspectives with certain type of signatures, we talk about AI signatures, for instance, right? And again, perspectives have different signatures, experience have different signatures, and facts have different signatures. You can use these annotations like sentence boundary annotation. You can also use experience annotations. Things like, "According to me," or "I tested this and I like that." For that, you need to use a more casual language. If you give the specific angles of other people, you will need to give a kind of, uh, transaction from someone else, you are giving someone else's thoughts there. "According to, according to the reports from X," let's say, "this this this is are better," and you can give the opposite opinion in the next angle. Both of them are accurate. It's not about the one major truth. It's about the angles and perspectives. So you can always classify your informational documents into the three different layers, like the facts, like the experience, like the perspectives. If you cover all of them, you will be always relevant. Whatever the consensus is, even if consensus changes 100%. Let's say today, I just will give maybe a random political example. This is not my opinion. I'm saying it because it's a very popular topic. Let's say today, Trump is a very successful president, and this is the consensus. And you can actually explain why he is successful for economics, for, let's say, jobs, etc. And in the second part of the document, you can give the opposite opinions and perspectives too. And if the consensus changes, Trump is successful to the Trump failed, let's say, if the consensus changes one day, still a part of your document is accurate, still you are relevant to that. So it is a kind of bulletproof document mindset with all of them. That's the angle in the framework a little bit.

It's funny because in a way, if you're trying to cover all possible perspectives, you're not really saying anything. You're just saying everything. But then you're always relevant and safe.

That's true. That's why I call it safe answers. You, you basically give every angle because sometimes I ask a question brief, and the author tells me that, yeah, but there is no, there is, let's say, there is no symptom of that, there is no evidence that it shows that. Then they say it, you don't have to say always yes in the answer. The answer doesn't have the positive. We just ask it. You just give the answer. It doesn't have to say, yeah, it works in that way. You can say, no, it doesn't work in that way. In the contrary, there is no evidence that shows XYZ. You can always actually make it negative as well.

Okay, if you want, I can share my screen and show your presentation a little bit to for people so that actually they can also see these valuable cornerstone resources. What is this, for example?

Uh, this is one of the really interesting, let's say, visual examples from one of early research papers that tried to separate human style of writing versus AI style of writing. And so on the left, in red, you can see the actual statistical probabilities. I'll explain what it is, but this is what belongs to AI. On the right side, this is a kind of statistical pattern that we can see that belongs to the human. What happens there is essentially they try to focus on a single word at the time. They try to understand the probability of this word appearing in combination with all previous words, or let's say the surrounding territory that captures this specific word. And because of the nature of AI, it is really heavily dependent on the previous tokens that have been mentioned or even the one that it has produced. It is really becoming highly predictable in terms of what it is likely to choose next. So in a way, it's kind of like, um, it's fairly easy to predict what AI will say if it already started to speak, because it will just continue in the same, yeah.

That's I believe that's why this is not fluctuating like this one, because humans are more creative. Yeah. So regarding to this picture over here, if you take, if you can zoom in a little closer, uh, if you take any of these points on the line and we take the middle one, because that's, let's say, the one that we focus on, that is a word where we put our focus, and then over here, if we start to do our analysis and let's say we go backwards or we go forward, when it comes to AI writing, it pulls this probability towards the exact same direction. You can see the line goes down. It points towards the same, let's say, territory, the same direction, because AI chooses, um, let's say, the words that are really likely to be chosen according to this AI-based statistics. On the right side, you have random directions. It goes up or down because people tend to choose more, let's say, creative words or more unpredictable words, and sometimes people also do mistakes, which is why this line, we can see it goes in a random direction, either up or down. So statistically, it's a very big difference. It's a really nice visualization of actually AI predictability and human randomness, because I believe every human has a different mind, and every mind of a human is a different universe. It's not the same, but AI is not like that. This is one universe, and this is like endless. Every human talks different than any other one. It's not like a model.

Not exactly. I can understand that. Let me continue a little bit. There are really good interesting things in your presentation. I already know that that comes slightly back. I believe people will definitely enjoy this, and some audience will tell, "Why didn't you show this earlier?" You keep talking, and this one, you talk about plummeted to near zero. Do you remember the meaning of this slide or the...?

This is this is the part where we I show actual snippet from the research papers that tries to break this detection system, and it just shows that if you do certain types of prompting or configuration, it is able to distort these statistics or break them significantly, that it's not possible to identify these clear statistical patterns anymore.

Okay, makes sense. Continue a little bit. Since your speech is close to two hours, I am going like five, five minutes here. Ghost Buster: Detecting Text Ghostwritten by Large Language Models. And I believe this is a research from Berkeley University, it looks like. Yeah. And what's the main message of this research? Do you remember?

Uh, I don't remember fully, but I believe this is, yeah, I, I remember this is the machine learning procedures. This is actually, um, all kinds of AI detectors that you can find that are commercially available, like, you know, if you go and search for software that allows you to classify or detect AI content, they are all working on this type of principle. And the Ghost Buster paper, it's one of the earlier ones, and it's also really highly often, like, re-cited and referenced, and this is why...

By the way, the whole process of how they do it, you can see that they're essentially collecting different types of patterns, things like unigrams and trigrams from the human-written text and from the AI-produced text to see these trends or different patterns of how AI tends to express itself versus humans. And they do it on a specific for different, let's say, understand. So they extract unigrams, trigrams, and I guess they process the text with the GPT-3. They run different vectorization functions to embed the text based on these these, let's say, units, semantic units. And they also run different functions, then based on an aggregation, I guess they run a kinds of regression classifier here, like a kind of average, I guess. So they don't use one method here, they use multiple methods and multiple units to classify them. True. Yeah. Yes, there's lots of different, what they call features, that can be used for, let's say, separating human content versus AI-generated content, but this is just a small sample that they show over here. But also this will be an evergreen method. The moment that I have a new GPT or new AI model, maybe they can run the same, same functions, same scalar or vectorization functions one more time to re-train. But I believe it will be different next time, you know, because I will be changing it more, and they will need different, maybe not just unigram, trigram, but some other things too.

And let me ask, why do you think that they go again with three G? Why not four G? Why not B G?

Well, if you ask me what I think, then I think there are multiple research papers that try like two grams, four grams, five grams, seven grams, and so on and so forth, average. And they just in general show, I, I don't know exactly why, but it is clear that on average, it's just much more efficient to use three grams because they are not as expensive, maybe, to process with, but they just give you enough information to kind of work with it. It just seems like the most efficient one.

Yeah, these ones are very interesting. So they check the most probable or efficient path for of the seconds in the model, and then they create a kind of coordinate here to calculate it earlier and faster. Do you think the the image on the bottom right on this right part over here, this is it shows a kind of like statistical probability of these words whenever they go in a sequence, and they call them something like a valley of token distribution. Essentially, if you give AI a task and it's going to write a certain text, let's say it's going to write like 50 variations of this text, you'll realize that the word choice becomes very, very, very narrow. If you let's say try to overlap all of these 50 writings. Uh, so that's where these red lines show that the valley of, let's say, word choices or token choices is really limited, and it's very predictable. That's a big vulnerability. But, uh, when it comes to, let's say, uh, human writing, you will see that this valley goes like highly up and down, because again, the word choice and word distribution becomes chaotic and highly unpredictable.

Really nice. And there are some concepts here like type token ratio. What does it mean TTR?

These are additional, uh, features or functions that they can be using as, let's say, bits of training data for machine learning classifiers. Over here, I was showing that, um, they really can take absolutely anything that you can possibly identify within the text. They can count, let's say, the nouns that humans tend to use versus what AI tends to use. They can count, let's say, technical words or quotes or punctuation. They can even count, let's say, the distances of, um, the not not the distance, but the length of the sentence, including the amount of words that appear there. They can check the size of paragraphs, the kind of punctuations you use. Uh, again, named entities, these different types of things.

Okay. And how would you, sorry, how would you describe textual lexical diversity, for instance?

So in my understanding over here, this is in a simple way, a kind of, um, richness of vocabulary, or just richness of expressions, maybe in a very simple way, whether, let's say, you have, um, not necessarily synonyms, but terms that tend to re-explain the same concepts from slightly different angles, which allows you to enrich the vocabulary, expression. And again, when it comes to AI, it will just tend to be very, very narrow in its expression overall.

Okay. Okay. Let me continue a little bit. This learned patterns are blurry and unclear. What does this section mean? I understand the uppercase, but this part.

Yeah. So if they not if, but when they do train these types of detectors based on machine learning, it is really hard to know what exactly the machine learning training has picked up, what kind of patterns it has identified, because it will live essentially within the, let's say, quote unquote neurons of the trained model, not the engineers, not the people from, let's say, like a third party or a third side. Nobody will actually know precisely what it is using to claim if it is AI written or not. So, um, that's why I say it is like unclear or blurry, exactly how precisely they're going to work.

Okay. And this part is actually, this is exactly what you mentioned. Uh, this is one of the biggest problems with this method is that it gets outdated with newer models, also whenever models are being tweaked or tuned, and then later on, we just show that there's actually way more problems that can be identified over there. About these GPUs are biased against non-native English writers. What does it mean?

There is, this is a very, very loud research paper where people, there's a lot of, I think it came from professors that realized they have non-native English students that write essays, then the lecturer or the professors, they check this essay whether it was AI generated or not, and these types of detectors say it was AI generated. So there is a lot of like drama and problems there. And then what happened is that some of the students, they started to take screenshots for every, like, three or five minutes as they were writing the essay, just to prove that they actually wrote the whole thing. So they would present like 300 screenshots proving that they have done it, and that came, that brought a realization that these AI detectors are not really accurate, because in a case of non-native English speakers, like myself, the vocabulary in itself is not that rich. So if you try to, let's say, explain something complex, you wouldn't really be able to be very, you know, diverse and rich in the way you explain yourself and the kind of like unique words that you're able to use. So that's a big problem because falsely, these types of detectors, they accuse people that are not native in English to be producing AI text. And in that paper, uh, they actually reference that, well, this is related to essays, but what about people who, for example, have their own blogs or their own websites that are also non-English speakers? Are they also going to get into trouble because of these types of detectors? And they argue there that essentially, it's important to preserve perspective of all like cultures and languages. That's why it's not a safe solution to use these types of detection mechanisms, essentially they just produce false positives.

Okay, about these sections, I believe you give some originality originality tips to the people. Yeah, different types of things that help you, let's say, tweak or in your content to be more efficient. Do you think that making sentences shorter or breaking longer sentences into shorter units helps for originality?

So this was actually this was taken from one of the research papers that tries to reverse engineer all of these things, and they've proven that it is a working solution. Uh, another thing there is that it does seem if you do not really do any kind of like heavy prompting and you just ask AI to produce you a certain type of writing, it really tends to stuff multiple meanings or multiple messages into a single sentence, and that's a kind of very common pattern across all LLMs. So that's why that's there.

Okay, let me ask this. Let's say we do everything that research papers do, but we see that tools, famous tools about originality, they tell that actually it became more like AI now. Which one would you believe then, to the tools or to the researchers?

I wouldn't believe any tools except maybe a few, depending on our goal or task, what we're trying to do. Um, if we're trying to produce the content, it's a different story. If we're trying to, let's say, hire a person and verify if they're working manually or or if they're relying on AI to produce the content for us, there could be like different types of solutions here. But overall, there is really no, I cannot suggest to rely on any kind like tools to rely on originality checking.

Okay, your actually presentation is so rich. We might need to even actually run through this actually later, maybe like an extension interview for that. Uh, I can say for now, I will be showing a few more slides a little bit. For example, watermarking, changing the output of the model. How would you describe it or how an SEO or content writer should use it?

Do you think this is a really tricky one? But the point is that certain companies, Mhm, make a decision to add an additional configuration to the way they produce writing or to the way they produce text. Over here, you can see they can have a certain key, and in one of the next slides, you will see that this key will affect different scores. Essentially, what this means, they are slightly controlling the word sequence that is presented to you whenever you're writing content with AI, and what that does is, if they want to detect whether the content was AI generated, if a watermark was applied at the generation time, it becomes extremely easy to be certain and essentially identify these types of generations. On this slide, I show that the statistical pattern becomes extremely obvious, and essentially it's like, let's say, at the time of that conference, it was very clear that it's one of the best and very efficient methods. But what I've seen slightly like, uh, just recently, maybe like a week ago or something like that, um, Google has a new research paper on that concept that shows there are actually weirdly unpredictable problems. So it is not as perfect as they anticipated, because new problems evolved essentially.

What is this Synth ID? Is it a tool? Is the method name?

Synth ID is Google's watermarking technology. So that's something that they would use inside all of the Gemini models. This is an example of how it works. On the right, you can see the watermarked scorings there. So, um, this, the PLM B, basically shows you the standard scoring for all of these tokens on the left, like Luchi, Mango, Papaya, Durian. I guess a pet by language model does it mean or something else? I don't really remember what it means, but the whole, whole working principle here is that the left, the scoring on the left, it shows a standard scoring distribution that would be used sort of naturally, but whenever you apply watermark, that distribution, that word choice shifts or changes slightly. And something that you can see over here, uh, if we configure LLM to just choose the best possible token, you'll see that the best possible token didn't change. It will be mango. But that's a very, very tricky nuance, because essentially that's what allows the quality of the outputs to still remain really high quality, but still be really easily identifiable. But there are still nuances.

Okay, Captain Od, thanks for also teaching me as well. It means a lot for me.

It's it's my honor to do that and my pleasure. And there is a very good actually visualization here with no watermark and with watermark differences. Would you like to explain the main difference between them a little bit?

Yeah, sure. So the text that is highlighted in the green, these are the tokens that will be identified as watermarked according to the statistical distribution and the key that watermark, that is attached to the watermark, or let's say the key that determines the watermark. Uh, on the top, you can see this is this text was produced by the LLM. So it is AI written, and it's just a kind of like standard writing. A very interesting part over here, if we add, uh, a third box over here that would be human written, you will realize that the human written text will also have green tokens in it. Which means whenever we try to identify watermarks, every single writing will have some sort of percentage of watermarking signifier element in it. But it's a matter of how much of that percentage or how much of that element there is present within the writing. And, uh, at the lower box, you can see that almost entire text is green, which means basically the entire output got watermarked, and that means, well, it's really easy to identify that, and they will know what this actually is.

Okay. Okay. That's a really interesting point, by the way. It's like there's always a sweet spot. You shouldn't exaggerate any angle, you know, like not overly, don't try to be too human as well. But by the way, remember this part now, uh, earlier we mentioned that whenever people talk with AI, they tend to adapt to the language that AI produces. So if...

Let's say you're writing content and you're using AI to, let's say, edit your content or give you suggestions. You will slowly adapt to the vocabulary that AI uses. And there's actually a paper that shows exactly this, and from Google, that there is a weird problem that appears over here. That once you start to give people watermark content, people start to manually write that watermark content, which breaks the whole purpose. But, um, well, that's one of the challenges that we are facing right now as a humanity. Basically, it makes sense. Uh, let me just move a little bit further.

The problem that you mentioned also, yes, it is, as I said, it's a loop. It's a curse, actually. There, there was a concept that I was using, something like a curse of information retrieval, because sometimes, uh, you increase the relevance, still it creates problems. Then you increase another component, but it also, it creates another problem. So it's always a compromising game. Yeah.

Let me create one more slide. I believe this is your just summary slide. I'm trying to find something more informational, or let me ask this. When you say, "generate the key, share the key," here, uh, what exactly do you mean in this specific, let's say, seconds? Go just a little bit, a few seconds further. It will be a full presentation or a full slide. Yeah, right here. So the point is, for this watermarking thing to exist and or to be used, essentially all of these chain of events have to happen. So you have an AI company, let's say someone like OpenAI or Google with their Gemini model. Uh, they need to generate the key. The key becomes a kind of formula that will mess slightly with the word choices, which then makes it much easier to, let's say, implement that. The point is, there are lots of different keys that exist, and there are things like dynamic keys, or let's say, lots of different variations. You can also have multiple keys at the same time. So there's a whole technology there and like a formula and algorithm that a company has to invent, or at least, let's say, uh, accept to use. Then they actually need to start to use that, uh, key to start watermarking all of their outputs. So that's, let's say, lots of decision-making that the company has to do. And then, uh, in order for the detection to be happening, they also need to actually share that with another company. So, let's say, um, if OpenAI wants to share that with, uh, Google, there has to be a communication or a kind of, let's say, agreement. So this process becomes very, very, let's say, political. And, um, there's another, there's another hidden problem there. So there's, it's like actually giving a product to the customers with a flaw on purpose. There's a problem there that you can leak some of your weights by doing this thing, which is something that a lot of companies don't want to do, because all of these companies essentially are competitors. And why would you show, let's say, the intelligence of your model or how it operates, uh, you know, under the premise of, you know, we're just detecting AI text? So there is a lot of, there's a long story there, and I try to present it in a simple to understand way.

It's one of the greatest speeches and the presentations that I have ever seen in the SEO industry. I must tell, I believe it was the best speech also in SEO Master Summit too. I can't even tell it without seeing the other speeches, uh, to be honest. And I believe you will be creating these level of, let's say, valuable cornerstone like, uh, artwork, like actually presentations. And maybe you doing one of them actually on our stage, uh, in Turkey, I hope. And I believe we were able to finish only the half of it, maybe, uh, maybe even later we can actually finish the other half, just going a little bit faster by creating a resource for the people, uh, by giving some practical suggestions, including practical constraints as well, maybe by showing differences in the text, in the headings, in the even entry text, etc. But I believe you created a great success here for anyone who tries to actually make their content more original, or anyone who tries to get AI in a better way for their own business, for making more money. So for my name, I thank you. It was a great resource. I believe Matt Singer should be feeling really lucky to have you on his stage as well.

And where exactly people actually do you think can find you? And what would you like to give people as a suggestion in terms of this? If you would like to use just one, let's say, one simple suggestion to people, what it will be? And where they can actually find you? Join holistic SEO communities and find the best information here. It's also free to join. So don't, don't be ashamed or don't have any problems with. Thank you, brother. It's always great to see you. I believe you'll be coming to Turkey this September. So we will be, uh, just doing lots of massage, jet ski, and other stuff. I'm looking forward to it. And I believe we definitely need a second interview to finish the presentation because I really want to do that. If you also have time for now, I believe for people's, let's say, uh, their attention threshold, I believe it's already over, over that. And I believe their brain is already melting if they watch this only in one time. But I, I believe just try to parse it and chunk it and watch it like two, three times. You will realize always more details to implement. Thank you one more time and see you the next time.

Thank you very much, Corey. One more thing I want to add, last thing. Uh, we had, uh, an interview or a video with you from like two or three years ago back in Istanbul. At the end of the video, you told me something like, "Well, maybe you can do the research and write like a research study." And I was like, "Yeah, maybe I can do." And the second I said it, I was like, "Oh, like there's so much work." Yeah. So, so now I've done it, and I feel relieved because I kept my word. Nice memory. Memory. I, I remember that moment. It was a really nice, nice day, brother. I believe you remember the kebabs still. Yeah, of course. Kebab, baby. Okay, my friend. I will look forward to meet again, and I hope we will, we will be able to do the second episode too. And the people, if you want to follow PL, he is actually active on LinkedIn, Facebook, and also Twitter too. And he's also a top leader in the holistic community. Just ask or shoot some questions to him. I believe he'll be happy to examine your site or your text or give you some suggestions based on your unique situation as well. But just don't message him every hour. Just sometimes, you'll be answering, I'm sure. And see you in the next time. Thank you very much. See you next time. Thank you.