Transcription
I'm so excited to be with you today to provide an update on what we've been building recently and how the next six to twelve months look in terms of innovation. I'd like to start with an update on our platform. The Coveo platform is really a composable AI search and generative Experience Platform that now supports semantic search, AI recommendations, generative answering—which is critically important—and we'll dig a little bit deeper on that in the second part of the presentation. And, of course, unified personalization. We power multiple use cases, as most of you know, across websites, commerce, service, and workplace. So that's our engagement in the app layer. We leverage content from a wide variety of data sources and content sources with our native connectors and our custom connector frameworks. We also leverage behavioral data from various data sources, and we capture a lot on our own.
So we have this connector layer, and we also have this unified index that now does both keyword search and semantic search with the ability to keep vectors or embeddings directly within the index. We have multiple AI models around machine learning, deep learning, and now the ability to deal with generative AI and large language models. We have a whole set of API frameworks and native integrations to allow us to integrate with those various cases at the top. We have analytics, finally, and a whole set of admin tools. If we get into the details, you will see those in the middle—those different AI models around machine learning, deep learning, and generative AI. I want to highlight also those native integrations around Salesforce, SAP, Sitecore, and Adobe Zendesk that we maintain to make those Coveo integrations more natively integrated. We have use case extensions like merchandising in the context of commerce, and then a whole set of ways to build dashboards and administer the platform.
So this is our broad platform, and our large customers will leverage multiple use cases from this platform, multiple components of this platform. But this platform is built on the same core principles. As always, we position ourselves as the intelligence behind. While we can build and use our experiences, most of the time, we are behind the experience from an API perspective or native integration perspective. We aim at high-scale projects with a focus on behavioral data so we can build specialized AI. We do all of this with the utmost respect for privacy and security.
Now, I'd like to move on to the generative AI side and provide an update on what we already announced earlier this summer about our GenAI initiatives. Most of the large organizations that are our customers have either been using Coveo AI or have a form of advanced enterprise search capability inside their organization. So this is Coveo here on the left side that you see, where there's all of this connectivity layer that allows the platform to reach all of the areas of their organization where there's high-value content. We have an index, and we have advanced AI relevance to power great results in the search box, with, of course, different UI frameworks, administration, and analytics.
What we've seen since the beginning of the year is that some areas of those organizations have been experimenting with starting those projects—those question-answering projects—that would involve a separate vector database to power semantic searches and then to ground a prompt and ask a large language model to provide an answer. While this may work in niche applications, it brings multiple issues for the enterprise. The first one is that by doing this, you have different search boxes to go after the same content, basically. So on the left side, that's the search box that people are used to and are familiar with. On the right side, this is a question-answering system that doesn't have the ability to reach all of the content, all of the information of the organization, and that, quite frankly, has little to no administration, no dashboarding, no analytics, limited security, and so on.
So the big deal at the end of the day is that those two different search boxes for the same question will provide different results. That's bad because they're dealing with different sets of facts. So what we have done is consolidate those two systems into a single one. Now, with Coveo, we've got integrated search and question answering. The left side of this slide is still the traditional advanced AI-powered search, but then we've added vectors, as you can see in the middle, within the index so we can do semantic search and basically deal with complex questions. The most relevant excerpts or passages from the result set from the semantic search will then be used to ground a prompt to a large language model. Currently, we're using GPT-3.5 on our Azure infrastructure, and this allows us to keep the data of our customers within the Coveo cloud.
The advantages are obvious from a backend perspective. We have depth, breadth, and freshness of content, security and governance that come with the Coveo platform, administration analytics, and it's optimized for scale and cost. On the front end, there's an obvious advantage to having one unified search box for all the queries and questions, and we think it's now an intent box. There are generated answers on the most relevant paragraphs only if we believe that a generated answer is bringing value. So there's personalization and contextualization that come with the Coveo system. We have all of the capability to link to the source of truth, so there is truth—it's truthful, current, and verifiable lineage.
All of this provides maximum protection against hallucination in the context of large language models. That's key, especially in the enterprise. At this time, what I would do is show you the system in action with real demos. I will start with Customer Zero. Customer Zero is Coveo. It's our own partner community that includes technical documentation, how-to guides, and basically a calendar of events—everything that is useful and informative for our own partners. It starts with a search box here. If I search for "Service Cloud," you can see that I have query suggestions here. It is based on AI that is predictive and provides query completion. If I search for it, I have results here that are ranked by relevance, and I have facets allowing me to slice and dice the content.
So this is the advanced AI search and recommendation paradigm. But let's say I want to ask a question: "How does Coveo determine relevance?" For instance, I have the results set here that came very quickly, but out of those results, the semantic search will surface the most important passages or excerpts that are then used to ground a call to a large language model, which then provides an answer. The answer here is great; it's specific to Coveo, and it's built based on those results at the bottom.
Let's take a step further. Let's type, "How does Coveo leverage permissions?" That's an interesting question because I know that we leverage permissions differently depending on the environment and depending on the product. So that's a great answer—a detailed answer. But then maybe I want to be more specific, and I want to click and understand how this could be leveraged in the context of Salesforce. And there you go; the answer is contextualized in real time with Salesforce. So the answer is about Salesforce here.
That's quite powerful. Obviously, there are multiple attributes that can be carried within a user profile, and those filters can be set at login. We are leveraging a generic large language model. The only thing that we do here is create a prompt on the fly based on the context—what I see, what is used in terms of relevance, and what is filtered. There are multiple ways to look at the source of truth. Here, we've got the citations—these are the key documents that are used to determine the answer. That's part of this document that the system extracts the most important passages from, which are then fed again to the large language model.
Let's try another one here. Let's get a step further and compare feature by feature two modules of Coveo. The sitemap and the web connectors are used by our partners to index web content. Which one should they use and when? If I compare feature by feature the sitemap and web connector, I see prerequisites, content coverage, indexing speed, and the differences between those two different connectors, which is quite powerful and interesting here. I can get into more answer styling, explaining in detail something that is quite advanced—comparing two different UI frameworks, the Coveo headless and atomic, and when to use them.
So you see that it can go pretty much in detail here. And then this one is cool. For you in the audience where maybe English is not your first language, what about querying in another language against English content and getting an answer in your own language? This one here is in French: "Comment fonctionne le pipeline de requêtes?" Basically, it is asking how the query pipeline works. If I search for this, it sends the best excerpts from those results at the bottom, which are in English, by the way, to the large language model, and then the answer is returned and translated automatically into French.
While this is more of a niche use case at this point, it proves the power of the model and of the system and where this can go in the future with instant translation from both the query and the result side. Finally, I'd like to do this last one, which is the best one as far as I'm concerned: "How to create a partner organization?" Oh, I don't have a result here. I don't have an answer. Sorry. Why is that? Oh, because I'm not logged in, and this content is not available for anonymous or public audiences. So let's log in as Art on our partner community and do the same query here. And there you go; I've got a great result here because I'm logged in.
That's a great way to enrich the search experience on our partner community, leveraging all of the information that has already been indexed, the security and permissions that are already in place, the wide and broad variety of content that is in there, and the contextual nature of search. So we're leveraging search and facets to slice and dice content, reorder content, filter content, and then that's what we leverage to create great answers.
Now, I would also like to show a real customer that has recently gone live on their support portal—Xero. Xero serves 3.7 million subscribers, small and medium businesses around the world, and it's all about providing predictive support directly from their support portal. Coveo has been part of the Xero experience for a long time, but recently they added generative answering directly on the portal. I want to show you a few examples of how it looks.
If I start with a complex query like "How does multi-factor authentication work with Xero?" we see that the answer is pretty detailed. Again, it comes from the multiple results that are at the bottom here that are leveraged by search. Let's say, "How do I update my subscription payment details?" Those are real queries that users typically have with Xero. There you go; you've got the answer right away. And maybe a last one: "How do I add a credit to an invoice?" There you go—a detailed, precise, relevant answer. No need to go through multiple documents and come up with your own answer.
So that's live. Xero is doing a great job on their support portal, and we're quite excited about this implementation. Going back to the presentation, I'd like to give a glimpse of our future capabilities and provide a little bit of an outlook on what we think is important as the next step. Answer styling is something that will appear very soon—the ability to ask for broader answers, step-by-step, bullets, and so on. This is coming very soon. In-line citations are something that we will provide, of course, with citation details.
Conversational search is super strategic for us. The search box for us is the universal way for users to interact with information, so we want to make this conversational. With the answers provided by generative answering, we are going to have a follow-up box here directly within the answer so the user can have a dialogue or a conversation with the system. We will also provide suggestions about what to ask, how to ask, and so on. This is coming in early 2024.
Finally, in the context of commerce, we are experimenting with what we call guided discovery, where some of our large customers have a lot of rich content about buying guides, how-tos, and those types of knowledge that typically enrich the commerce experience. This is an experiment that we are doing, and we're expecting that if it creates the value we think, this will become a product in the future.
The example here is for a home improvement store. I'm typing tips to build an outdoor kitchen with a barbecue. The generative answering will look at all these buying guides and how-to documents and surface an answer that is pretty darn good. It provides various tips and various steps that need to be considered, but also we're going to provide links to products directly in there. So it's really a way to connect the generative answering with a product catalog in the context of commerce. Basically, it's a new way to shop, and it's a new way to buy.
We will have top categories that are linked to that answer, and we will have the ability to navigate, slice and dice, and get more into a classic commerce experience where you want to navigate, shop, and discover products by yourself. Connecting those two experiences has huge value for us, and we expect this to move forward in the coming months.
As a summary in closing, I'd like to also say that all of those generative AI experiences need to consider cost. Large language models and semantic search are expensive technologies in terms of computing, so cost matters. Trust is not an option. All of the data and all of the content from a large organization cannot go outside to an external large language model system. There will be domain adaptation, and open-source domain adaptation means that there will be fine-tuning. We believe that those large language models will be adapted in certain verticals, certain industries, or even certain organizations. Therefore, Coveo will be large language model agnostic. Today, we're using Azure-hosted OpenAI GPT-3.5, but down the road, we expect to be able to connect with multiple alternatives.
Finally, in the end, we believe that any system will require relevance across all content and all interactions. Therefore, that's why we built this platform.