Transcription
[Music] Welcome to the AI research lab explained. In this series, we're breaking down complex concepts and sharing real-world insights, all from the forefront of research. I'm your co-host, Shelby, and I'm here today with my colleague, Juan Carlos.
Thanks, Shelby. I'm excited to dive into today's episode. As researchers at Salesforce AI research, we're constantly exploring new possibilities for AI and how these breakthroughs can change the way we work and interact with technology.
Right. And the way we work is evolving with AI, but efficiency is just as critical as intelligence. Speed, cost, and easy deployment are key to making AI more effective. In our research, we have seen that one way to achieve this is with small language models. While much of the AI world focuses on large models, our research looks at smaller models that offer unique advantages in speed, cost, and accessibility.
To kick things off, let's start with the most important question. What exactly is a small language model and how do they work? Small language models, which are also known as SLMs, function a lot like large language models. Their primary function is to generate text, just like LLMs. But SLMs are small because they have a significantly reduced number of parameters, which are the weights in their deep neural networks. To put that in perspective, large LLMs have hundreds of billions of parameters. In contrast, SLMs have millions or a few billion parameters. So, SLMs are often a fraction of the size of a typical LLM.
Now, this substantially smaller number of parameters in SLMs brings some differences when comparing small and large language models. We can think about these differences by comparing them across some of the key factors of model deployment. And let's start with inference speed. Inference speed refers to the speed to generate text. An SLM will generally have significantly faster inference speed than an LLM. Compared to an LLM, the SLM simply has fewer operations to perform since it has fewer parameters. Ultimately, this means it generates text more quickly.
Yes, its small number of parameters also means that it is cheaper to serve and maintain compared to an LLM. After all, all these transformer-based AI models, such as SLMs and LLMs, are feasible today due to advances in computation power, such as GPUs. But GPUs do come at a high cost. A typical LLM with hundreds of billions of parameters may require many top-of-the-line GPUs for inference, whereas an SLM may require only a fraction of those GPUs. And in some cases, for very small models, CPU-based computation alone may suffice. Practically speaking, this means that SLMs are generally cheaper to serve. And because they're consuming less hardware, they're also producing less CO2 emissions, so they are more sustainable in the long term.
Great point. The fact that SLMs can run on a few, if any, GPUs also means that SLMs can be deployed on devices with constrained hardware, such as phones, smartwatches, personal laptops, tablets, and more. This is what we refer to as on-device generative AI. And there are so many benefits to deploying a model on small devices.
Yes. And one more benefit of on-device AI is offline access to the model. In contrast, for most third-party LLMs, you need the internet to access the server on which the model is stored. Instead, SLMs are small enough so that they can often fit on a variety of devices. This means no internet is needed to access the model since it's physically stored and running on the device.
Right. And an added benefit with this is privacy. With models stored directly on your device, it means what you pass into the model stays on your device and it is not stored on a third-party server. Similarly, the outputs of the model are also for your eyes only and not stored on a third-party server. This is particularly useful if you want to apply AI to your personal data, such as your personal images, documents, and more. You wouldn't have to worry about a third party seeing your data.
Now, all that sounds great, but none of it matters if the SLM is not accurate, right? At the end of the day, we need the AI model to generate good output. Remember that LLMs are large for a reason. More parameters typically means more learning capacity. In that sense, SLMs with fewer parameters are thought to have a smaller learning capacity. But the good news is that the accuracy of SLMs can be at least as good as LLMs on a focused set of tasks. SLMs, with their small size, should not be considered a jack of all trades. They're more like specialists. So, SLMs are particularly powerful in settings where the tasks are more focused and not too broad.
Exactly. SLMs can be extremely powerful when used in the right settings. This is why here at Salesforce, we've developed several state-of-the-art SLMs. One SLM to check out is our XLAM 1B model, also known as the Tiny Giant. This model is a function-calling model with only 1 billion parameters. It's shown a lot of promise in our research, displaying impressive performance, beating many larger industry-leading models on function-calling benchmarks. We've also built SLMs tailored specifically for developers. Models like Codegen and XGen Code are designed to excel at code generation and developer-focused tasks. For example, Codegen 2.5 matches the performance of models twice its size, all while being far more efficient. Meanwhile, XGen Code is optimized for both code generation and natural language outputs, making it a powerful choice for multi-turn conversations and complex developer workflows. In short, our SLMs are smaller, faster, and built to deliver big results.
Now that you're aware of some of our freely available SLMs, such as XLAM 1B, Codegen, and X-Gen Code, let's discuss some compelling applications of SLMs. The first being, of course, AI agents. Imagine the possibility of personal agents. And by that, I mean agents that can access and learn from your personal data to help you with any task. In these applications, you would want full privacy, no data shared with third parties, making SLMs shine here.
A natural extension to AI agents is robotics. Imagine a robot that has to make decisions quickly and that cannot rely on internet access to take action. This is another case where having a model directly on the robot's hardware would be beneficial, avoiding the dependency and latency of cloud communication. And even if the robot did have a cloud model, we would still benefit from a small model locally that can generate actions quickly.
We hope this video gave you a glimpse into the fast-moving world of small language models. As AI evolves, we're committed to exploring and defining the leading edge of what AI has to offer. If you enjoyed this video, please be sure to like and subscribe and turn on notifications so you never miss out on our AI research content. Thanks, and we'll see you in the next one.