Transcription
Hi, my name is Nick. I'm an engineer at LangChain, and welcome to LangSmith. LangSmith is an observability and evaluations platform for LLM applications. Today, we're going to talk about setting up an account in LangSmith. We're then going to use LangSmith with an application, a very simple LLM application that we've built. LangSmith is going to allow us to take a peak under the hood of how our application works, and we're going to do this through a process called tracing, which is what we're going to set up today.
There are a few ways to get started in LangSmith. Uh, I already have LangSmith attached to one of my own Google accounts, so let's go ahead and navigate over to the LangSmith home screen. Cool. We can see there are a few concepts on this home screen, and we're going to talk a lot about each one of these in later videos, but today let's focus on projects. Projects are pretty one-to-one with an actual application that you're working on, and a project, at its core, is just a collection of traces or logs from your application.
The natural next question is then how do we send traces from our application to LangSmith? And the first thing that we need to do there is we need to create a LangSmith API key. So we can navigate to the settings pane and the API Keys page beneath that, and we can go ahead and create a new API key. The personal access token is fine. I created a key earlier, so I'm just going to go ahead and use that.
Now let's pivot over to our IDE. This is just a notebook, a simple Python notebook that actually only contains a single LLM call, so maybe application is a strong word here. And just to note, I'm going to first talk about how to set up tracing when our application is leveraging LangChain or LangGraph. Uh, I'm also going to talk about setting up tracing uh without either of those packages in play. If you're not familiar with LangChain and LangGraph, those are our open-source uh package offerings that make it really easy to orchestrate LLM calls, uh, and LangGraph is specifically geared towards uh, agent systems. If you're using either of these packages, tracing is pretty native with LangSmith, and all you need to do is set three environment variables. The first variable is the API key that we just generated in the LangSmith UI. You also need to set LANGCHAIN_TRACING_V2 equal to true, and then you can optionally define a LangChain project name, and this project name is where all of our traces are going to be sent in LangSmith. I've set these variables already in a .env file, uh, and this is just a nice util that we can run to make sure that tracing is enabled.
From there, our application itself is very simple. We are kind of simulating document retrieval here where first we're going to retrieve a document of facts from a text file that we have locally here. The prompt that we end up giving to our application is we do a little bit of role-playing. Um, essentially we tell the LLM that you are a parrot named Polly. Here are some facts about yourself. The facts are what we retrieve from this text file. If we take a look, it's just some pretty fun information like Polly likes animal crackers but doesn't like goldfish, uh, likes playing soccer, etc. We then uh ask Polly to respond to questions uh about themselves, but we do add one extra requirement that is because Polly is a parrot, they always have to repeat the user's questions back before they respond. Cool. So that's our simple chain. We're using GPT-4 0 mini here, uh, and let's go ahead and run our our application. So we see we get an AI message back. We see that Polly does in fact uh respond first by repeating the user's question, and then there is an impressive bit of reasoning here from GPT-4 that deduces that Polly likes soccer more than basketball because Polly is better at it.
Let's hop back over to LangSmith, and let's go ahead and click into our project. We can see that our project name has now been created; it's called LangSmith onboarding. And if we click into it, we now see this first trace here, and this trace was invoked uh when we ran our application. If we click into the trace, we get a little bit of information here. Uh, the first thing to note is we get some pretty useful telemetry on the actual trace itself, so we can see the whole thing took about 1 and a half seconds, consumed 143 tokens. We also see the exact input and output of our application, uh, and so we can see we took in questions and facts, and then the AI was able to answer with this. And specifically for the chat OpenAI uh run here, we can see the exact system and human prompts that we put together and the returned AI message. Cool. So that was tracing in um LangSmith, but with LangChain and LangGraph. Now I want to talk about setting up tracing uh on any arbitrary Python function. So this was a big point of emphasis for our engineers; we wanted tracing to be really easy even if you weren't using our open-source frameworks, uh, and it's as simple as importing a decorator from the LangSmith SDK and adding that decorator to a function that you want to trace. So this looks very similar to what we just had before; the only difference is we have this decorator here. There are a few default run types that we can choose from, and we're just going to use retriever here because this is mimicking a document retrieval step. So we're going to go ahead and ask the exact same question. Now we're going to get the exact same AI message back, and if we go back into LangSmith, we'll now see our second run populate live. Clicking into this, we can now see that our runnable sequence has two steps within it. We have uh an additional fake DB retrieval step that's now been traced because we added the decorator to the function. This is a really important concept here called the run tree. Um, essentially every trace is made up of a tree of runs, and at the top level, you see the top-level input and output of the application. You see the question that we got, and we see the answer from the model. But if we click into these individual steps here, we can see the individual inputs and outputs at each step. So our fake document retrieval uh took in the question and retrieved the facts, and then our actual invocation of chat OpenAI uh again we created that system prompt, passed the human message, and then got the final AI output. So to recap, we just talked about setting up tracing in two different ways, both when you're using LangChain or LangGraph and also when you're not using LangChain or LangGraph. You can trace any arbitrary Python function. See you soon.