Transcription
Hello everybody, this is Leno Tadros. I am very excited to start this short series of a few videos to talk about Azure AI Foundry.
The first video will just set up the stage to show you how you can create a hub and a project inside of AI Foundry. From that point on, we'll keep adding to it. I'm not going to have to repeat every video about how to set up a hub and a project; I'm just going to keep continuing after the first video.
So, let's go ahead and get started. The first thing we're going to do is go to the Azure portal. I logged in already, and I'm going to go ahead and create a brand new Resource Group. Let me click on create here, and I'm going to call this one "RG AI demo." How about that? Sounds good!
I'm going to put it in whatever region I want. For right now, we are actually in December 2024 doing this video, and East US is not happy just because there's so much stuff going on in the retail business for Christmas and the New Year. So, we are going to move it to US East 2; otherwise, we're going to have problems. Hopefully, starting January, things will get better.
Anyway, I'm going to go ahead and create this Resource Group. Excellent! Now I go to the Resource Group, and I would like to start playing around with Azure AI Foundry. Usually, you can go to ai.azure.com, and that will take you to the Azure AI Foundry. It used to be called Azure AI Studio, but as of the end of 2024, it has changed its name to Azure AI Foundry.
You don't have to do it from there; you can actually still do it from the portal itself if you'd like. So, I'm going to go ahead and say create, and let's take a look at something called Azure AI Foundry. We'll probably find one right there.
Azure AI Foundry— that is the logo for Azure AI Foundry. Let's go ahead and create one of these resources. It's going to ask us a few questions. Yes, the one I just created, the Resource Group for AI demo, I am going to put it in US East 2 as well.
We'll call this, for instance, "Leno AI Hub." Okay, so we'll call this one "Leno AI Hub." I'll give it a friendly name. The default project Resource Group is the same, and also, I'm going to allow it to create AI services for OpenAI. I can create new and give it a better name, but I'm going to be okay with this name. You are more than welcome to change it to something a little bit more readable than this long name, but it's okay with me.
Then we're going to go to the storage. Again, I can change the name by saying create new, create something else, but for the sake of this demo, we'll leave that long name there; it's fine.
For the credentials, I'm going to actually use the Azure Key Vault, which is a good thing to do to keep all your secrets and passwords inside of there, as well as the API Keys. For the Key Vault, again, I'm not going to create a new one; I'm going to use that long name that got created for me automatically.
You can turn on Application Insights and Container Registry if you want. I'm not going to need them for this video, but in the future, if I need one, I will come and make that modification as well.
The next one is networking. For the sake of this demo, I'm going to keep it public, but of course, in a professional setting for enterprise level, you probably need to lock it down using private endpoints and having a jump box to get on the inside to do all that stuff. It's possible; you can definitely do that. I'm going to leave it public for right now.
For the encryption, I'm going to actually use the Microsoft Keys instead of using my own, so we're good with that. For the identity, I'm going to keep it as a system-assigned identity. This is a very important part, by the way. Otherwise, once you start uploading files like PDFs, docs, and Excel spreadsheets, you will not have access to them from the Hub.
This is an important piece as well. We'll use credential-based access on the system-assigned identity. I don't need to add tags, even though they're a good thing to have, of course, for budget reasons so you can run reports.
Finally, we'll review, make sure everything is good, and we'll click on Create. That will take us probably about maybe 30 to 40 seconds or so, so I'm going to stop the video and come back when the resource is created.
Indeed, it took about maybe 45 seconds or so; it was pretty fast. Let's go to that resource. There is my Leno AI Hub that just got created. I actually want to go to the Resource Group to show you something—what exactly got created for me.
You'll notice there is the Azure AI Hub that got created, the storage account, there is also the Azure AI services, and the Key Vault. But you will notice here there is no Azure OpenAI or Azure AI Search. All the stuff will come in later, but these are the minimum ones that got created for us at this point.
All right, let's go ahead and open up the resource for the Leno AI Hub. At this point, it's necessary for us to move over and go to the Azure AI Foundry itself. So, I'm going to launch it from here. Let's go and click on that; it will open up another tab here in my browser.
Once the Azure AI Hub or the Azure AI Foundry loads, the first thing it will say is, "Hey, by the way, you don't have a project." So, I don't have a project available, and it's going to ask me if I want to create a project in here and give it a name.
All righty, let's go and call it the "Leno Project." We'll just give it whatever name you want. We will use the Hub that got created for us, the Leno AI Hub, and that's good enough for me. We'll say create this project.
By creating this project, it will take a few seconds, and after that, I will be taken to the project itself in the Azure AI Foundry to start my work. Indeed, there it is! Now I'm in the Leno project.
You'll notice on the left side I can take a look at all the model catalogs available for me. I can play around in the playground to try to use some of the models, but right now I don't even have any models deployed. This project is completely empty, but I do actually have an API key, and as you can see, an Azure OpenAI endpoint was created for me automatically as well.
Great! That doesn't mean that I have a model available for me to try things on. So, let's go to the bottom here where it says "My Assets," and I'm going to click first of all on the model and endpoint. No models! I'm going to create my first one.
We'll say "Deploy Base Model," and you have tons and tons of options—not only OpenAI base but other things as well. If you go down on this list, you will see Mistral, Mera, 53.5, 54, Llama—there are tons of stuff.
In my case, let's go ahead and deploy GPT-4. I'm going to use it in several videos in this series. We'll say confirm that, and now it's going to ask me, "Do you want to change the name of the deployment?" I'm okay with GPT-4; that's good enough for me.
The only thing that I will change in here will be probably the capacity of 10K tokens per minute. I usually like to bring it into 40. Again, if you don't have that capacity, you're more than welcome to use 8, 10, or whatever you have available.
I'm usually going to do a lot of different things with audio and video and stuff later on, so I'm going to keep it to 40 for right now. We'll say deploy, and again, that will take a few seconds. You will notice I have it ready to go, and it's successful!
Another thing that I would like to do is to also deploy a model for the embedding. Do you actually have to do that? Not really. When you add your data and try to create an index, let's say for instance in Azure AI Search, if you want that to be your vector store, if you don't have an embedding model, the default one is always 8,8002 with 1,536 different dimensions.
So even if you don't have one, one will be created for you. I usually like to create these things myself, so I'm going to go here to the endpoints again and for the models. Let me click on that, and I'm going to say, other than GPT-4, let me go ahead and deploy also an embedding model.
I'm going to search for something, say with Ada, and hopefully, we'll see an Ada model here in a few seconds. There you go—text embedding 002. We'll say confirm this guy, and again, I'm going to leave everything the same; it looks good to me. We'll say deploy, and again, it will take a few seconds.
Successful! Great! So now if I go back to models, I have two things: I have the GPT-4, the LLM, and I have the text embedding 8,8002 for the embedding if I need it later on as well.
From that point on, folks, I can actually go ahead and click on playground, and I can try things out. Say, try chat in the playground. Of course, it found out that you already deployed GPT-4. If you deployed more than one, they will all be available in here, and that is your AI Azure OpenAI Hub that you created in here as well.
So, I'm going to go against the one I just deployed. We will leave, "You are an AI assistant that helps people find information," and I will come in here and will say, "Why is the sky blue?"
All righty, the typical question to make sure everything is working. Once I click on that, that will go against our GPT-4, and it will hopefully get me an answer here pretty soon.
All righty, there you go! It will explain the Rayleigh scattering and explain all that stuff. But look what happens, for instance, if I come in here and say, "What is the temperature in Orlando, Florida tonight?" That's where I live.
Look what happens, for instance, when I ask a question like this. It will say, "I'm unable to provide real-time weather information." Remember, this model was trained with data from several months ago and has no idea how to definitely answer a question like that because it doesn't have the latest and greatest information.
But the newer models like GPT-4 are so nice; they will tell you, "Well, if you want this kind of information, please head over to weather.com or ask Siri, Google Assistant, or Alexa." It's very helpful, but it cannot answer that question.
So, in the next video, we will see how we can ask those questions and get an answer directly from here as well using a prompt flow.
I hope this was a useful video for you. It's really quickly just to show you how to set everything up, and from the next one, we will start creating some prompt flows using Azure AI Foundry and also try to ask questions that are current—not based on the LLM that was trained several months ago that might not be able to answer that.
Let's go ahead and do that in the next video. Thank you!