Fundamentals of Generative AI AI-900

[ Music ]

ANDREW CONNIFF: Hello and welcome to the AI Fundamentals course, Generative AI. My name is Andrew Conniff, and I'm a Microsoft Technical Trainer, or MTT, with Microsoft. Let's get started with Session 5. You may have heard of large language models, or LLMs, like ChatGPT and others. We will learn what makes LLMs different and how to harness them for your applications in Azure. So let's get started.

This session is in three parts. We're going to talk about the fundamentals of generative AI, then we're going to do an introduction to Azure AI Studio and explore responsible generative AI in AI Studio.

In this module, we explore the way large language models enable AI applications and services to generate original content based on natural language input. We're going to learn how generative AI enables the creation of AI-powered copilots that can assist humans in creative tasks. By the end of this section, you'll be able to understand generative AI's place in the development of artificial intelligence and understand large language models and their role in intelligent applications. We'll also be able to describe how Azure OpenAI supports intelligent application creation and describe examples of copilots and good prompts.

AI imitates human behavior by using machine learning to interact with the environments and execute tasks without explicit directions on what to output. Generative AI is a subset of AI in which an AI model creates original content in response to a natural language prompt. Generative AI applications take in natural language input as a prompt and return the appropriate responses in the following formats: Natural language generation. So to generate a natural language response, you might submit a prompt such as, "Give me three ideas for a healthy breakfast including peppers," or "Write a cover letter for my resume."

Image generation. Some generative AI applications can interpret a natural language request and generate an appropriate image. For example, you might submit a request like, "Create an image of an elephant eating a plant-based burger," or "Create a logo for a florist business."

Code generation. Some generative AI applications are designed to help software developers write code. For example, you could submit a request like, "Show me how to code a game of tic-tac-toe with Python," or "Write Python code to add two numbers together."

Well, the mathematical principles behind language models can be complex. A basic understanding of the architecture used to implement them can help you gain a conceptual understanding of how they work. Today's large language models are based on transformer architecture, which builds on and extends some techniques that have been proven successful in modeling vocabularies to support NLP tasks, or natural language processing, particularly in generating language. So transformer models are trained with large volumes of text. This enables them to represent the semantic relationship between words and use those relationships to predict probable sequences of text that make sense. So the transformer model with a large enough vocabulary can generate language responses that are tough to distinguish from human responses.

In practice, the specific implementation of the architecture varies. For example, the bidirectional encoder representations from transformers, BERT, is a model developed by Google to support their search engine and uses only an encoder block to generate semantic vector representations of text. While the generative pre-trained transformer, or GPT model developed by OpenAI uses a decoder block to generate sequences of natural language.

While a complete explanation of every aspect of transformer models is beyond the scope of this module, an explanation of some of the key elements in transformers can help you get a sense for how they support generative AI. The encoder and decoder blocks in a transformer model include multiple layers that form the neural network for the model. We don't need to go into the details for all of these layers, but it's useful to consider one of the types of layers that is used in both blocks: Attention layers. Attention is a technique used to examine the sequence of text tokens and try to quantify the strength of the relationship between them. So self-attention involves considering how other tokens around one token influence that token's meaning.

In an encoder block, attention is used to examine each token in context and determine an appropriate encoding for its vector embedding. The vector values are based on the relationship between the token and the other tokens with which it frequently appears or which appear in similar contexts. This contextualized approach means that the same word may have multiple embeddings depending on the context in which it is used. So, for example, "the bark of a tree" means something different to "I heard a dog bark."

In a decoder block, attention layers are used to predict the next token in a sequence. For each token generated, the model has an attention layer that considers the sequence of tokens up to that point. The model considers which of the tokens are the most influential when considering what the next token should be. So remember that the attention layer is working with numeric vector representations of the tokens, not the actual text.

In a decoder, the process starts with a sequence of token embeddings representing the text to be completed. During training, the goal is to predict the vector for the final token in the sequence based on the preceding tokens. The attention layer assigns a numeric weight to each token in that sequence, and it uses that value to perform a calculation on the weighted vectors that produces an attention score. That can be used to calculate its possible vector for the next token. So in practice, a technique called multi-head attention uses different elements of the embeddings to calculate multiple attention scores. Then a neural network is used to evaluate all possible tokens to determine the most probable token which will continue the sequence. The process continues iteratively for each token in the sequence, with the output sequence so far being used regressively as the input for the next iteration, essentially building the output one token at a time.

The first step in training a transformer model is to decompose the training text into tokens. In other words, identify each unique text value. For the sake of simplicity, you can think of each distinct word in the training text as a token, though tokens can be made for partial words or a combination of words and punctuation. With a sufficiently large set of text on which to train a vocabulary, many thousands of tokens can be compiled. As you continue to train the model, each new token in the training text is added to the vocabulary with the appropriate token IDs. "I" is one, "herd" is two, "A" is three, "dog" is four. With a sufficiently large set of training text, a vocabulary of many thousands of tokens can be compiled.

To create a vocabulary that encapsulates semantic relationships between the tokens, we define contextual vectors known as embeddings. Vectors are multivalued numeric representations of information. For example, 10, 3, 1, in which each numeric element represents a particular attribute of the information. For language tokens, each element of a token's vector represents some semantic attribute of the token. The specific categories for the elements of the vectors in a language model are determined during training based on how commonly words are used together or in similar contexts. Vectors represent lines in multidimensional space describing direction and distance along multiple axes. You can impress your mathematical friends by calling these amplitude and magnitude. It can be useful to think of the elements in an embedding vector for a token as representing steps along a path in multidimensional space. For example, a vector with three elements representing a path in three-dimensional space in which the element values indicate the units traveling forward, backward, left, right, up, or down. Overall, the vector describes the direction and distance of the path from the origin to the end. The elements of the tokens in the embeddings space each represent some semantic attribute of the token so that semantically similar tokens should result in vectors that have a similar orientation, or in other words, they point in the same direction. A technique called cosine similarity is used to determine if two vectors have similar directions regardless of the distance and therefore represents semantically linked words. For example, the embedding vectors for "dog" and "puppy" describe a path along an almost identical direction, which is also very similar to the direction for "cat". But the embedding vector for "skateboard" will have a different journey, and it describes that journey in its embedding. Now, the example here shows a simple model in which each embedding has only three dimensions. Real language models have many more dimensions. There are multiple ways you can calculate appropriate embeddings for a given set of tokens, including language modeling algorithms like Word2vec or the encoder block in a transformer model.

The encoder and decoder blocks in a transformer model include multiple layers that form the neural network for the model. The encoder and decoder blocks in a transformer model include multiple layers that form the neural network for the model. And we don't need to go into the detail for all these layers, but it's useful to consider one of the types of layers that is used in both blocks, the attention layers. Attention is a technique used to examine a sequence of text tokens and to try to quantify the strength of the relationship between them. So self-attention involves considering how other tokens around one token influence that token's meaning.

In an encoder block, attention is used to examine each token in context and determine an appropriate encoding for its vector embedding. The vector values are based on the relationships between the token and the other tokens with which it frequently appears. This contextualized approach means that the same word might have multiple embeddings depending on the context in which it's used; for example, "the bark of a tree" means something different to "I heard a dog bark."

In the decoder block, attention layers are used to predict the next token in a sequence. So for each token generated, the model has an attention layer that considers the sequence of tokens up to that point. The model considers which of the tokens are the most influential when considering what the next token should be. So remember that the attention layer is working with numeric vector representations of the tokens, not the actual text.

In a decoder, the process starts with a sequence of token embeddings representing the text to be completed. During training, the goal is to predict the vector for the final token in the sequence based on the preceding tokens. The attention layer assigns a numeric weight to each token for that sequence so far, so it uses that value to perform a calculation on the weighted vectors that produce an attention score that can then be used to calculate a possible vector for the next token. In practice, a technique called multi-head attention uses different elements of the embeddings to calculate multiple attention scores. A neural network is then used to evaluate all the possible tokens to determine the most probable token with which to continue the sequence. The process continues iteratively for each token in the sequence, with the output sequence so far being used regressively as the input for the next iteration, essentially building the output one token at a time.

The following animation shows a simplified representation of how this works. In reality, the calculations performed by the attention layer are more complex, but the principles can be simplified as shown. A sequence of token embeddings is fed into the attention layer. Each token is represented as a vector of numeric values, and the goal in the decoder is to predict the next token in the sequence, which will also be a vector that aligns to an embedding in the model's vocabulary. The attention layer evaluates the sequence so far and then assigns weights to each token to represent their relative influence on the next token. The weights can be used to compute a new vector for the next token with an attention score. Multi-head attention uses different elements in the embeddings to calculate multiple alternative tokens. A fully connected neural network uses the scores in the calculated vectors to predict the most probable token from the entire vocabulary. The predicted output is appended to the sequence so far, which is used as the input for the next iteration. During training, the actual sequence of tokens is known. We just mask the ones that come later in the sequence than the token position currently being considered. As in any neural network, the predicted values for the token vector are compared to the actual value of the next vector in the sequence, and the loss is calculated. The weights are then incrementally adjusted to reduce the loss and improve the model, and when used for inferencing or predicting a new sequence of tokens, the trained attention layer applies weights to the -- predicted the most probable tokens in the model's vocabulary and that is semantically aligned to the sequence so far.

What this means is that a transformer model such as GPT-4, the model behind ChatGPT and Bing, is designed to take the next input, called a prompt, and generate a syntactically correct output called a completion. In effect, the magic of the model is that it can string a coherent sentence together. This ability doesn't imply any knowledge or intelligence on the part of the model, just a large vocabulary and the ability to generate meaningful sequences of words. What makes a large language model like GPT-4 so powerful is the sheer volume of data with which it has been trained. So public and licensed data from the Internet is used. The complexity of the network and its ability for the model to generate completions is based on the relationships between words and the vocabulary on which the model was trained, often generating output indistinguishable from a human response to the same prompt. A diagram that shows how the attention technique helps the large language models identify the next word in the sequence. The image shows words and their positioning in the encoding layer coordinates.

Organizations and developers can train their own language models from scratch, but in most cases, it's more practical to use an existing foundation model and optionally fine-tune it with your own training data. There are many sources of models that you can use. On Microsoft Azure, the Azure OpenAI service includes a curated set of models from OpenAI hosted in Azure. This offers the benefit of cutting-edge language models like the generative pre-trained transformer, or GPT collection of models on which ChatGPT and Microsoft's own generative AI services are based, as well as the DALL-E model for image generation. Using these models from the Azure OpenAI service means that you also get the benefit of a secure, scalable Azure Cloud Platform in which the models are hosted. The Azure OpenAI models are included in the model catalog, a curated source of models for data scientists and developers, using Azure AI Studio and Azure Machine Learning. In addition to the Azure OpenAI models, the model catalog includes the latest open-source models from Microsoft and multiple partners including OpenAI, Hugging Face, Mistral, Meta, and others.

There are many language models available that you can use to power generative AI applications. In general, language models can be considered in two categories: Large language models, or LLMs, and small language models, or SLMs. Large language models are trained with vast quantities of text that represent a wide range of general subject matter, typically by sourcing data from the Internet and other generally available publications. When trained, large language models have many billions or even trillions of parameters. These are weights that can be applied to vector embeddings to calculate predicted token sequences, as we've discussed, enabling them to exhibit comprehensive language generation capabilities in a wide range of conversational contexts. However, their large size can impact the performance and make it difficult to deploy locally on devices and computers. Additionally, if you want to fine-tune the model with your own additional data to customize its conversational subject expertise, the process can be time-consuming and expensive in terms of the compute power required to perform the additional training.

Conversely, SLMs, or small language models, are trained with smaller, more subject-focused datasets and typically have fewer parameters than the LLMs. This focused vocabulary makes them very effective in specific conversational topics but less effective in more general language generation. The smaller size of SLMs can provide more options for deployment, including local deployment to devices and on-premises computers. And it makes them faster and easier to fine-tune.

Copilots are generative AI assistants that are integrated into applications, often as chat interfaces. They provide contextualized support for common tasks in those applications. Business users can use copilots to boost their productivity and creativity with AI-generated content and the automation of tasks. Developers can extend copilots by creating plugins that integrate them into business processes and data or even create custom copilots to build generative AI capabilities into apps and services. The slide here shows Microsoft Copilot for Microsoft 365, which enables you to summon a chat assistant when you are working in Windows or a Microsoft 365 application, such as Outlook or Word. Using a copilot like this can help you get more things done in less time.

On the next few slides, we'll explore some of the ways in which Microsoft Copilot has been integrated into apps to provide contextual assistance. There are three levels of copilot adoption in an organization. You can use off-the-shelf copilots like Microsoft Copilot for Microsoft 365 to empower users and increase their productivity. You can extend Copilot to support custom business processes or tasks using your own data to control how Copilot responds to your prompts in your organization, or you can build custom copilots to integrate generative AI into business apps or to create unique experiences for your customers.

You can use Microsoft Copilot to browse the web more effectively. The Microsoft Copilot app at copilot.microsoft.com provides Microsoft Copilot's home on the web. You can go here and ask questions and generate content such as text and images. Signing in with a work or school account enables you to use Copilot in the context of your organization's data and services or your own. It enables you to get assistance without internal resources and information.

Microsoft Copilot for Microsoft 365 integrates Copilot into the productivity applications that information workers use every day. For example, you can use Copilot in Microsoft Word to generate a new document based on a natural language prompt and then refine, summarize, and improve the document with a few prompts. You can use Copilot in Microsoft PowerPoint to create a whole presentation based on the contents of a document or an e-mail and then add graphics, reformat slides, and otherwise improve your presentation. In Microsoft Outlook, Copilot can help you summarize your e-mails, check your schedule, even find relevant e-mails and documents to prepare for meetings.

Microsoft Dynamics 365 is a suite of business tools that helps users in specific roles perform business processes. Copilot for Dynamics 365 provides contextualized assistance to those tools and helps users be more efficient and effective. For example, sales professionals can use Copilot to quickly find relevant customer and industry information by integrating with the company's customer relations management, or CRM database and beyond. This can enable an account manager to quickly review and qualify a lead, generate a proposal, and then set up a customer engagement to close the deal. Customer service agents can use Copilot in Dynamics 365 Customer Service to analyze support tickets, research similar issues, find resolutions, and communicate to the users with only a few clicks and prompts.

Data analysts must work with code and visualization tools to analyze data and report insights. The Microsoft Fabric Copilot enables an analyst to automatically generate the code that they need to analyze, manipulate, and visualize data in Spark Notebooks. And when creating Power BI reports, Copilot can analyze your data and then suggest and create the appropriate data visualizations from that.

Copilot for Azure is integrated into the Azure portal and can assist infrastructure administrators as they work with Azure Cloud Services. Microsoft Copilot for Security helps security professionals as they assess, mitigate, and respond to security threats.

GitHub is the world's most popular place for developers to manage their code and develop applications. GitHub Copilot helps developers maximize productivity by analyzing and explaining their code, adding code documentation, generating code based on natural language prompts, refactoring, optimizing code, generating test cases for code for existing functions, and integrating into developer tools like Microsoft Visual Studio Code, which means you can take it anywhere. And it does so much more.

While some copilots provide buttons and other visual tools to interact with language models, you will often use a copilot by typing or speaking a natural language prompt. The specific prompt that will yield the most effective results can vary depending on the specific task you're trying to accomplish, the copilot being used, and the language model it's being supported by. However, there are some common prompting techniques that you can apply to get the best out of your Copilot. You start with a specific goal in mind. What is it you want the copilot to do? And you'd be very explicit here. You want to provide a source to ground the response in, so a specific scope of information. This will help ensure the response is based on real data. And add context. Context is key. This can help the language model craft an appropriate response. Then set clear expectations for the response. Be very explicit about the format and scope of the expected results. Iterate based on your past prompts and responses to refine your output.

In most cases, a copilot does not just send your prompt as is to the language model. Usually, your prompt is augmented with a system message that sets conditions and constraints for the language model behavior, and the conversation history for the current session can also be included if you don't start a new session. And this enables you to refine your response iteratively while maintaining the context of the conversation. That current prompt, potentially optimized by the copilot to reword it and appropriate it for the model or to add additional grounding to the data to the scope of the response, is then sent to the model.

The success you have with copilots and generative AI in general depends largely on how well you craft your prompts. The importance of using good prompts has led to a new discipline in prompting engineering. And let's pause here for a second. And I want to stress, don't shy away from longer prompts. We see a lot of examples that are short, but when you add a lot of context and you add all those pieces together, don't be afraid to have a long prompt to work with, and you're going to get the best results if you do that.

If your organization makes the decision to customize Microsoft Copilot or develop custom copilots, Microsoft provides two tools that you can use. Copilot Studio is designed to work with low-code and no-code development scenarios in which a technically proficient business user or a developer can create conversational AI experiences. The resulting copilot is a fully managed software as a service solution, hosted in your Microsoft 365 environment, and delivered through chat channels like Microsoft Teams. With Copilot Studio, the infrastructure considerations and model deployment details are taken care of for you, making it easier to focus on creating an effective solution.

Azure AI Studio is a platform as a service development portal for professional software developers that gives you full control over the language model you want to use, including the capability to fine-tune the model with your own data. You can define prompt flows that orchestrate conversation flow and integrate your own data augmentation and prompt engineering logic, and you can deploy the resulting copilot service in the cloud and consume it from custom-developed applications and services.

Let's look at generative AI with Microsoft Copilot. Here I am in the browser and I have my OneDrive open. And I'm signed in with my personal Microsoft account, and I have this document, this business ideas document. And it's in my OneDrive, and this is important because I'm in the browser. Then I opened up Copilot in the browser, and the web experience here, it's going to be the same for everyone, right? So I've got the Copilot open, and whatever webpage I have open it can read. So let's ask it, "What is this document?" All right. So what is it about? It's going to go ahead and semantically understand. So we know it's going to take all of this information, send it to the model, and say, "What is this document about?" and send it back. So now I want to find out, well, it's about a business idea. "How do I open up a business in New York City?" That's what we want to do, right? So here's all of the steps that I would have to take. This is all relevant to my document, and this conversation is ongoing. So now I want a new name for my business, and I'm going to go with "Pure Clean New York City". So let's go with that. And now I want a business plan. So I've gotten ideas all just in a matter of minutes here. I know how to open up a business. I've got a new name for my product, and I've got a business plan that I can now add to my document. And of course, I would keep refining this. These are all first drafts when we get these back from Copilot. We want to go through and do our fact-checking here.

Now, the next step. "What is the next step?" We can take the power of this AI, and here I am logged in with my work account. This is Copilot for Microsoft 365. You can see I have my labels. But what I'm doing here is I'm prompting against my own documents, and I'm going to call three documents, and I want a comprehensive strategic analysis for my Contoso Learn and put these separate items in there. So an executive summary and an account and all that. So look at this. There's my new document. So I have three documents that are 670 pages each, and now I've got a new summary document that's three pages long. And, again, it's a first draft. All right. So what else can I do?

With this? Well, if I go to PowerPoint, I can now call this document from PowerPoint. And I put the URL in it, and I basically say, hey, you know what, create a PowerPoint from this document. And it's going to go semantically read my summary document with all of the steps. You can see it has all of the executive summaries, the app overview, market research, and it's going to semantically understand how to build an appropriate PowerPoint and even give me speaker notes. And that's been a quick tour of Microsoft Copilot and Microsoft Copilot M365.

This section we will discuss Azure AI Studio and how it provides a single solution for AI development with multiple Azure AI services. An Azure AI resource defines a collaborative workspace for AI development with Azure AI Studio, and an Azure AI project provides a share collection of assets and code for a particular AI solution. Azure AI Studio is a PaaS, or platform as a service, development portal for professional software developers that gives you full control over the language model you want to use, including the capability to fine-tune the model with your own data. The growth of the use of artificial intelligence, AI in general, and generative AI in particular, means that developers are increasingly required to create comprehensive AI solutions. These solutions need to combine machine learning models, AI services, prompt engineering solutions, and custom code. Microsoft Azure provides multiple services that you can use to create AI solutions, but until now, developers have needed to work with multiple tools and web portals in a single project. Azure AI Studio brings together capabilities from Azure Machine Learning, Azure Open AI Service, and other Azure AI related services into a single centralized workspace within which developers can collaborate with data scientists and others to build AI solutions.

Azure OpenAI Service is Microsoft's cloud solution for deploying, customizing, and hosting language models. Azure Machine Learning is Microsoft's solution for training and deploying machine learning models and managing machine learning operations, or MLOps. Azure AI Services encompasses Microsoft's Cloud solutions for creating cutting-edge, market-ready, and responsible applications with pre-built and customizable APIs and models. Azure AI Studio is a web portal that brings together multiple Azure AI related services into a single unified development environment. Specifically, Azure AI Studio combines the model catalog and prompt flow development capabilities of Azure Machine Learning service and the generative AI model deployment, testing, and custom data integration capabilities of Azure OpenAI Service and integration with Azure AI services for speech, vision, language, document intelligence, and content safety.

AI hubs in Azure AI Studio are top-level resources that provide a centralized setup and management for AI projects. Offering features like data upload, artifact storage, connections to Azure services, base model endpoints, compute resources, and governance controls. You can perform the following tasks in the AI hub level: You can create and manage connections to resources such as data stores, GitHub, Azure AI search indexes, and others. You can create and manage compute instances on which to run experiments, prompt flows, and custom code. You can set up security by creating members and assigning them to specific roles and set up governance by defining policies to manage behavior, such as automatic compute shutdown.

Projects are organizational containers within a hub that allow for AI customization and orchestration. They help your organization work, save state across different tools like prompt flow, and collaborate with others. Projects can use shared resources from the AI hub and have dedicated storage containers for uploading files and sharing with project members. With an AI project, you can deploy foundation models from the model catalog and test the models in the chat playground. Augment prompts by connecting to custom data sources and connections and build copilots with prompt flow. Evaluate model and application performance by using built-in and custom metrics and manage the deployment of your models and apps. With role-based access control, or RBAC, you can assign roles to users and give them access on the AI hub or the project level.

Things we can do in Azure AI Studios are create and manage AI projects. So Azure AI Studio provides a centralized hub for all your AI projects, allowing you to manage resources, collaborate with team members, and streamline your workflow. You can develop generative AI applications. If your goal is to develop an application that can generate content or build your own prompt flow, Azure AI Studio's generative AI capabilities are essential. You can explore available AI models. So experiment with various AI models from OpenAI, Microsoft, Hugging Face, and more in Azure AI Studios model catalog. You can leverage retrieval augmented generation, or RAG. For projects that require combining the power of retrieval and generation, Azure AI Studios RAG features enhance the quality and relevance of the generated content. You can monitor and evaluate AI models. So Azure AI Studio provides robust tools for evaluation and monitoring of your prompt flows and AI models, ensuring they meet the desired performance metrics. You can also integrate with Azure services. When your AI applications need to work seamlessly with other Azure services, Azure AI Studio offers easy integration, making it a versatile choice for your complex multi-faceted projects. You can build responsibly. Azure AI Studio emphasizes responsible use of AI, providing guidance and tools to ensure that your applications adhere to ethical standards and best practices.

Prompt flow is a feature within Azure AI Studio that allows you to author flows. Flows are executable workflows integrating with language models, and your prompt flow in Azure AI services is a development tool designed to streamline the entire life cycle of an AI application powered by the large language models. It simplifies the process of prototyping, experimenting, iterating, and deploying AI applications. Here are some key features: Visualized graphs; you can orchestrate executable flows with large language models, prompts, and Python tools through a visual interface. Prompt variants; create and compare multiple prompt variants to refine your AI models, and you have evaluation tools. Built-in evaluation flows help assess the quality and effectiveness of your prompts and flows. You can also share and iterate on flows easily with team collaboration, and the prompt flow, it should be noted, is available as an open-source project on GitHub and as a feature within Azure AI Studio and Azure Machine Learning Studio. A flow is an executable instruction set that can implement the AI logic. Flows can be created or run via multiple tools like a pre-built canvas, LangChain, etc. Iterations of a flow can be saved as assets. Once deployed, a flow becomes an API. Not all flows are prompt flows, rather prompt flow is one way to create a flow.

The model catalog in Azure AI Studio is the hub to discover and use a wide range of models that enable you to build generative AI applications. The model catalog features hundreds of models across model providers such as Azure OpenAI Service, Mistral, Meta, Cohere, NVIDIA, Hugging Face, including models trained by Microsoft. Models from providers other than Microsoft are non-Microsoft products as defined in Microsoft's product terms and subject to the terms provided with the model. As discussed in the earlier section, generative AI models are excellent at understanding and creating natural language. They can write code from natural language prompts. They can edit and create images based on the model type. The model that works with images is called DALL-E, which supports image creation, image editing, and image variations creation.

In this demo, you'll see Azure AI Studio's capabilities in action. So here we are in Azure AI Studio. And as you can see, we have all of our Azure AI services right here. And what I want to do is I want to work with my Azure OpenAI. And if I go to my OpenAI, I can see I have some legacy services that I can see there, but I'm going to create a new service inside of Azure Studio. You can see when I go there, there isn't anything there. So we're going to go back and I'm going to create a new project. So part of Azure AI Studio is we have the projects and the hubs. And so here I'm going to create a new project, and with it we're going to get a hub. And that hub is going to allow us to then share and invite people and collaborate with the results of creating this project. So give that just a minute to create. All right. Now that we're there, we can go through and take a look at the different models and the different things that we can create. What I want to do is I want to build or deploy a new model. In this case, I'm going to choose a GPT model. I want GPT-4o. So I'm going to go deploy model, I'm going to choose 4o, and then I'm going to go ahead and fill out all of my stuff here. So here I'm lowering the token amount. Your tokens are based on the region, and we don't want to overuse. We want to definitely dial that number in specifically. All right. Now I can see the details about my GPT-4o deployment and I can go to the chat playground. So really quickly, we're just going to ask it a question. We just did this. So what's going to happen here is it won't necessarily be available immediately. And as you see, it wasn't available. So let's try it again. Give it a minute to -- there we go. Artificial intelligence is, so what is it? Maybe we can even do things like ask it to do that in another language. That has been a quick tour of Azure AI Studio and Azure OpenAI.

This section has two parts: Plan a responsible generative AI solution and operate a responsible generative AI solution. The Microsoft guidance for responsible generative AI is designed to be practical and actionable. It defines a four-stage process to develop and implement the plan for responsible AI when using generative AI models. The four stages in the process are identify potential harms that are relevant to your planned solution, measure the presence of these harms and the outputs generated by your solution, mitigate the harms at multiple layers in the solution to minimize their presence and impact, and ensure transparent communication about potential risks to users. Operate the solution responsibly by defining and following a deployment and operational readiness plan. These stages correspond closely to the functions in the NIST AI risk management framework. The remainder of this learning objective discusses each of these stages in detail, providing suggestions for actions you can take to implement a successful and responsible generative AI solution.

Azure Content Safety has several key features such as prompt shields, scans for the risk of the user input attacks on language models, groundedness detection, if the text responses are grounded in the user source content, protected material detection which scans for known copyrighted content, custom categories, and then also defining custom categories for any of these new and emerging patterns. Let's look at Azure AI content filters. Here I am in the chat playground, and I'm going to open up my content filter section. So again, we're in the Azure OpenAI inside of Azure AI Studio. I open up the content filters, and this is where I can set the thresholds for incoming and outgoing text. So if I am on high threshold, that means I'm going to allow more, and if I'm on low threshold, that means I'm going to allow less information through. So I'll leave that one set to low, and I also have my jailbreak attack, which is an attempt to get the model to do something it's been trained to not do, and then we have prompt shields for indirect attacks. In this case, we're just going to annotate if something happens. Now on the return from the model, I can do the same. So if the model decides to return something that's going to be something I don't want my customers to see or my employees to see, I can allow less or be less tolerant, or I can allow more and be more. It's important to know that you can have all of these filters removed from your model. If you have a need, you just ask Microsoft and they'll remove them for you. So here what we're going to do is we're selecting our model, and once we're done and we create our filters, it's going to be applied directly to that 4o model. You can do this later as well, but that is a quick walkthrough of how to set the thresholds for your content filters and have custom content filters on your different models.

This concludes Session 5. And there are many ways to continue your learning journey, and we encourage you to watch other videos in this course or search out your next favorite topic on Microsoft Learn at aka.ms/learn. And I hope to see you next time.