Python Essentials for AI Agents – Tutorial

Welcome to this Python essentials for building agents. This course will help you master the technical stack behind autonomous intelligence. This course is designed to take you from the core syntax of Python through the complexities of data handling and API integration, culminating in the deployment of sophisticated large language models. You'll learn to write code and to architect systems that can reason, use tools, and solve real-world problems using both proprietary and open-source AI.

Hi, I'm Prashant Sahu, and I'm excited to have you here as we take our first steps into the world of coding, data, and artificial intelligence. In this course, we'll take a hands-on approach to learning. We'll start with the fundamentals in Module One, where we will dive into Python basics, variables, data types, functions, and more. By the end of this module, you'll be writing clean, efficient Python code and we'll have a solid foundation for tackling complex tasks.

Next, in Module Two, we'll work with files and databases. We'll explore Pandas, one of Python's most popular libraries for data analysis, and get hands-on with databases and SQL. You'll learn how to connect Python to databases and manage data effectively, a crucial skill for any aspiring AI practitioner.

In Module Three, we'll explore APIs, what they are, why they are essential, and how to use them in Python. You'll learn to connect with external services, work with JSON data, and handle common challenges like authentication and rate limits. We will introduce you to how APIs power AI agents and large language models, which is where things get really exciting.

Finally, in Module Four, we'll dive into working with language models, often called LLMs. You will learn how to use libraries like OpenAI and explore open-source tools like Hugging Face. We'll cover how to access, prompt, and interact with LLMs, giving you a peek into building intelligent systems.

Overall, by the end of this course, you will not only understand Python programming but also have hands-on experience working with data, connecting with APIs, and interacting with cutting-edge language models. I'm here to guide you at every step of the way, and I can't wait to see what you will create. Also, in case you want to dive deep into generative AI, machine learning, and deep learning, feel free to check out some free courses made by my team.

Before we dive into the details of Python and its relevance in data science, I would like to give you a quick overview of what Python is and why it has become so crucial for today's data professionals. Python, as you may know, was created by Guido van Rossum in 1991. Since then, it has been developed and maintained by the Python Software Foundation, growing into one of the most widely used programming languages globally.

Now, let's talk about the background of Python. Python is known as a high-level, general-purpose programming language. Python has become incredibly popular because of its readability and flexibility. It is used in various fields such as scientific computing, web development, and artificial intelligence, amongst many others. One of its biggest advantages is that it is an interpreted language, meaning it runs in a virtual environment, which makes it platform-independent. This means that it can run on almost any system like Windows, Mac OS, or Linux without much hassle.

Now, why should we use Python for data science? The short answer is that it ticks all the boxes. First, Python boasts a large and active community. This is really helpful because you have access to countless tutorials, forums, and even solutions to problems you might face, making the learning curve much smoother. Secondly, Python's syntax is very easy to understand, which allows data scientists to focus on solving complex problems rather than spending too much time on the intricacies of coding. It is also open-source, which means it is free to use, and there's a continuous stream of improvements for developers from all over the world. Another big advantage is Python's rich collection of libraries like Pandas, NumPy, Matplotlib, and Scikit-learn. These libraries make tasks like data manipulation, visualization, and machine learning easier and more efficient. The widespread use of Python in data science also translates into plenty of job opportunities, which is another huge plus.

Before we start coding, it's important to set up your system properly. For data science, many developers prefer working in an integrated environment like Jupyter Lab or Jupyter Notebook. Jupyter Lab allows you to write, execute, and document Python code all in one place, making it perfect for data analysis and visualization. We'll walk through the steps of installing and setting up Jupyter Lab, which will ensure you have the right environment for working with Python throughout this course.

Now let's go to Google and search for Anaconda. We'll click on the first link, and this is the webpage from where you can start downloading the Anaconda distribution for Python. On this page, you can find the link to download the Anaconda distribution. So if you see here, below the submit button, there is a "skip registration" link. Let's skip the registration and straight away go to the download page. So this is the download page. Based on your operating system, you can click on the links to download the installer files. So here I'm using a Windows system. I'll click on the 64-bit installer, which is 912 megabytes. So it might take some time for you to download. So let's wait.

So once we have downloaded the Anaconda installation file, double-click to start the installation. Okay. So I have double-clicked on the installer file, and this is the installation wizard which will appear. So let's click on "Next". Agree to the license agreement, and then it shows you a screen whether to select the installation just for you or all the users. Go with the "Just Me" option, which is the recommended, and then it will select a default path based on the operating system for installation. Make sure you have enough space in your C drive. So by default, the installation directory is C:\Users\<username>\Anaconda3. And there are some options given to you whether you want to add Anaconda or Python in your system path. Go with the default options and just hit "Install". It will take some time for you to install this, around 2 to 3 minutes, so based on your system's configuration. So let's wait. Once it is installed, we'll come back.

Now, once the installation is complete, this is the window that will appear. Just click "Next" and "Next" and "Finish". So it will give you an option to launch the Anaconda Navigator, which we will leave checked and uncheck this last option. Okay, the Anaconda Navigator has been launched. This is how the window looks. I'll just click on "Don't show this again". So let us see what all packages have been installed within Anaconda Navigator. So whichever says "Launch", all these softwares have been installed when you install Anaconda Navigator, which means PyCharm is not installed. If you see Anaconda Toolbox is installed, Notebooks are installed, Jupyter Lab is already installed. Okay, Jupyter Notebook is installed, and we'll be using Jupyter Lab and Jupyter Notebooks extensively in this course. So let's click on the Jupyter Lab launch again. It will take a couple of minutes for this to launch. Remember that the Jupyter Lab will launch in the browser window. So I've got my Jupyter Lab up and running in the browser window. There is one small warning: do not close the Anaconda Navigator window as long as you are running the Jupyter Lab from here. Okay.

So the default installation directory is shown on the left side. This is the file browser window you can see, and from here you can actually launch Jupyter Notebook. You can have access to the Python console or a terminal. You can also create a requirement.txt file from here, and you can also create Markdown files with an extension .md. You can create standard Python files with an extension .py, and you can also access the other applications from Anaconda. Okay. So, first we will learn how to create a folder within Jupyter Lab, and then we'll learn how to create a new file, a new Jupyter Notebook from the Anaconda environment and see. Okay. So first, you need to decide where you want to keep your code files. Let's say I want to keep my code files within this is your C drive and Users directory. Okay. From here, I am just browsing to Desktop. So just right-click and create a new folder from here. Okay. Specify a folder name. So my folder name is "Coding Essentials for AI". Double-click on that. So the folder appears to be empty as of now. And again, if you want to create a subfolder, you can go ahead and do that. Let's say I create "Module One". And within "Module One", double-click. So to create a new Jupyter Notebook, right-click and click on the "New Notebook" option. This is the window that will appear. This will ask you to select the Python kernel to associate with the notebook. So you can see that I have no other kernel apart from this basic Python 3. So I'll go ahead and take that. You can also check this to start with Python 3 as your preferred or the default kernel. So kernel is the program which interprets your Python code and executes it. You can resize these windows. So this is how your coding window looks like. This is my coding window. And on the left side, I have again my file browser window. Here you can type your Python codes. My first Python program, let's say, print "Hello, world!". And to execute this, you can do a Shift+Enter or Ctrl+Enter. You can also click on this button to execute this cell. Okay. So codes are written in code cells. This is one code cell, whichever is highlighted in blue boundary. So that's a code cell. You can see that this is currently in the code mode. So you can also change the mode of the cell. So this is currently in the code. So I'll click on this. This will execute the cell. So once the cell is executed, the output of the cell is displayed just below the cell. So this is one way you can execute the cell. Otherwise, if you want to do it from your keyboard, you can do a Shift+Enter or Ctrl+Enter. Shift+Enter will execute the current cell and advance to the next cell. Ctrl+Enter will execute the current cell and stay there.

Now let's have a look into another coding environment which does not require any installation and can be accessed from anywhere in the world, and that is Google Colab. So that also opens in another browser window. But you need to first sign in into your Google account. So we'll go to Google and search for "Google Colab" and here in the first link, click on that. So assuming that you are already signed into your Google account, this is the window that should appear. If you have already previously opened any Google Colab notebooks, that will also be displayed. So on the left side, you can see there is a menu, table of contents, some standard Colab notebooks would be there. So what I'll do is click on "File" -> "New notebook in Drive". This will create a new Jupyter Notebook, and that will be saved in your Google Drive. Okay. So on the top, you can double-click and provide a name to your Jupyter Notebook. Your Jupyter Notebook will have the extension .ipynb, that is interactive Python notebook, as the default extension. And again, as usual, this is your code block. Okay, you can type your Python code. So let's say this is my first "Hello, world!" program in Python. And again, you have three options to execute. But before that, on the top right, make sure that you have connected your Jupyter Notebook with a remote Python kernel, which will be running on Google's server. Okay. So you have many options, but we'll go with the default option right now. Click on "Connect to a new runtime". So make sure that your Jupyter Notebook is connected to a runtime. Once it is connected, it will show a tick mark. This might take 15 to 20 seconds for it to connect. Please note that in the free account of Google Colab, you cannot run parallelly more than one Colab notebooks. If you try to run another Colab notebook, it will ask you to shut down the previous notebook. So this is one limitation with the free account for Google Colab. So as you can see, there is a green tick, which means my Colab notebook has already been connected to the remote runtime. So now I can go back to my code and again I can execute this. So again, execution has three options. You can click on this icon to execute the cell, or you can do a Ctrl+Enter or a Shift+Enter. So I'll click on this link, and you can see the output of this cell is below the cell. Okay. So this is how you can quickly create your codes in a Colab notebook environment. And if you're curious to know where this notebook is saved, you can go to "File" -> "Locate in Drive". So this will go to the exact Colab notebooks folder in your Google account. So now, as you can see, all my Colab notebooks are located in the "My Drive/Colab Notebooks" folder. Okay. So this is the Module One Jupyter Notebook which I created on Colab, and there are some other notebooks which I had already created. So this is present within my Drive/Colab Notebooks. So this is how you can actually also upload your Colab notebooks to this particular drive and execute it from there. So let's say if you want to execute it from here, you can open any of the existing notebooks by right-clicking on it and "Open with Google Colab". This is how you can open any Jupyter Notebook. Once you have uploaded it on Google Colab, right-click and open it on Google Colab. So this will open it in another tab from where you can again connect to a runtime and start executing. So that's it for this particular video. See you in the next one. Thank you.

Hello and welcome back. In this video, we'll dive into some foundational concepts that are essential for coding in Python: variables, data types, and operators. By the end of this session, you will understand how to declare variables, use different types of data, and perform operations on them. So, let's get started.

Let's get started with variables and data types. First, let's talk about variables and data types in Python. Think of variables as a container that stores information, like a number, a word, or even a list of items. Python makes working with variables very easy. You don't need to declare them explicitly, unlike some other programming languages. Just give them a name and assign them a value, and voila, you are ready to use them. For instance, here we see different data types like numbers, strings, and even more complex data types like lists and tuples. Each of these represents a different type of information that Python can handle.

Now, let's understand how to declare and assign values to variables. In Python, you can declare a variable simply by assigning a value to it with an equals operator. Let's say we want to store the number 50 in a variable named `a`. We just write `a = 50`. That's it. No need for any extra keywords or statements. Python is very intuitive in this way. If you want to store `a` in another variable, we can do something like `b = a`. This assigns the value of `a` to `b`, making both variables contain 50. Simple and effective.

Python has a few core data types that you will be using often. These include integer for whole numbers, float for numbers with decimals, string for text, boolean which represents true or false, and None which represents a variable that has no value. We also have some collections like lists, tuples, dictionaries, and sets, which are often very helpful for grouping related data. Here you can see strings as well, because strings are considered to be a collection of characters in Python. Understanding these data types will give you a powerful toolbox for tackling almost any coding problem.

There are a few simple rules when it comes to naming variables in Python. First, variable names are case-sensitive. So, `abc` and `ABC` are considered different variables. Also, variable names must start with a letter or an underscore and never with a number. You can also use numbers in the name, but they can't be at the beginning. And avoid using Python's reserved keywords like `if`, `else`, `for`, `while`, `def`, `return`. These have special meanings in the Python language. As an example, a good variable name might be `student_score`. While something like `123_score` is not allowed. Keeping these rules in mind will help avoid any errors as you code.

Now it's your turn. Let's practice what we have learned by declaring and assigning some variables in a Jupyter Notebook. So let us first start the Jupyter Notebook. I will search for "Jupyter Notebook". So run this Jupyter Notebook app. Uh once it is started, it will open in a new browser window, whichever is your default browser. So you'll have access to all these folders. This is typically present within your C drive users. So these are all the folders which are present within the C:\Users directory. You can browse all your files from Desktop, Documents, Downloads, and so on. Now, the only problem is my code files, which I have downloaded for this particular module, are in the D drive, AI Agent, and Code Files. So first, we will have to actually browse and change the directory to this one. Okay. So for this, we'll close this and start with Anaconda Prompt. Make sure you are selecting the Anaconda Prompt. Now, this is different from your normal command prompt. How your Anaconda Prompt will have this `(base)` written. Normal command prompt will not have that `(base)`. Okay, close this. Now, since my folders are in D drive, I'll directly go to D drive first, and from there I can actually start my Jupyter Notebook. If you do not do this, you will not be able to access D drive files. You will only be able to access the files which are present within C:\Users and whatever the username. Press Enter. And this might take some seconds to open. Okay. So the interface is still the same. It's just that the technique that I've used is slightly different so that I get access to the actual code files which I have saved in my desired directory. So you can see these are the folders present within the D drive. So this is where my AI agent code files are. Okay. So now you can see all the Jupyter Notebooks for this particular module. I'll double-click on the first one, which is "Implementing Variables and Data Types in Python". So let's get started. Okay. So you can execute all the codes by using or by clicking on this button, which is the run button. Okay. Or a keyboard shortcut you can use that is Shift+Enter or Ctrl+Enter. So I'll use this keyboard shortcut key, Shift+Enter. So let me define a variable `a` and assign a value equal to five. So how do I know that the value has been assigned? Let's print the value of the variable. So when I executed this cell by using Shift+Enter, this is the value of five that I have got for this `a`. So this way you can define any variable name and using the equality operator, you can assign a value to it. Okay. You can also use the print function to print the value of the variable. You can see that `print(a)` gives me five. Fine. So here I am defining `a = 7` and I am creating another variable `b` and assigning the value of `a` to `b`, and then changing the value of `a` from 7 to 3. Let's see what happens. So what will be the value of `a` now? Since the value of `a` has been changed from `a = 7` to `a = 3`, this should give me `a = 3`. Perfect. And since `b` was assigned a value equal to `a` when `a` was having a value of seven, `b` should still point to the original value of `a`, which is seven. Super. So this is how the variables work in Python.

Now we have already discussed the variable name rules in the presentation. So here is a quick recap of those rules. Python variable names are case-sensitive, which means `a` and `A` would be different. Identifiers or variable names can be a combination of letters, lowercase or uppercase, and digits. You can even use an underscore. You can use the underscore at the start or at the end. So these are all valid variable names. You cannot use a variable name which starts with a digit. So, for example, `1_variable` is not valid, but `variable_1` is perfectly fine. So let's see. And then finally, you cannot also use keywords as variable names because keywords are reserved. So if you try to assign any value to any of these keywords, Python will throw you an error. So here I've defined, let's say, `a = 400` and `a` (lowercase) as 400 and `A` (uppercase) as 4. Let's see what happens. So you can see that `a` still points to 400 and `A` points to 4. So some of the rules, let's verify them quickly. So `_a` is valid. Okay. But if you say `a_`, that is also valid. Okay. Because underscores are acceptable. But if you have any other special characters either at the start or at the end, that will be invalid. Okay. You can see that's a syntax error. Same syntax error you will get when you try to define `1a` as five. If you try to assign `a1` as five or 500, whatever that number is, that is perfectly fine.

Now we'll move on to the section of data types. Let's create some variables from different data types. Here I'm creating `a` as an integer. We can check the data type by using the `type()` function. `type()` is a built-in Python function which is used to check the data type of any Python object. So you can see that the data type assigned to `a` is an integer. Similarly, let me create a string data type. So `b = "Luxe"`. So you can very well see that the strings are created by using double quotes. You can also create strings by using single quotes. So the data type of `b` is a string. For floats, you can assign the value directly. Just make sure you have a decimal point. So here the value assigned to `c` is 5.55, and you can see the data type is float. And boolean, you can create either `True` or `False`. Okay. So you can see the data type, let's say `d = False`. So `d` and `e` are both boolean. And if you are interested to see what all variables are present in your current namespace, just type `whos` and you can see that all these variables are existing in the current namespace. Okay. You can use all these variables for calculations.

So let's do some quick data type conversions. Let me convert a float into an integer. So my age is 26.3333 something. I am converting that age into an integer by using the `int()` function. `int()` is used for type casting float to an integer. Okay. So you can see 26.333 has been type-casted into an integer. So what this function does is it simply ignores the decimal point and anything beyond the decimal point. It only takes the integer part of that float. Similarly, I can convert a string into an integer also. So if I'm trying to add a string to an integer, this will give me an error. So this is a string in single quotes, and this is an integer. So you cannot add a string to an integer. This operation will give me an error. You can see. But if I typecast this `2` from string to an integer, this I'm converting this `2` from string to an integer `2`, I can still add with this integer `6`. Okay, so let's do that. Perfect. So this is how it works. Okay. Here the `2` string was type-casted into an integer, and then this becomes integer. This is already integer, so integer addition happens, and finally, the returned value is also an integer. You can check that the data type of `my_int` is also an integer. I'll see you back in the presentation, and we'll continue with the operators in Python.

In Python, operators are symbols that perform operations on variables and values. These include arithmetic operations like addition, multiplication, division, and more. Think of operators as tools in a toolbox. They help you manipulate the data stored in your variables. For example, if you want to add two numbers, you would use the plus operator. Easy, right? An operand is the value on which the operator is applied. Python provides several types of operators: the arithmetic operators for basic math, comparison operators to compare values like greater than or equal to, logical operators for combining multiple conditions, bitwise operators for working with bits, assignment operators for assigning values, and identity operators to check if two variables refer to the same object. These operators allow us to perform complex calculations and make our code much more powerful. For example, you could use a comparison operator to check if a student's score is greater than 50 and then use a logical operator to decide what to do next.

It's time to practice using operators. Let's open our Python editor and try out some arithmetic and comparison operations.

So, we have already learned the concept of variables and data types. Now, we'll proceed to operators in Python. I'll double-click this file and open it. So, as we have already seen, there are different types of operators in Python: arithmetic operators for mathematical computations, comparison operators, logical operators, bitwise operators, and so on. Let's experiment with some of these operators. So here I'm creating four variables: `a` with a value of 15, which is an integer; `b` with a value of 8, again an integer; `c` with a value of 15.5, `c` will be a float; and `s` is "hello" with single quotes, it's a string. So let me execute this. Okay.

So these are the most common arithmetic operators available in Python: `+` is used for addition, `-` for subtraction. This is `*`, used for multiplication of two numbers. This is `/` for division of two numbers. This is the modulo operator. This will give you the remainder after division. For exponentiation, you can use two asterisks `**`. Okay. And this is for floor division. Floor division typically gives you the quotient part after division. So let's see addition of `a` and `b`. I'm subtracting `b` from `a`. So let's see some of these operators in action. So I'm adding `a` and `b`. Uh now I'm subtracting `b` from `a`. So I'm dividing `a` by `b`. So `a` is, we have seen, 15, `b` is 8. So this is 15 divided by 8. This is 15 * 8 = 120. And this is exponentiation. So 2 to the power of 3. Similarly, you can just change this number. 2 to the power of 10 would be 1024. So this is for exponential or powers. So this is floor division. So this is 15 divided by 8, and you are only looking for the quotient part, which is 1. And the modulo operator will give you the remainder. So the remainder should be 7. So when 15 is divided by 8, you'll get 1 as the quotient and 7 as the remainder.

Let's have a look into some of the comparison operators. So `==` (two equality signs) will be used to check the equality between two operands. This is for not equal to `!=`. For greater than `>`. This is for less than `<`, less than or equal to `<=`, and greater than or equal to `>=`. So I'm checking whether, let's print the value of `a` and `b` before we go ahead. So `a` is 15 and `b` is 8. I'm checking whether `a` is equal to `b`. It says `False`. Certainly, `a` is not equal to `b` because `a` is 15 and `b` is 8. Now I'm checking whether `a` is equal to 51 or not. So `a` is not equal to 51 because `a` is actually equal to 15. So here I am checking whether `a` is equal to `b` or not and checking the data type of the returned output. Here the output will be a boolean. You can see. So for all these comparison operators, the returned output will be of boolean type. You can see `True`, `False`, `True`, `False`. So that `True`, `False` is basically a boolean data type. Now I'm checking whether `a` is greater than `b`. Yes, indeed it is greater than `b`. `a` is less than `b`. This should return `False`. `a` is greater than 51. No, 15 is less than 51. So it is `False` again.

Now for logical operators: `and`, `or`, and `not`. So is `1 == 1`? This will return me `True`. Is `2 == 2`? Yes, this will also return me `True`. So if this is `True` and this is `True`, the net thing will give me `True`. So `True and True` will give me `True`. This is `True`. This is `False`. So `True or False` will give me again `True`. That is how the truth table for the `or` operator works. So `and` operator will give you `True` only when both of them are `True`. If either of them are `False`, okay, let me make it clear. So let's make it 20 here. So this is `True`, but this is `False`. Right? So this will give me `False`. So `and` operator will return `True` only if both of these conditions are actually returned to be `True`. Okay. Simultaneously, if either or both of them are `False`, then the net returned output would be `False`. You can see here. Okay. So this will go back to `True`. But in the case of `or` operation, there are two conditions again which I'm checking. So if either of them are `True`, it will give me `True`. So this is `True`. This is `False`, and still it gives me `True`. Okay. But if both are `False`, let's say this is 10 now. Okay. So `1 != 10`. So this is `False`. `3 != 4`. This is also `False`. So `False or False` will return me `False`. If either of them is `True`, then it will give me `True`. You can see that. Okay.

Now you can go for more compound operations. Okay. So you can focus on the bracket here. So first I'm checking this condition within this bracket, and then I am compounding it with the `or` operation with this condition. Okay. So first we check whether `2 > 3`. This will give me `False`. This will give me `True`. So `False or True` will give me `True`. So the output from here to here will be `True`. Now this is `True` and this is already `True`. So the net thing should return to be `True`. That's it for this coding exercise. I'll see you in the next part.

Hello and welcome back. In this video, we'll explore conditional statements, a fundamental concept in Python that helps us to make decisions in our code. If you have ever wondered how a program knows what to do based on different inputs or conditions, well, this is it. Let's dive right in.

Conditional statements help us make decisions within our code. Imagine you are coming home from work. If you are home on time, you might cook dinner, but if you are late, you might just order food. This kind of decision-making is exactly what conditional statements allow us to model in Python. We use conditional statements for various purposes such as making decisions, controlling the flow of the program, validating inputs, and even handling errors gracefully. They form the backbone of any program that needs to adapt to different situations.

Let's break down a scenario. Picture this: You arrive home. That's our event. Now, there are two possible conditions: either you arrive on time or you arrive late. If you are on time, you decide to cook dinner. But if you are late, you decide to order food. This is exactly what conditional statements in Python do. We set up conditions, and based on whether they are true or false, we determine the appropriate action to take. It's like a flowchart for decision-making. Quite intuitive, right?

Python gives us several types of conditional statements: the simple `if` statement executes code only if a condition is true. The `if-else` statement to specify what should happen if the condition is false. The `if-elif-else` statement for handling multiple conditions in sequence. And finally, the nested `if` statements, where you have an `if` statement inside another `if` statement. These different types of conditional statements allow us to write complex decision-making logic in our programs.

Here's a quick look at some pseudocode to help understand how we write conditional statements. "If our condition is that the arrival time is greater than 10 p.m., we decide to order food. If it's earlier, we cook instead." This pseudocode represents the thought process behind the decision-making. Using such simple pseudocode helps us think about the logic before jumping into the actual Python code, making it easier to plan out what we need our program to do.

Let's look at the actual Python syntax. For a single condition, we use the `if` keyword followed by a colon. The code block that runs if the `if` condition is true is indented right underneath. For example, if you have a variable `x` and you want to check if it is positive, you would write `if x > 0:` followed by an indentation of four white spaces: `print("positive")`. So, Python syntax is very simple and easy to follow. Just remember, indentation is the key.

When we have multiple conditions, you can use `elif` and `else`. Here's a simple example. Let's say you have a variable `x`. If `x > 0`, you would print "positive". If `x == 0`, you would print "zero". Otherwise, you would print "negative". Using `if`, `elif`, `else` helps us make our code readable and logically organized. It allows us to handle multiple possibilities efficiently rather than using lots of separate `if` statements.

Now, it's time for some hands-on practice. Let's open up our Python environment and try writing some conditional statements.

Before we wrap up, let's talk about some best practices. First, always try to write clear conditions that are easy to understand. Proper indentation is critical in Python. It defines which code belongs to which condition. Try to avoid deep nesting of conditions whenever possible, as it can make your code hard to read. Instead, use logical operators or refactor the code. Lastly, remember to use comments to explain any complex condition. This will help anyone reading your code, including your future self. Following these best practices will help make your code more maintainable and easier for others to understand.

So, before we start with conditional statements, let's have a quick look into indentation and comments in Python. So, indentation means providing spaces and tabs which are used at the beginning of any statement. So these statements with the same indentation will belong to the same group. So here in this code, I am defining `a = 5` and then `b = 6`, but here I have got an indentation of one tab. One tab is equivalent to four white spaces. Typically, you can see this in your settings, text editor indentation, and the default is four white spaces. So you can actually count, there are four white spaces here: 1, 2, 3, and 4. Okay. Here, the indentation is not expected, and this will give me an error. Why is it not expected? Because indentations are typically expected in control structures like `if-else` statements, `try-catch` statements, `for` and `while` loops, function and class definitions, and so on. Here you have the `if-else` statement here. So I'm checking whether `2 == 20` or not. So this will give me `False`. So I can change this to `2`. So now I'm checking whether `2 == 2`. So this will make it `True`. So if `True`, that means I'm entering the `if` condition. This print statement will be printed. Correct. But if I just forgot to give this indentation, let's see what happens. It says that "IndentationError: expected an indented block after the 'if' statement". So remember that whenever you are using the `if` statement or any of the loops and function definitions, the first line onwards, everything has to be indented. Okay. So here indentation is mandatory. This helps Python to know that this statement has to execute whenever this condition is true. Okay.

Now you can write down comments in Python by using a hash mark (`#`) before the place where you want to place your comments. Okay. So you can start your comments at the end of lines of code files, or you can also start your comments from the start of a line as well. Okay. So when I execute this block of code, this line is executed. This line is commented. Okay. So let me uncomment that. So I deleted this hashtag. And let me re-execute. So when I executed it again, you can see I get the print statement from here as well as this one. This was earlier commented. So this second line was not appearing in the output. Okay. Now I can repeat it. So I am commenting the first line and executing this block of code once again. You can see that since the first line got commented, this is not executed, and only this line got executed. Okay. So this way you can write down multiple lines of code within a code block and select which line you want to comment and uncomment. So the keyboard shortcuts for commenting and uncommenting is, you can, in a Windows system, you can click on Ctrl and this backslash (`\`). Okay. So whichever place your cursor is in, whichever line you want to comment or uncomment. Okay. And just press Ctrl and `/`, this will be commented. And again, if you do the same, Ctrl and `/`, it will be uncommented. Okay.

Now let's check some conditional statements. Okay, we have already seen. So here I'm going for the `if` statement. I'm assigning a value of 1 to `x` and checking if `x == 1`. Let's see. Yes, indeed `x` is equal to 1. So it entered within the `if` condition, and this print statement was executed. Now here, since `x` is already equal to 1, I'm checking whether `x == 3`. So you now know that this will return `False`. Since it is returning `False`, this print statement will not execute. So nothing happens here.

So now I'm having `a = 5` and `b = 10`. I'm checking whether `a > b`. So is 5 greater than 10? No. So it goes to the `else`. Okay? Because this is `False`. So this statement is not executed. So the condition automatically goes to `else`. If this is not true, the control goes to `else`. Once again, I repeat, if this is true, this will be executed. If this condition is false, the control automatically goes to the `else` block. So the `else` block has this print, and so the next print will be executed.

You can also check multiple conditions. So here `a = 8`, `b = 10`. I'm checking whether `a > 10` as well as `b > 10`. So it's a, you can say, compound condition. So two conditions are being checked. So this `if` will be true only and only if both these statements are true because of the `and` operator. Okay. So `True and True` will give me the `if` condition as `True`. `elif` here I'm checking whether `a > 5` and `b` is also greater than 5. So if this is `True` and this is also `True`, then this will be printed. Okay. `else`, if none of these are executed, then the control automatically goes to `else`. Let's see. Perfect. So here this condition got satisfied. It seems that this condition got satisfied.

Now I'm taking the user input by using the `input()` function. Okay. So the user input goes to `x` as a string. I have to convert it into an integer. Okay. So by default, the output of the `input()` function will be a string. I have to typecast it into an integer. So now `x` is replaced. Originally, your `x` was a string. Now that is replaced as an integer in the second line. Now I'm dividing that by two and checking the remainder. Is the remainder equal to zero? Okay. This is a modulo operator. So the modulo operator will return you the remainder after division. Okay. So the remainder after division is checked whether it is zero. If the remainder after division with two is zero, that means the number is even. So I'm checking here whether the number given by the user input is even or odd. Simple. So let me give some number here. Let's say 15. So 15 divided by 2. You can just manually check, 15 modulo 2 will not give you zero. So `1 != 0`. So the `if` condition was not satisfied. So automatically the control went to the `else`, and so "odd" was printed. Now I repeat this. Let me provide some big number. However big, that doesn't matter. This number when divided by two leaves a remainder of zero, and hence the print statement corresponding to "even" was printed here.

Now in the second problem, I'm going to take the user input and assign it to another variable `y`. Okay. By default, again, this `y` will be in the string format. I have to typecast it into an integer, and then I have to print the grade: whether the grade is A, B, or F based on the score provided by the user. So the grade is greater than 90. If the number is greater than 90, then I'll assign a grade A. If the number is greater than 60, I'll assign it to be grade B, and else the student fails. So let me provide some number. Let's say 95. Okay, fantastic. I got grade A. Let me rerun this with another number. Let's say 72. So what happened here? First, `y` is checked to be greater than 90. So right now, your `y` is 72. 72 is certainly not greater than 90. So this condition is not satisfied. So control automatically went to the `elif`. So this condition is checked now: whether 72 is greater than 60. Yes. So this print statement got executed, and the program stops. Okay. So only one of these three statements will always execute. Only one of these three will execute. If this is true, this will execute and stop. If this is not true, then it will check whether this is true. If this is true, this will execute and stop. If both of them are not true, then the control automatically goes to the last `else` statement, and this is always printed. So if you want, you can check it once again with another number, 45. Yes, 45 is a failure grade.

I'm thrilled to welcome you to this video on looping constructs in Python. Loops are incredibly powerful. They allow us to repeat tasks efficiently without writing tons of repetitive code. Today, we'll learn how to make our code smarter, more efficient, and much easier to manage. Let's jump right in.

So, what exactly are looping constructs in Python? These are tools that allow us to repeat a block of code as long as a specific condition is met. Think of it like an instruction that says, "Keep doing this until I tell you to stop." This can help us automate repetitive tasks, process data more easily, and solve complex problems with fewer lines of code.

Let me give you a simple scenario. Imagine Jake wants to print his name 10 times. Now, he could write out `print("Jake")` 10 times, but that's pretty tedious, right? This is where loops come in handy. Instead of writing that code over and over, we could use a loop to do this for us. Using a loop, we can tell Python to print Jake's name 10 times with just a couple of lines of code. This makes our work faster, easier, and much more elegant.

So, when do we use loops? Well, there are several situations where loops come in handy. First, repetitive tasks like our example of printing a name multiple times. Next, processing user input when you want to keep asking questions until you get a valid answer. Iterating over data structures like lists or dictionaries. Conditional repetition, where you keep doing something until a condition is met. And even simulations or games where repeated actions are a part of some experience. Loops save us a lot of time, especially when our tasks grow.

Let's take a closer look at the types of loops in Python. In Python, we have two main types of loops: the `while` loop and `for` loops. First, we'll discuss about the `while` loop. A `while` loop is used when we don't know exactly how many times we need to repeat the task. It keeps going on until the condition is no longer met. For example, you might use a `while` loop to ask a user for input until they provide a valid answer. Next, we have the `for` loop. This is used when we know exactly how many times we want to repeat something. It's great for iterating over sequences like a list of numbers or a set of items. It's predictable, making it perfect for situations where we know how many iterations are needed.

Let's talk a bit more about the `while` loop. For instance, we want to keep asking a user for their age until they provide a valid number. A `while` loop is the way to go. It repeats until the condition becomes false, then it stops. It's perfect when you need to repeat a task an unknown number of times. The `while` loop is like saying, "Keep repeating this block of code while the condition is true."

Now let's move on to the `for` loop. The `for` loop is used for iterating over sequences, whether it's a list, string, or even a range of numbers. It's great when we know exactly how many times we need to repeat something. Think of the `for` loop as

going through a to-do list one by one. You tackle each item until you have completed them all. Whether you are looping through the numbers from 1 to 10 or printing each item in a list, the for loop is the perfect tool. We also have nested loops which are loops inside other loops. These are often used when we need to work with multi-dimensional data. Imagine you have a grid of values like rows and columns in a table. You could use a nested loop to go through each row and within each row visit every column. While powerful, nested loop can be a bit tricky and it's important to be careful when using them. Too many nested loops can make your code harder to read. So, use them wisely.

All right, enough theory. It's time for some hands-on coding. Let's open our Python environment and practice both the while loop and the for loops. Thank you for joining me in this video on looping constructs. I hope you now feel more comfortable with loops and how they can simplify your code. In the next session, we'll continue building these concepts to make our programs even more dynamic and interactive.

So, let's start with opening our Anaconda prompt. So, from the Anaconda prompt, let me browse to the directory where my files are kept. Okay. And from here, Jupiter notebook. Okay. So my Jupyter notebook is up and running. I have my code files here, module one, and we'll go on to looping constructs in this video.

Okay, in this coding tutorial, we'll have a look into how to use the for and the while loop. So let's start with the for loop first. So we would use the for loop when we want to run a block of code certain number of times. So you are already clear that you will be running your block of code some specific number of times then for loop is the best choice. Okay.

So here is a simple example for number in range four. So range is actually a generator function. This will create indices from zero to the stop number minus one. So it will create index from 0 1 2 and 3. So essentially this number in the first iteration will take a value of zero. Then in the next iteration this number will take a value of 1 then two and then three. It will stop one before the stop index. Okay. So let's print this and see. So essentially I'm running this for loop for four times. That's exactly what this for loop statement says. I want to run a for loop for four iterations. So this number will pick up the values from the range generator. Okay.

Now let's say you do not want to start from zero. You want to start from some other number. Let's say in this case one. Then you can give uh two arguments to the range function. The first argument is your starting point and the second becomes the end point. As usual again remember that the number will stop or the iterations will stop once the range value has reached one before the stop point. So it will go from 1 to four this time. Okay? Because the starting number is one ending number is five. So it will end one before that.

You can also use steps. So in this case I'm providing my starting number, ending number and this is my step size. So step size is by default one. Here I have increased it to two which says that give me all the numbers from 1 to 9. It will stop one before 10. So it will be 1 to 9. Give me all the numbers from 1 to 9 by skipping every alternate number. So every second number is what you want. Okay. So let's see. So 1 2 3 5 7 and 9. Yeah. So you are printing the number at the end you are printing a space a white space that is the exact reason why it is not coming one below the other it is here the default end is back slashn which is basically a new line feed that is the default end you have changed it to a a white space which is why the subsequent iterations are printed one after the other in the same line.

Now here I want to iterate over a string using a for loop. If you remember in the first video we had seen that string is essentially also a collection. It's a collection of characters. Okay, it's also a collection. So here what I'm saying is for character in analytics with okay I want to fetch each and every character and print them. Perfect. So in the first iteration a was fetched in the next iteration n was fetched and so on. So one by one each character is fetched and it is being printed. Okay. And again if you want to use the end here you can very well go ahead and use end. So I'll just rerun this and you can see.

Okay now for the nested for loops. So nested for loop is nothing but a loop within another loop. Fine. So there will be an outer loop and there will be an inner loop. Okay. So if you see the indentation, okay, so this entire loop is indented within the outer loop. Okay, and this statement is indented within the inner loop. So this will be executed for every iteration of the inner loop. And for every iteration of the outer loop, this entire thing will be executed. Okay.

So let's see what this does. In the outer loop, I am iterating from numbers 1 and two. It will stop before three. So it will go from one and stop at two. Now the inner loop uh is going to fetch the characters in ABC. Okay. So in the first iteration, the inner loop character would be A. The next iteration it would be B. In the next iteration, it would be C. Okay. So here in the first iteration of the outer loop this will be one and then this will be a then 1 b and then 1 c then it will go from 2 a then 2 b to 2 c that's how it will be let's see perfect so these three ones are coming because of the execution of the inner loop a b and c and then the outer loop increments to two automatically the default increment is one we have already seen yeah perfect.

Now for a while loops. So here, let's say you want to execute a block of code without specifying initially how many times you want to execute. You just keep executing until some condition is met. So the code in the while loop will keep on executing as long as the while condition is set to be true. Okay, let's see.

So in the below example we have initialized a variable with a number of zero and we want to increment its value and u the loop will continue to run only if the value is less than seven. Okay, so here I have a number equal to zero and every time before executing these statements, okay, before entering into the loop, I'm going to check whether this number's value is less than seven. Okay, so let's say I start with 10 and execute this. This number is already 10 now. Okay, so what happens? The while loop condition itself becomes false and so these print statement and the increment doesn't execute. Let's see nothing happened. I am not getting any output print here. Okay.

Now let me go back to one and execute this once again. So the number in the first iteration was one. It is less than seven. So it entered the while loop. It prints the number and then increments it in the next line. Okay. The number gets incremented. So it becomes two. So in the next iteration this is two. Prints two and then it gets incremented to three. In the next iteration this is already three. So three is less than seven. again true. So it prints three. Okay. And then increments to four. Four is already less than seven. So it prints four and then it gets incremented to five and then to six and then to seven. So the moment it is incremented to seven. Now 7 is not less than seven. So what happens then? The while loop condition becomes false and so nothing gets executed. The while loop exits on its own gracefully. Yeah. So while loop will exit on its own as long as this condition becomes false the moment this condition becomes false. Okay.

So here I'm incrementing by 22. So this will jump to two units. So the same thing it's just that the while loop terminates a bit faster.

Now now let's say you do not want to continue all the iterations. Okay you want to jump between the iterations or you want to skip the iterations in between. Okay. Or you want to break out of the while loop or even the for loop based on some condition. Okay, you want to break out of the loops based on some condition or you want to jump some of the iterations. You don't want to complete that iteration. You want to continue to the next iteration based on some condition. This is what exactly we are going to see next. Okay. So some external factors may influence your programs and when this occurs you may want the program to exit the loop completely or skip a part of the loop before continuing and ignore.

So let's see the break statement first. So break statement provides an opportunity to exit out of the loop when some condition is met. Okay, let's see. So in this case I've got the number again set to zero initially. I'm going to increment this number one by one until 9. The range starts from zero goes until 9. So range value will automatically increment because of the range generator function. The moment this number is equal to five, I want to break out of the loop here. So the for loop is being forcefully broken. Okay. So let me execute this. You can see the number is 0 1 2 3 4. In the next iteration, okay, the number is five. The moment it is five, okay, this condition becomes true. So, it breaks out of the loop. Okay, and the moment it breaks out of the loop, this is not executed. So, anything please remember this. Any Python statement which comes below the break will never get executed. Okay, below the break and part of the loop. Okay, now this print is not a part of the for loop. So this gets executed. So any statement below the break and part of the main loop will not get executed. This print is out of the for loop. So gets executed.

Let's take another example. So here the user will be asked to enter a password which is already set to be data and if the password matches it will print welcome to the Python course otherwise it will print wrong password. Okay. So the user will get four trials. Okay. So uh the trial number 1 2 3 and four. Okay. The moment the trial four is reached it will warn you that this is your last trial. Okay. And uh as usual it will keep on asking for entering the password and if the password matches in any of the iterations okay it will simply print welcome to the course and break out of the loop. So now this break can get executed in the very first iteration also if the password matches. If the password doesn't match then uh the else condition will be executed which will notify the user that the password entered is wrong and automatically continue for the next iteration. Yeah, make sense. So let me execute this. So enter a password. So I am writing 1 2 3 4 5 this. So again this is wrong. So I have to enter A B C D which is also wrong. So now only one trial is left. Okay. So let me try hello. So in all the four iterations I gave the wrong password. Okay. So four iterations were set here. So all the four iterations are done. It just prints wrong passwords and the for loop is finished. Now let me execute this again with the correct password. This time it tells me welcome to the Python course. So here the password entered is correct. So it breaks out of the loop and no more iterations are done. Okay. So two iterations were remaining. The first iteration I gave this. Second iteration I gave this. So two more iterations were remaining but it break out of the for loop.

Now let's see the continue statement. This gives you an option to skip a part of the loop when some condition is triggered. Okay, let's see. So again, I have my numbers initially set to zero. I'm going to have 10 iterations from 0 to 9. And if the number is five, skip that number. So continue means skip this execution and continue to the next iteration. Okay, that's what it says. So as similar to the break statement any statement which comes below continue and the part of the for loop or while loop will not be executed. So the iterations will continue from this point. The control will go from here back to the first line. Okay. So this line which comes below continue will not get executed but the iterations continue. Okay. That's exactly what a continue statement says. It says that continue to the next iteration there itself. Okay, do not execute this and continue to the next iteration. Okay, the moment the number is five. Let's see. So number is 0 1 2 3 4 and if you see here the number is five is not printed because what happened? The moment the number is equal to 5, we continued the iteration instantly. So this print statement was not triggered.

Okay, here is another example. Okay. Uh the user will be asked to enter a password which is set to be the OTP value which is 1 2 32. And if the OTP is correct then the user will be asked to enter the name and the city. Okay. And again as usual the user will get four trials from 1 to four. Fine. So if there are four trials the moment the trial value is equal to four that is when the user is left with the last trial this will be displayed and every time the user will be asked the OTP. Okay. So let me enter some OTP. So it says try again because the entered OTP is not equal to the OTP which is set to be 1 2 32. So it says the user that try again and continue to the next iteration. So from here itself the control goes to the main for loop. Okay. From here itself the control goes to the main for loop. That's it. So no else condition is checked. Okay. Else condition is not checked. So let's say 1 2 3. And now you see my entered OTP becomes equal to the OTP. So in that case the control automatically went to the else because here the if will return false. And since it returned false, the control went to the else where it is now asking me to enter my name. So I'm just entering my name and city. Done. Perfect. So the main difference between the continue statement compared to the break statement is that our code will continue despite the disruption when the variable is evaluated and equivalent to five. Right?

Okay, finally, we come to the pass statement. Pass is essentially a null operation. It's a valid Python statement which does not execute anything. Okay, that's what it says. So the number is again set to zero and this number will have value from 0 to 9. Okay, if the number is five, don't do anything. Pass says don't do anything. If you let's say comment this statement, it will throw an error because under the if condition, nothing, no valid Python statement was executed. Okay. Now this print is a part of the for loop. it is not a part of the if condition because it is indented in the same level. Now this pass which is indented within the if is actually a part of the if condition. Okay, so that is exactly the use case here. Okay, indentation error. So if I do not have a valid Python statement, it could be anything like a simple print statement, this will work. Okay, let me Yeah, this will work. But let's say if I do not give any valid Python statement, this will again give me an error. Okay. So let's say I do not want to execute anything if this condition is met. Maybe I want to write down the code tomorrow or maybe after some discussion with my colleagues. Okay. What exactly has to be done when the number is equal to five? Right now I do not have the answer to it. Let's say in a simple example scenario. So you have created a placeholder. Okay, that if the number is five, do something. But what exactly? You don't know, you don't want to specify, you do not want to write down the code immediately. So just pass. Okay, so this becomes a valid Python statement and the iterations continue. That is exactly the purpose of pass. You can use the pass statement along with um loops, any of the loops, if conditions. You can use it in the try except blocks, function definitions, class definitions. Everywhere you want to create an empty Python construct without specifying lines of code, you can use pass.

Functions are one of the most powerful tools you can use in your code, allowing you to reuse blocks of code, making your programs more organized and much easier to maintain. By the end of this session, you will be comfortable defining your own functions, using built-in ones, and even working with Lambda functions. Let's get started.

So, why are functions important? Imagine you had to bake cookies and every time you did it, you had to measure all the ingredients from scratch. Instead, wouldn't it be easier to just have a recipe? Functions are like recipes in programming. They let you write code for a specific task once and then reuse it wherever you want. This saves you time and also helps to avoid mistakes. Let's look at a scenario. Say Anna wants to send a personalized greeting message to a list of 50 people. She needs to make sure that each message is consistent except for the name of the recipient. One approach is to write out the message for each person, but that's a lot of repetitive work and it's easy to make mistakes. Instead, we can use a function to handle this task. By defining a greeting function, Anna can simply call the function for each recipient and personalize the message automatically. This makes the code much more efficient and prevents mistakes.

Let's break down what a function is. In Python, a function is a block of code that performs a specific task. You define a function by using the def keyword followed by a function name, input parameters, and the code that should run inside the function. Here's a simple example. Let's define a function called greet that takes a name as a parameter and prints a greeting. In this example, greet Anna will print hello Anna, how are you today? Once defined, we can call this function as many times as we need, saving us from typing out the greeting each time.

Python offers several types of functions. For example, built-in functions like print, length, and sum. Userdefined functions which you can create yourself for custom tasks. Lambda functions which are small oneline functions used for simple tasks.

Python provides a number of built-in functions that are incredibly useful. You don't need to import anything to use them. They are ready to go. Some common examples include for input output functions like input and print. For data type conversions, functions like int, float, string, these let you convert between different data types. Mathematical functions like sum, max, abs. These help with basic calculations. For example, if you want to find out the largest number in a list, you can simply use max and then it will return some number. There are other very important functions like length, range, and type.

Let's move on to userdefined functions. These are functions you can create to perform specific tasks in your own programs. To create a userdefined function, you use the dev keyword followed by the function name, parameters, and the code block. Let's say you are creating a program to greet a user. Just write a function and call the function with the username. It's that simple.

Next, we have the lambda functions. Lambda functions are small anonymous functions that are typically used for simple tasks. They are defined using the lambda keyword and can contain only a single expression. Lambda functions are often used along with functions like map, filter and sorted. For instance, if you have a list of numbers and you want to filter out only the even ones, a lambda function can make that very easy.

It's time to put what we have learned into practice. Let's write some functions in Python. Thank you for joining me in this session on functions in Python. I hope you now see how powerful and useful functions are for making your code more efficient and organized.

Now let's have a look how do we create functions in Python. As already discussed the purpose of functions is to group a particular line of code or few lines of code that needs to be executed multiple times. So we'll be using the defaf keyword to create the function definition that should be followed by the function name. So here def is used to create the function. The name of the function here is welcome. So what does this function do? This will print welcome to the Python course. So let me create the function here. Now calling the function is simple. You just have to take the name of the function followed by the parenthesis. So let me call the function here. So it prints welcome to the Python course. Again I'm calling the function. It again prints. So as many times I keep calling the function. Okay, this will execute all the lines of code within the function definition and so on.

Now I'm creating another function here which is going to add two numbers. Okay, these two numbers are already set as 3 and 4 and the total is 3 + 4 7 and it will print the total. So let me create the function and create another function. We'll see what is the difference between these two. So let me execute the function. You can see the print statement got executed from here and the total is printed as seven. Okay, you can change these numbers and see uh this is 30 + 4 now. So this will become 34.

Okay, now I'm assigning the output of this function to a variable here. Okay, let me print the value of the output. It doesn't print anything. Okay, let me see the data type of this output. It says none. So essenti it seems that nothing was stored in the output one that's why the output data type is none. So nothing was stored here. It's a null pointer. Essentially this is a null pointer. Nothing was stored here. So I'm slightly modifying the main function to return the value of the total. So this is the additional code. So till here it was the same. This is the additional code which returns the value of the total. So when I save the output of the function with that return into another variable. So the output of the print is executed and the output is also stored. So whatever the function returns in this case it is returning the total that returned value is stored in the output variable here. Now in the previous case the function did not have the return statement. So the function was not returning anything. So nothing was saved here. And now here the output value of the total is returned. So that returned value is saved and that is what we are able to retrieve.

Till now we created two functions which did not need any input parameter to execute. Now I am rewriting that function which will take the name of the user as the input parameter and use that name to construct this string and this will be returned by the function. So let me try with my name. It says welcome to Python course prashant. So this string was concatenated to the original string and the end you can see there is a white space. So this is name is essentially a string and this plus sign is basically doing a string concatenation. Okay, that concatenated string is finally returned by the function. You can actually save the output and verify. Okay, so output equal to welcome let's say Harry this time. Let me save. And uh output is retrieved here. Perfect.

Now you can also specify the default values of the input parameters. Okay. So here in this case the name has got a default value of. Okay. So you can see if you provide the name of the input argument that will overwrite the default value. The advantage of the default value is if you forgot to provide any input parameter to the function while calling the function the default value would be used. Okay. So let's see. So it still says otherwise without providing the default value this empty function call would have returned you an error. So I'm just changing this function so that the default value is removed. And now you can see this one works because anyway it is taking some input argument which is mapped to name. But here this will throw an error. One required positional argument is missing. Okay. And that required positional argument is name. And since the value of the name is missing, this is the error you are getting. Okay. So to encounter such scenarios you can provide some default values that can be used if the required value of a particular input parameter is not provided at the time of the function call. Okay. If you made changes to the function do not forget to execute that block of code again so that the updated function goes to the memory and then you recall it.

So just now we saw the concept of positional argument. Now there are two types of input argument. One is positional arguments and one is the keyword arguments. Okay. So what is the positional argument? Let's have a look into more details. So this is one u function which I'm creating math functions and this takes two numbers a smaller number and a larger number. So what does it return? It returns the difference between the larger number and the smaller number. Very simple function. So 4 and 16 and 16 and four. If you can see clearly the order in which you provide the values matter a lot. Okay. Because the difference between four and 16 is basically so if you see this four gets mapped to the smaller number and here 16 gets mapped to the larger number. But in this case what is happening? 16 is mapped to the smaller number and four is mapped to the larger number. So the same sequence is followed as it was declared in the function definition. Okay, that is exactly the concept of positional arguments. So the position of the input arguments matters. Okay, in which position you are giving the numbers, the same positions are followed here. Okay, so now let's have a look into the keyword arguments. So if you are defining the keywords then we can define the input arguments in any order. So now you are saying larger number take it to be 16 smaller number take it to be four. So here I'm providing the larger number before the smaller number. You can also provide the smaller number before the larger number. As long as you are assigning the values based on the exact name of the input arguments. Now this cannot be any variable name. This has to be the same variable name which the function expects. Okay. So if you're defining it as a and b here also you have to provide it as a and b. Okay. So you have to remember that. So advantage of doing this is you can see since larger number is mapped to 16 whether at the start or at the end doesn't matter the difference between the larger number and the smaller number is still 12. Okay. So this time a consistent output is returned even though the sequence in which the arguments were passed is different. Okay.

Now the problem starts when you try to mix and match. Here you have a keyword argument and here is a positional argument. This will throw you an error saying that the positional argument follows a keyword argument. So if you are passing an input, if you're providing an input to the function while calling without mapping it to the exact input parameter name, that becomes a positional argument. All the positional arguments have to be defined before any of the keyword arguments. So here since you are passing the value of four by defining the variable name to which it has to be mapped, this becomes a keyword argument. And here 16 is not being specified which variable to be mapped to. So this is actually a positional argument. Simple. So that has to be done in the other way around. Okay. So here four is the positional argument and 16 is a keyword argument. And then that's exactly what this statement says. All the positional arguments have to be defined. Okay. Before the keyword arguments, correct? So this will again work.

Now let's have a look into the scopes of the variables within the functions and out of the functions. So what do you mean by the scope of a variable? So this is the part of the program where we can access a particular variable. Okay. So all these variables that you have created in your notebook right now can be accessed from the global name space here. So you can see the global name space has all the functions in the memory. These are the memory locations in which all your functions are right now present. Okay, along with that all the variables that you have created is also present here. Your name output 1 2 3 and welcome. So all these are the functions and variables that you have created in this particular notebook. So this is the scope the global scope of this notebook. Okay. So this is where your all the variables are stored.

So global variable and local variable. So each function has its own local scope or name space where it stores the variables and those variables are called as the local variables. So first let's see I have got a variable called as name which is defined outside the function. So by default any variable which is defined out of the functions they are in the global namespace. Okay, they are in the global namespace. Global namespace means this is a global namespace because all these variables were defined out of the function. So this will go into the global namespace. All the functions which are present in the global namespace. You can see that the there are the functions which we created. So these functions present in the global namespace will have access to all the variables present in the so these functions will have access to all these variables. All these functions will have access to all these variables in the global name space by default. So here this name is accessible to this function and that is why this function was able to return that name or you can also do some computation. So let me multiply that name by five and or by two and see what happens. So let me call that function. You see this is replicated three times. Multiplication of a string with a integer means replicate that string three times essentially. Okay. So this is the first time, second time and then the third time. So the string gets replicated three times. I'm able to access the variable which was created out of the function. And further if you see the value of that variable didn't change. So within the function it is computed I mean it is operated but out of the function it still remains the same. So I'm printing the value of the name out of the function and it still is the same.

So let's say you want to change the value of this variable within the function. So this name will be present in the global name space. So this is essentially a global variable and I have created another copy of that name within the function. Just some time back I mentioned that each function creates its own name space. So this is the name space within the function. I'm creating this variable. So this variable becomes local to this function. So essentially this is this name is local to this function. Okay. So this name goes in the local variable name space. Okay. This name is in the global variable name space. Okay. So function can overwrite. So this function has overritten uh the name. Okay. And uh it creates a local variable with the same name. So this will no longer be accessible. The function will only access this and return that. Okay, perfect. So if you see the value of the name is already the same as the variable outside the function. Okay, so the function can have right now the readonly access as in this case you can see it had a readonly access but let's say you want to change or update the value of the global variables. Basically you want to have a write access also to the global variables. For that you need to define that variable which you want to provide this function as a read write access. Okay. So that should be defined within the functions space or the local uh variable space as a global. So this global keyword enables this function to have a read write access to this global variable. Now, so this name will be modified by this function. So I'm modifying the value here. And when I call that function, you can see the value of the name has changed. So this was defined out of the function. And within the function, the value was changed. And now I'm accessing the value of the variable out of the function. And it still gives me the value which was defined within the function. So that is how you can create a global variable which can be shared by even multiple functions. So each function can still share the same value in a readr write access mode provided that you provide the variable with a global keyword in all those functions where you want to share in a readr mode.

Welcome to this session on modules and packages in Python. Modules and packages are incredibly useful tools that help us keep our code organized, reusable, and easy to manage. Today, we'll explore what they are, how to create them, and why are they important for any programmer, especially as your projects grow. Let's dive in.

Let's begin by understanding modules in Python. Simply put, a module is a file containing Python code, typically functions and class definitions, which we can reuse across different programs. It helps us organize related code in a logical way. The file extension for a Python module is py. To create a module, all you need to do is create a new Python file. For example, my_module py. Write your Python code in this file and then save it. It's ready to use. Modules are like drawers in a cabinet. They help keep related items grouped together, making it easier to find and reuse them when you need them.

Imagine Redford School has been tasked with developing a school management system. This system needs to handle student management, teacher management, fee calculation and grading. Initially, they might put everything into a single file. Let's call it as school management. py. But as the number of features grows, this single file becomes huge and unmanageable. It becomes hard to find specific pieces of code. And making changes without breaking something else is a nightmare. This is where modules come in. Now, instead of one big file, Redford School decides to break down the school management system into smaller logical modules. Student management. py for student related functions. Teacher_management. py for teacher related functions, fee calculation. py for handling all fee related tasks, grading system. py for managing grades. This way the code is modular, easier to manage and much less errorprone. If you need to make a change to the grading system, you can just open the grading system. py without worrying about the student or teacher management code.

Now let's see how to use these modules. All the different modules for student, teacher, fee and grading can be pulled together in a main script py. The script coordinates between all the modules making it the control center of the application. Now imagine if the school notices that the management system keeps growing, they can further modularize by breaking functionalities into more modules. This flexibility helps keep code organized as the project evolves.

When modules are not enough to manage the growing code base, we turn to packages. Imagine Redford School decides to expand its system further. They create a student management package with modules for enrollment, attendance, and profile management. They also have packages for teacher management, fee management, and more. Each package represents a logical area of the system and each module inside handles specific tasks related to the area. It's a lot like how a department store is organized. You have different sections for clothing, electronics, and groceries.

In Python, a package is essentially a folder containing multiple module files plus an underscore_init_. py file which tells Python that this folder should be treated as a package. Packages can even contain subpackages which are simply folders within folders each with their own init py file. For instance, our school management package could contain subpackages for student management, teacher management, and fee management. This allows us to scale our codebase in an organized and logical way making it easy to find and maintain different parts of our system.

So how do we use packages in our main script? It's quite simple. You refer to the package name, subp package name and then the module or function you want to use. For example, if you want to use the enrollment function from the student management package, you might write something like from school management dot student management dot enrollment import enroll student.

Thank you for joining me on this journey to understand models and packages in Python. I hope you now appreciate how important these tools are for organizing and managing your code. They help keep your projects clean, efficient, and scalable, making your life as a programmer much easier. Keep practicing. Try creating your own modules and packages, and I'll see you in the next session.

In this video, we'll learn about the Python best coding practices. So, the Python best coding practices are needed to have good code readability, maintainability, and the best efficiency for the codes. So we'll be following some guidelines especially the PEP 8 and PEP 257 style guidelines that will help us to write cleaner more Pythonic and AI development friendly ready code. So let's get started.

So in this notebook we are going to cover these four topics. Uh how to write down clean codes in Python, effective use of comments and then how do you write Pythonic codes for AI development. we'll take some uh 9 to 10 examples and then we'll take some examples of PEP 8 and PEP 257 style guidelines.

So for the first one, how to write down clean code in Python. I'm giving a simple example where I'll be demonstrating a bad code followed by a good code. Okay, let's take the first example here. So in this first example, I am demonstrating how to write down meaningful variable names and function names. Okay, so in the bad code, can you see that the function name is very short and the variable names are just a and b. So we should avoid single letter variable names and uh the function names also should be self-explanatory. Okay. So although this is a working code you can just check but a better version of this code would be write down a proper function name which is self-explanatory illustrative and whatever the inputs or variable names that you are using that also has to be slightly more descriptive semantic. Okay so you are providing two numbers as the input. So let the inputs be called as first number and the second number and then you can do any computation on these two numbers and return them. So here this function is more descriptive compared to the cal function and the parameter names are also more descriptive.

In the second example we are demonstrating that a bad code would have hardcoded values of the constants which can make it difficult if you want to reuse the code or if you want to debug the code. Okay. So for example here you want to return the discounted price. So the function name is good calculate the discount and it takes the input as the price and the 10% of that price will be the total discount. But the problem here is that 10% discount is hardcoded. So later on if you want to change that discount rate you might want to dig down where that function is present and then back to this variable. So not a good idea. uh can we provide that discount rate as a more like a global variable and then use that because functions by default can access all the global variables. So you are accessing that discount rate which is set to be a constant in this notebook let's say and that uses the discount rate to calculate the total discount. So that is more readable and uh easier to change because now you just have to change the value present in this variable.

Now I'll demonstrate how to write down proper comments and commented blocks of code. So even before going to that there is one bad code example which tries to comment every single line of code even though the code might be quite obvious. This is also not a good idea. So for example if you see in this case x= 10, y= 20 and z= x + y. So these are the comments assign 10 to x assign 20 to y and then add x and y. So it's very clear that I'm assigning x as 10 y as 20. So there was no need of these comments actually and this operation is also quite self-explanatory right. So in a good code you don't need to actually uh so this is again a constant birth year current year and I am trying to get the difference between the current year and the birth year to calculate the age. So this calculation may not be very intuitive. So that's the reason I have uh provided a comment here. But I skipped providing the comments in these two lines because anyway it's quite self-explanatory, right? It is pretty obvious code. So ideally comments have to be added in your code when you are trying to explain why something is being done.

Another example uh you have written a function here and a good code would give a quick idea about what that function is doing. So this is called as dock string. You can see three double quotes start and three double quotes end. So anything in this anything between this block that is called as dock string. Okay documentation string in short as dock string. So ideally in the dock string you should be able to provide what that function does and what kind of input it takes and what computation it does and what kind of output it returns. So everything should be documented in the doxing standard. So we'll see more details about the docking in the example in the later part of this notebook. So currently I've got a function which at least gives us an idea about what that function does. Okay. So let me run this code and show you the advantage of having this documentation string. So I am just using calculate area and uh if you see now I'm pressing in the keyboard shift tab okay or shift tab tab shift tab will give you a quick dock string about that function. So without this without this dock string no it's very difficult to figure out what that function is doing. Okay. So let me just remove this and uh show you that function again. So now again I'm doing shift tab tab. If you see dock string no dock string is provided and that is exactly what I was trying to convey. So it's a good idea to provide the dock strings so that we get to know what that function does.

Now let's take some more examples of pythonic guides uh for AI codes. So in the first example we see that we should be using list comprehensions more frequently instead of using nave simple loops for data transformations. So let me give an example here. Um I have created an empty list okay and I want to store the squares of numbers in that list. So how do I do that? So what I have done here is I'm iterating over a range of 10 numbers from 0 to 9 and for each of those number I'm storing or appending the square of that number into that empty list. So one by one with these iterations the numbers would be squared and appended to that empty list. Okay. So at the end I'll get a list containing the square of 10 numbers from 0 to 9. So we can condense these three lines of code in a single line by using list comprehensions. So this is my list comprehension here. So what I'm doing here is creating a list. You can see that square bracket means creation of a list and use the for loop as usual. And for i in range 10 save the value of the i square in that list. So you are creating the list and iterating over that range of 10 numbers in the same line of code. So this is what we call as a pythonic code. So these kind of pythonic codes are preferred over these kind of nave for loop codes.

Another example here is let's say you want to access both the um index as well as the value of an iterable. Here my iterable is a list containing three alphabets here A, B, and C. Okay, this is a list and I'll be iterating over this list. So I want to have access to the index which is 0 1 and two as

Well, as the values. So this is a for loop. I'm iterating over that data for item in data. Print the index, item. I'm initializing the index to be zero and then I'm manually incrementing this index inside the for loop after accessing that index. So this is a very nice way of accessing the index and incrementing it.

A better way is to use the enumerate function. So for item, for index, item in enumerate data. So what does the enumerate function do? Enumerate function will create an index plus it will give you a way to access the value directly. So it generates two things. It is also a generator function like range, but range function only generates numbers. The enumerate function generates two things. One is the index, which is numbers, and then the values are fetched from the data itself. So when you run this, so let me just run this. So the outputs are same, but this is much more Pythonic. Okay, this is much easier to comprehend. So the enumerate function actually returns you a tuple. The first element of the tuple would be index. The second element of the tuple would be the item. You are directly able to access the item as well as the index of that item.

Now, some more examples of PEP 8 and PEP 257 style guidelines. So let's start with the PEP 8 first. So PEP 8 talks about how to write down better Python codes in terms of nomenclature of the variables, of the functions, of the classes. How many spaces you should be using? Where should you be using those spaces? How many spaces you should be using for indentation? Where should the indentation should be provided? How nesting or how deep nesting indentation are done and so on. Okay. PEP is a Python Enhancement Proposal. So you can check the PEP 8 guidelines in detail by following this link, and this has been written by none other than Guido van Rossum. He is the creator of Python language. Okay. So you can get the details about the PEP 8 guidelines from this documentation page. So here I'll be talking about the most important style guidelines.

First is about the variable naming conventions. So again, I have already repeated, but once again I repeat here. So do not provide variable names which are just single letter variables. So they are unclear and very difficult to make out what they signify, what kind of data it holds and so on. Okay. Write your variable names which are more descriptive and semantic. So that is what we call as the snake case. Uh, basically separating multiple words within your variable names using underscores. So if the width is 10, height is 20, the area of the rectangle would be width times height instead of simply writing x, y, and z equal to x multiplied by y. So this is much more descriptive and much more meaningful. So later on, when you come back to your code, by looking at this piece of code, you instantly know what you are doing, what calculation you are doing, where exactly you are going to use that calculation.

Function naming conventions. So all the function names must start with lowercase always. So and you should use again a snake case, okay, instead of using camel case. So camel case starts with a capital alphabet and in between you have more capital alphabets. That's what we call as the camel case. So camel case is reserved for naming classes in Python and not functions. For functions, we should be using again snake case. So if you want to use calculate area, so this is also not a very good function name. Why? Because this is again too short to understand what kind of calculation is happening. Right? If you wanted to calculate area, write down calculate area, and this could be simple calculator, income calculator. It could be EMI calculator. It could be just any other random calculator. Right? So this is a better way of writing the function name. Or if you are calculating the total price of an item, okay, write down completely calculate total price. And the function names cannot contain spaces. So substitute the spaces with underscores.

Again, uh, for naming the constants, try to use uppercase. As you would have already noticed in one of our first examples, we used the discount rate in capitals. So if these are defined as constants in your notebook, try to have capital letters for those constants.

So the class name should use camel case instead of snake case or lowercase. So this is bad nomenclature. So P should be capital here, and we should also avoid using underscores. Okay. So this is a camel case. P is capital. D is capital. So avoid using underscores. Although this is not wrong, Python will not throw an error in this case, but it is against the PEP 8 style guidelines.

Next example, uh, we should be adding a single space around the operators for better readability. If you see here, uh, there are three operators being used. Okay, one is the equality sign, the other is the plus sign, then the last is the multiplication sign. So these three operators, but there are no spaces. So ideally, we should be using spaces. Enough spaces. Enough means just one, not too many. So you should be separating the operators and the operands, okay, by a single space at least on both sides.

Another example here is about the indentation and the line length. So in this case, if you see, we should be using consistent indentations. So all the indentations should be exactly four white spaces, and the total line length should be less than 79 characters. So if your line tends, including comments, to exceed 79 characters, better break that line. Even that means breaking the comments, that's perfectly fine and acceptable. So functionality-wise, both are same. Functionality-wise, both are same, but make sure that you are using exactly four white spaces for the first indent, and then eight white spaces for the next indent, and so on. It cannot be three white spaces and then four white spaces. Uh, you cannot randomly increase or decrease the number of white spaces here. So if you see, if you try to run this code with inconsistent number of white spaces, this might even give you an error.

So you should also use blank lines while defining functions and classes. At least two blank lines. Here I've defined a function and a class, but there is no space between the class definition and the function definition. So here I've got two line breaks. Okay, before I start defining my, okay, so again here, uh, the class names, which is again a machine learning model, that is my class name, that should again be in the camel case instead of the snake case, which is the naming mistake again we are doing here. And, uh, if you see the function definition, that is in the camel case. Rather, that should have been the snake case. So you cannot interchange them.

Another very common mistake people do is trying to overly use the equality operator. Okay. This is to check, uh, equality of something, whether a variable is equal to something or not. So let's say your variable value is, uh, 10, and if you want to check whether that variable is none or not. Okay. So instead of using this, it's better to use "if variable is none." Okay. Instead of checking that the variable's value is none, check whether the variable itself is none or not. So this is a better way of handling. If you want to check whether it is being compared to none or not. If it is being compared to some other number, then it is fine. Uh, if you want to check it with respect to let's say 100, then this code is perfectly fine. But if you are checking with respect to none, then the better way is to go with this.

Finally, we should avoid importing any unused modules. So, for example, in this code, I want to find out the square root of a given number, and I'm defining a function for that. So here you can see, uh, I have imported the math module, the OS module, and the random module, and I'm not using any of them. So this is not a good way of writing code, simply importing the modules which keep occupying the memory. So a better way is, if you can reproduce the functionality by your own code or by some custom functions, that is the best. So in this case, only this function remains in the memory instead of all those redundant modules which we are not using anyway. So in this case, if you want, you can also import the math module, and actually the math module is being used. So this is also acceptable.

In this last demonstration here, we are going to demonstrate the PEP 257 style guidelines. So, PEP 257 talks about the docstring standards for Python coding. So, typically, uh, the docstrings should describe the purpose, the input, and the output parameters of that function, and what it returns. So everything that you see just below the function definition, which is present within these three double quotes, this is called as docstring in short, or documentation string. If you define a function without the documentation string, it's still usable. You can see I'm able to create a function which calculates the volume of a cylinder by taking the radius and the height of the cylinder as input, and I'm able to call that function by giving some value for the radius and the height. However, when I do a shift tab tab to check the function's documentation here, you can see very clearly there is no documentation string mentioned because we deleted that documentation string.

Now, let me post it back, uh, reinitialize the function, and then use the function. So, the functionality didn't change at all. However, if you see now, I'm going to do shift tab tab by placing my cursor within the function. So this balloon pop-up appears which talks about the docstring. So this gives me a clear indication what this function does, in this case, calculating the cylinder's volume using a radius and height as the input. What are the input arguments and their formats, and what the function returns, in this case, which is clearly the volume of the cylinder in the float format. Right? So this is the appropriate way of, uh, defining the functions and further we are using them. So that's all for this video. Thank you.

Hello everyone, and welcome to this session on NumPy. NumPy is a cornerstone of Python's data science ecosystem, providing the ndarray object for efficient array operations and a vast range of numerical functions. In this video, we'll explore how it works, why it is essential for data science, and key features such as universal functions, random sampling, and linear algebra routines.

NumPy stands for Numerical Python, and it's built around the ndarray, the N-dimensional array object data structure. So unlike pure Python lists, ndarray objects are much more efficient and support vectorized operations, meaning that you can apply operations on the entire array without writing a single loop. And this makes your code run faster and keeps it easier to read and maintain. Plus, because NumPy is so fundamental, other popular data science libraries like Pandas for tabular data, or SciPy for scientific computing, and even Scikit-learn, which is used for machine learning, they heavily use NumPy objects under the hood.

NumPy's universal functions, or ufuncs, perform element-wise operations across arrays. For basic arithmetic, for example, addition, subtraction, multiplication, and division, it is very straightforward. We call either the np.add function or simply write a + b. We also have exponential and logarithmic functions that operate element by element. Now, this level of vectorization is fast and makes our code concise and also readable. If you're dealing with any kind of scientific or engineering data, trigonometric functions are likely to come up. NumPy offers a full suite of trigonometric functions like sin, cosine, tangent, and so on, and their inverse counterparts, all of which work element-wise. Now, same goes for the hyperbolic functions as well. So whether you are modeling waves, or working on signal processing, or even analyzing periodic data, NumPy has got you covered.

Sometimes you need to clean up your data by rounding values. Maybe you are dealing with currency, or you just want fewer decimal places, and that's where these rounding functions come in handy. So you can use the np.round function, which will round off your number to the nearest decimal place. The np.floor function will round off to the lower integer. np.ceil function will round off to the higher integer, and so on. So you also have the np.sign, that is, sign function, which returns the indication of whether a number is negative or positive. If the number is negative, it will return -1. If the number is positive, it will return +1. And if the number is zero, it will return zero.

Now, these comparison and logical functions make it very easy to generate boolean masks. So, for instance, you have the function np.less. So this returns an array of true or false values depending on whether each element is satisfying a condition, maybe less than some number. So you can use these boolean functions and arrays to filter data and apply conditional logic. There are bitwise variants which are very similar but act at the bit level, which can be very useful in certain low-level and specialized tasks.

No data science project is complete without computing summary statistics. So NumPy's built-in stats functions are very straightforward. You have the np.std function, which calculates the standard deviation of the NumPy array. You can also specify an axis for these functions to compute these statistics, whether you want it row by row or column by column, making it really very flexible for multi-dimensional data.

Moving on to the heart of NumPy, the core functionality. This is where you learn to create arrays using basic functions like np.array, np.zeros, ones, and so forth. When working with different shapes, you can use the np.shape, reshape function to change the shape of your vectors and matrices. You can use the ravel function to flatten your arrays and matrices. And finally, the transpose function to flip the axes, rows into columns, and so on. There is a dtype argument which can be used to fetch the data type of the elements present within the NumPy array, whereas you have the astype function, which is used to change the data type of the elements within the NumPy array. And that is also called as type casting. Mastering these basics is a key to getting the most of NumPy.

Broadcasting is one of the most powerful features in NumPy. It's, it lets you perform arithmetic operations between arrays of different shapes by automatically stretching them along any dimension with length one or no dimension. So, for example, if you add a scalar, let's say five, to a two-dimensional array, NumPy effectively replicates that scalar across the entire shape of the array to make that operation successful. Let's consider another quick example here. Here you have an array A which has a shape of (3,) and B which has a shape of (3, 1). Essentially, A is one-dimensional, whereas B is two-dimensional with three rows and one column. So these numbers 10, 20, 30 are present in one column. It's essentially a column vector. Whereas A is a one-dimensional object. So now, if you want to have an operation between them, let's consider A + B. Okay. So you are adding a one-dimensional NumPy array with a two-dimensional NumPy array. So in this case, NumPy will stretch A or B as needed. So the shapes become compatible for the operation to be successful. So the final result is a 3x3 matrix. So when used correctly, broadcasting saves a ton of code and can keep your calculations very efficient. So it is also important to understand the rules of matching these shapes from the trailing dimension so as to avoid mismatch and errors.

Now, let's talk about linear algebra. So many data science or machine learning algorithms involve vectors and matrices. Think of large datasets or transformations or solving systems of equations. Now, with NumPy's linalg, or the linear algebra submodule, you can multiply matrices using the np.dot function or a very convenient operator, which is the at symbol. You can also solve systems of linear equations like Ax = b using the np.linalg.solve function instead of just coding up your own solver. And if you need to find the inverse or a determinant of a matrix, you have to choose those functions which are readily available in the linear algebra module. So for more advanced use cases, you have the eigenvalues and vectors via the np.linalg module. You can also do the singular value decomposition by using the np.linalg.svd, which is very critical in methods like principal component analysis for dimensionality reduction. You can also find vector and matrix norms using the linalg.norm function.

While NumPy doesn't have a separate stats submodule, it does include quite a few statistical functions right in its main namespace. So, for example, np.mean can calculate the average value. You can specify an axis to do so for each row or for each column in a two-dimensional array. Similarly, you have the np.median function, np.quantile function that helps you understand the data distribution and so on. While np.std and np.var, these functions calculate the standard deviations and variance of the data, that helps you understand the spread of your data. So this makes NumPy a good first stop for exploratory data analysis before you move on to more advanced tools.

If you need to understand the relationship between the variables, for example, calculating the correlation coefficients, you have the np.corrcoef function, and you can also calculate the covariances between these matrices by using the np.cov function. You have the histogram function, np.histogram, that helps you bucket the continuous data into bins for a basic frequency distribution. Now, these functions, while simple, can be incredibly powerful for a quick look at how your data behaves before diving deeper with specialized libraries like Pandas or Seaborn for visualizations.

Finally, there is the numpy.random module, which is where we generate random numbers for simulations, randomized algorithms, or even test data. The recommended approach is to create a random number generator using the np.random.default_rng function. You can provide a seed, whatever number, the default is anyway 42, and this makes your results quite reproducible, which is critical in many scientific and data science contexts. You can also sample from various distributions like normal, uniform distribution, or binomial distribution, and many more. Additionally, you have the functions called shuffle and permutation, which come in handy when you need to shuffle your data, like randomizing the order of the rows in a dataset or even create permutations. Whether you are performing Monte Carlo simulations or just need a quick sample of your data for testing, the numpy.random module is the go-to function of your choice.

This wraps up our exploration on NumPy's key features from array creation to broadcasting to universal functions, linear algebra, and even random sampling. Understanding these concepts will help you to write faster, cleaner code and give you a strong foundation in Python-based data science.

Hello and welcome back. In this video, I will demonstrate how do we use the NumPy functions. I'll show you how to create and manipulate NumPy arrays and matrices. We'll also perform vectorized arithmetic operations, use some statistical functions, and we will see how broadcasting works in NumPy. We'll also take some of the functions from linear algebra and the random modules. So let's get started.

So first, we will need to import NumPy. NumPy typically comes pre-installed in any Anaconda installation. Whether you are running the Jupyter notebook or Colab, NumPy comes pre-installed, so you don't have to really bother about installing NumPy. So let's import NumPy as an alias np. And for reproducibility, especially while working with random numbers, I'm using a default seed of 42. The NumPy version that I'm running in this notebook is 1.26.4. Okay, let's see some of the array creation functions. So, we'll be using the np.array function primarily for creating the arrays. Along with that, we can use zeros, ones, arange, and linspace functions. Let's see each one of them one by one.

So, here I've got a list of numbers: 1, 2, 3, 4, 5. This is a normal Python list. I'm going to convert this normal Python list into a NumPy array. Okay, by using the np.array function. So this is kind of type casting a Python list into a NumPy array object. We can verify this by using the type function. You can see the data type is numpy.ndarray. So it's a NumPy N-dimensional array object. It's no longer a list. Right?

So now, let me create a two-dimensional array of zeros this time. Okay. So I'll be using the np.zeros function where it will have two rows and three columns, all filled with zeros. So these are the two rows, one and two, and three columns. Similarly, I can create another two-dimensional object by using the np.ones function. So again, this is going to be two rows and three columns. Okay, all filled with ones. And you can also check the dimensionality by using the function. So let's see array_ones.shape. So you can see that it is having two rows and three columns. So there is another function to check the dimensionality. So that is ndim. You can see the dimensionality is displayed to be two. So you can check the ndim argument for any NumPy array to find the dimensionality of that NumPy array. Okay.

So now I want to create an array with another function, arange. What this function does is it takes a starting number, which is in this case 5, and then there is an ending number, which is 15, and the step size of two. So it will start giving you numbers or it will create an array starting from the number 5, and it will stop one before the ending number. So 15 will not be included in that array. Okay. So 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 would be the last number, technically. But then you have said, give me a step size of two. So which means it will skip every alternate number. Let's see what we get after I just delete that two. So that is exactly expected from 5 to 14. But if I get that two back, it will skip every second number. So I'll get 5, and then I'll get 7. I'll get 9. The alternate numbers are skipped. Okay. So I'll get 5, 7, 9, 11, and 13. That will be the last number. Perfect. Okay. So, step size you can set to be 2, 3, 4, whatever that number, that many numbers would be skipped. Okay.

Linspace on the other hand will always make sure that the starting number and the ending number are always included in the NumPy array. So this time I'm going to create a NumPy array which has both 0 and 1. And how many numbers do I want in that array specified by this five. Okay, instead of specifying the spacing between the numbers, you specify how many numbers do you want in that array, which includes 0 and 1. Okay, so Python will back-calculate the spacing, and you are talking about the linear spacing between these numbers. Okay, so let's see. So you can see that 0 and 1 both are included, and Python back-calculated three more numbers such that the spacing between all the five numbers is equal. That is the linearly spaced numbers in that array.

Let's see the concept of indexing and slicing now. So with or by using indexing and slicing, you can access the elements within the NumPy array. And if you want, you can actually even modify some of these elements. Okay. So let me create a simple NumPy array: 10, 20, 30, 40, 50. It's a simple one-dimensional array, and the array elements are accessed by using square brackets. So this is my indexing bracket. If you see, this is my indexing bracket, and the element that I want to access is having an index of two. Now, here I have written that Python is zero-indexed, which means the first element in that array will have index zero. This will have index one. This will have index two, and this will have index three, and this will have index four. So the last index will always be the length of the total array minus one. Okay. So in this case, I'm going to get 30 because that is the second index, basically. So 0, 1, and 2. This is 0, 1, and 2. Okay.

If you want, you can also specify the starting and the stopping index, and optionally even the step size while indexing as well. So in this case, I have given my start index as one, the stop index as four, and again, the stop index will never be included in the slice. It will always stop one before the stop index again. So here I'm going to get the numbers which has the index one, index two, index three, that's it. Okay. So three numbers I'll get in the output. Yeah. So that is 20, 30, and 40. So 20 is my index one, 30 is the index two, and 40 is the index three. So in this case, colon, so this colon means take the entire array and give me every second element of that. So since you have not provided the starting index, it will start from zero. Since you have not provided the stopping index, which is supposed to be placed in between these two colons, so since you have not provided that number, it will go till the end of the array. So from starting till the end, essentially selecting the entire array. But now you want to pick up the numbers with a step size of two. So every second element will be picked up. Okay.

Now, let's see how does indexing and slicing work with multi-dimensional arrays and matrices. So let me first create a 2D matrix. So here I have got the numbers 1, 2, 9 arranged in a 3x3 format. So three rows and three columns. These are the rows: 1, 2, 3, 4, 5, 6, 7, 8, 9. These are the three rows, and these are the columns: 1, 4, 7; 2, 5, 8; and 3, 6, 9. Now, here I want to extract the elements with the index (1, 2). So this is my row index and this is the column index. Now, what is the row index? So this is the first row with an index of zero. The second row will have an index of one, and the third row will have an index of two. Okay. So I am asking it to fetch me the first indexed row, that means the second row. So this row has got the index of one. So second row and column index is two. The first column will have an index of zero. Second column index has one. And the third column will have an index of two. So essentially, you are saying, give me the element present in the second row and the third column. So here you can see it intersects at six. Perfect.

You can also provide a range of row indices and column indices. Okay. So 0:2 means it will take rows 0 and 1, 2 will be excluded. So rows index 0, 1, and column index 1 and 2, because again, 3 will be excluded. So row index 0, 1 means the first two rows, and column index 1, 2 will be the last two columns. So first two rows and last two columns, essentially, will give me 2, 3, 5, 6. Let's see. So this is exactly what we call as two-dimensional slicing. Okay. You can also provide handpicked row column indices. So in this case, let's say I want to select the elements from the first row and the last row. Last row will have index two in this case. So that is the indices you are providing, and colon means take all the columns. So here you are providing the row indices, which is 0, 2, and colon means here in the column side, this colon means take all the columns. So give me the indices 0, 2 for the rows and all the columns. So let's see what I get. Since I asked for all the columns, so you can see that I'm able to get all the three columns and the first two rows. Okay. So what happens if I make this 2, 0? So the good thing is I'll get the last row and then the first row. So just watch, the first row actually becomes the last row now. Okay. So you can see that these two rows actually flipped because the indices got flipped. So you can actually retrieve the rows and columns based on handpicked indices that you want. And that's the fancy indexing I'm talking about. Okay.

Let's see how does the boolean indexing work. Here you'll provide a condition. So here, what am I saying is, give me all the elements of this two-dimensional array which are greater than four. So that will create a kind of a boolean mask. Boolean mask is nothing but an array of the same shape as this, which has only True and False. So True at those positions where this condition is met, and False at all those positions where the elements are less than four. Okay, where this condition is not met, essentially. So let me first print the boolean mask. You can see at these places I have a number which is greater than four. That's the exact meaning of the mask. And when you pass that mask, that mask becomes a boolean index. Essentially, it's a boolean index. When you are passing that boolean index to that two-dimensional array, you will be able to fetch the elements which, or against which, you've got True here. Okay. All the elements which gave you True by using this boolean mask, those elements will be fetched out. Okay. By providing that boolean mask, you can provide multiple conditions combined together. So in this case, I'm asking to give me all the numbers which are greater than two, and the same numbers must also be less than eight. So two conditions I'm applying. This is an AND condition. Okay. So again, the same boolean mask created, and I'm able to retrieve the numbers greater than two and less than eight. Makes sense?

Now, let's see how can we manipulate the shapes of the NumPy arrays. Okay. So let me create a NumPy array here. I'm using the arange function. So it will give me a one-dimensional array with the numbers 1 to 12. Okay. And that one-dimensional array, I'm going to reshape it into a two-dimensional array by taking three rows and four columns. Okay. So I'll reshape. So I'll reshape array of one dimension into a two dimension. Let's see. Yeah. So this is what I get. So if you want to print this matrix here. Yeah. So this is 1D, and that got reshaped into 2D. One thing you have to note here is, you cannot provide any random number here. Okay. So the product of these two numbers must be equal to the total number of elements present in the array that you want to reshape. So here, the array that I'm taking is a matrix. Whether it is an array or a matrix doesn't matter, but the product of these two numbers must be equal to the total number of elements present in this matrix. So this matrix already has 12 elements, and 3 * 4 is already 12. So that works. If I try this by making it six, I'll get an error because again, the same error message: 3 into 6 is 18, and that cannot be fit into 12 positions. Super. Yeah. So you can also reshape into 2x6. So two rows, six columns.

What does this do? This will take a two-dimensional matrix and flatten it. So flattening is exactly the reverse operation that we did here. So flattening means take all the elements of the 2D array and put it in one dimension. So you can see that this matrix was reshaped to 2x6, that is two rows and six columns, and now it has been flattened back to 1D. Okay. So this is the original matrix, and this is the raveled matrix. Okay, fine. So I'm just printing the original matrix so that I can do the transpose operation. So what does the transpose do? So it will exchange the rows with columns. So essentially, the first row becomes the first column. The second row becomes the second column. Third row becomes the third column. So we had three rows. In the transposed matrix, we'll have three columns. Okay. Exactly. So you can see the first column was the first row. Second column is the second row, and the third column was the third row originally. Okay. So rows and columns are just flipped.

Let's see the data types and how we work with data types in NumPy. So again, I'm creating a NumPy array with the numbers 1.2, 3.14, and 5.6. You can see that these are floating-point numbers. Okay. And let me just print the data type. So it says float64. Essentially, I'm fetching the data type of the elements present in my array. This astype method will change the data type. So my original elements were in float64 data type, and now they are being type-casted to an integer data type. So you can change the data type. So you are saying that take this array and change the data type to int64 and save it back. Okay. So you can see that my numbers are 1, 3, 5. So the floating-point numbers 1.2, 3.14, and 5.6 have been converted into the nearest integers. Okay.

Now, there are some universal functions for arithmetic operations. Let's have a quick look. So you have the functions for addition, subtraction, multiplication, and division as well. I have my arrays X and Y. You can use these functions to carry out these operations. And you can actually also carry out these operations directly. But you have to make sure that these two arrays which you want to multiply, divide, subtract, and add, they have exactly the same number of elements. Okay, these are element-wise operations. So X + Y works, X - Y also works and gives you the same result. If you can see, X multiplied by Y is also an array, is also an element-wise multiplication. You can see this result matches with this result. And similarly, X divided by Y is also an element-wise operation. You can see each element of the array 1, 2, 3, 4 was divided by the elements 10, 20, 30, 40 respectively. So each of them become 0.1. Perfect.

You can take the log by default. So I've created some array again. Here I have got the np.e, which is a constant e in mathematics, the exponent e. So 2.718 value, and one value is one, and the other value is 100. So three values I'll be using for log calculations. So by default, this will give you the natural logarithm with the base e. What is that value of e? 2.718. So the base for this logarithm is this e, and that is why the value of log for that e is one. Okay. Log one is always zero. So that's what I got here. And this is the log of 100 with the base e. Now, you can change the base to two. In that case, the two base log values are these. So log of e to the base 2 is 1.44, and log of 100 to the base 2 is 6.64, and so on. Now, here you are changing the base to log to 10. So again, the log of 1 is 0. This is the log of e to the base 10, and this is the log of 100 to the base of 10, which is two in this case. So np.e is same as np.exp with the power of one. You can use this function to raise some powers to this e. Okay. So the values are the same 2.718.

So the exponential functions. So I have the same array 2, 3, 4. I am going to use this array as exponents. Okay. So I'll be using the function called as power. So each element of this array will be raised to the power of 2 in this case. Okay. So I'll get 2 squared, which is 4, 3 squared, which is 9, and 4 squared, which is 16. So let me print this: 4, 9, and 16. Perfect. And what are these three numbers? These are basically the exponent e raised to the power of these numbers. Okay. So this is essentially e to the power of 2, e to the power of 3, e cubed, and this is the e to the power of 4. You are using that array as the exponent, okay, to be raised.

Some basic trigonometric functions: sin, cos, tan. So I'm getting the sine. By default, these are going to compute the numbers or values taking that these angles are in radians, remember that. Okay. So sine of 0 radian is 0. Sine of pi/2 radians, which is 90 degrees, sin 90 is 1. And sine of pi radians, essentially is 180 degrees, that is again zero. Okay. So sine values would be 0, 1, and 0. Cos values would be 1, 0, 1. Okay. And then we have the tan values as well. You can see, however, some of the numbers are not zeros. That is interesting, and that is because of the loss of precision during these floating-point calculations. Okay. So this is practically zero. This is also zero. Okay. Okay, any number which is close to 10 to the power of -6 or lower is supposed to be interpreted as zero. Make sure that, okay, so these, this is again infinity, and this is again zero. Okay. And similarly, we have the hyperbolic functions.

Let's see how does the rounding functions work. I'll demonstrate the absolute function, the sign function, and the rounding functions like round, floor, ceil, and truncate. Okay. So let me create an array again, and I want to get the absolute values of these numbers. So this is negative, so in the absolute, it will become positive. This is again negative, absolute will become positive. Yeah. So the absolute values are all positive. You can see, and those numbers which had negative sign here, they are represented by -1. All the positive numbers are represented by +1. Now I want to do the rounding. So rounding function works by rounding these numbers to their nearest decimal place, nearest integer, let's say. So the nearest integer to this would be -2. The nearest integer to this would be 0. The nearest integer to this would be 1. Nearest integer to this should be 2. And the nearest integer to this should be 3. Let's see. Yeah, correct. So floor will round off the values to the lower integer. Okay. So, and ceil will round to the higher integer. Okay. Truncate will simply exclude the decimal points, the values after the decimal point. So simply 75 will be ignored. 22 would be ignored. 58 would be ignored, and so on. Okay. So that's how you get these numbers. The signs will be retained. However, now let's see some of the comparison and logical universal functions.

Again, I've got my arrays here: 1, 2, 3, 4 and 22, 222. So, I can actually compare whether each element of A is less than each element of B. This again works element-wise. Here you are checking for each element of A. Is it greater than each element of B? Again, you are checking for the equality of the corresponding elements in A and B, and not equality. Okay, let me see. Okay. Now, these can be made simpler by also saying A < B. That also works. You can see that the result is very similar. Okay. To this. For equality, you can go with two equality signs. So you are checking for the equality. This result should be same as this. Okay. And for non-equality, this should be written as this exclamation equal to. So this is not equal to B. You can see the result. And greater than or equal to can be written as greater than or equal to B. So this result would be again same as this. The same thing in the logical format. Okay. Okay.

Now, statistical and aggregation functions. I'm creating a simple array 1 to 9 numbers in a two-dimensional format. So I want to calculate the sum of all the elements. So np.sum function, all the elements, sum from 1 to 9. Okay. Here the mean of all the elements, the standard deviation of all the elements, and the variance of all the elements, okay, that entire array. So sum is 45, mean is 5, standard deviation 2.58, and the variance is 6.6767. Similarly, I want to get the minimum value, the maximum value, the index of the minimum value, and then the index of the maximum value. That is the meaning of arg in that is the argument. Short form of that argument is argmin and argmax. So that will give you the index of the minimum and the maximum values respectively. So zero, that is the very first element here, and eight is the last element. Anyway, again, we can get the median. So median of all the nine numbers is five, and the first quartile is at three. Okay, you can change the quartiles here. 0.25, 0.5 will give you the median, 0.75 will give me the third quartile, and 1 will give me the maximum value. Zero will give me the minimum value. So these are the five-point summary of any, you know, data. You can also do the summation along a particular axis. Okay. So what does this mean? If you want to do the summation along the rows, you can specify the axis as one. And if you want to do the summation along the columns, then you can specify the axis as zero. So how do you want to do the sum? If you want to do the sum all together of all the numbers, then you do not have to specify the axis parameter. It will take the sum of all the elements anyway by default. But if you want to do the summing along the rows or the columns, then you have to provide the axis parameter.

Now, let's quickly have a look into the broadcasting concept that enables to take or to conduct mathematical operations between arrays of different shapes following certain rules. Let's see. So I've got a simple NumPy array as 1, 2, 3, and I'm going to add a scalar to that NumPy array. Okay. So what happens is this scalar is indirectly type-casted. It is converted into another array in the memory behind the scenes, and that gets replicated three times. So you get an array like 10, 10, 10, and that array of 10, 10, 10 gets added to each element here. So the addition operation is still occurring element-wise. So that creation of this additional array behind the scenes of 10, 10, 10, that is exactly what the broadcasting actually does. If you see, this will create an array of three rows and one column. So it's essentially a column vector. All the numbers in one dimension. Okay. And this is a one-dimensional array. So this is a 2D array, and this is a 1D array. Okay. And you want to add a 1D array with a 2D array. And so again, you'll have to use the concept of broadcasting. So this is how it works.

Let's take a couple of linear algebra functions which are very important ones. I create two two-dimensional matrices: 1, 2, 3, 4 and 2, 0, 1, 2. A and B. I'm going to multiply A and B. So note that A multiplied by B is not the same as B multiplied by A because matrix multiplication, which is done by using the at operator, that is not an element-wise operation. Okay, let me show this. So these are the values of A and B. And the matrix-wise multiplication between A and B. So this is A multiplied by B. This is the matrix multiplication. Let's see B multiplied by A. What does it give? Okay. So B * A is this. A multiplied by B is this. So that's what exactly I said. A into B is not same as B into A because these are not element-wise multiplication. Okay. You can also do that same operation by using this function A.dot(B). So the result will be same as what we got before. Okay. So you can use the dot function. You can also use the matmul function. So, np.matmul(A, B). Again, make sure the order in which you are passing the arguments. So this is again A into B. If you say B, A, then this will be B multiplied by A. You can use the linear algebra module to get the inverse and the determinants. Okay. So this is the inverse of A and the determinant of A.

I'll use a random module to generate some numbers and sequences here. So I want to generate the random numbers, integers in this case. The lowest value being zero, highest value being five, and I want to pick five numbers. Okay, so these are the numbers that I get, and these numbers can also be repetitive. Okay, so randomly I'm picking five numbers by replacement. Okay, so I get this array. Okay, here I'm creating random numbers which are in the range of 0 to 1. Zero is inclusive, one is exclusive. Okay, so it's an open interval on the other side. I'm going to create five random numbers in that range. So these are the five random numbers that you have got. If you want, you can also round off these random numbers. So let me round it to two decimal points. So you can see the random numbers were generated and also rounded off to two decimal points. I can also create normally distributed random numbers. Location is the mean of the random numbers. Scale is the standard deviation, and size means how many random numbers do you want to create. Okay. So size determines how many numbers you want. And the mean of these five numbers would be close to zero. It cannot guarantee that it will be exactly zero. And the standard deviation of these five numbers will be close to one. Here you have your numbers. Okay. You can also do the shuffling. So let's

Say I've got an array of these five numbers and I want to shuffle them. So this is the shuffled array. Here you are not generating anything, but you are generating another array with the numbers which are shuffled from the previous array. You can also create random permutation of these numbers. Okay.

Okay. So in conclusion, in this Jupyter notebook, I have demonstrated how you create different arrays by using the built-in NumPy functions. We have done enough examples on indexing, slicing, array manipulations in terms of shape, reshape function. We have used also flattening function like rival arithmetic operations. We have done logarithmic trigonometry functions, along with that, rounding functions, comparison and logical operations. I've also demonstrated broadcasting and linear algebra functions. Finally, we also saw some of the random generator functions. Thank you.

Hello everyone and welcome to this session on the Matplotlib basics hands-on. So in this notebook, I will demonstrate the most commonly used plotting functions in Matplotlib and how to customize these plots. So let's get started.

So first, we need to install Matplotlib if it is not there by using pip install Matplotlib. Most of the Anaconda installations come with Matplotlib pre-installed, and also your Colab comes with Matplotlib pre-installed. So let me see what is the version of Matplotlib in my machine. That comes to 3.1. Okay.

So now let's import the pyplot module from Matplotlib and assign it as an alias of plt. So pyplot is basically used for one-dimensional plotting. If you want to go for two-dimensional or 3D plots, you have different modules for that. I'm also importing NumPy as np to create some arrays which will be actually plotting in Matplotlib.

Now, in the earlier versions of Jupyter notebook, you will have to uncomment this line and run this Matplotlib inline so that the plots appear within the Jupyter notebooks. But it is actually no longer needed if you are running the most recent versions of the Jupyter notebooks or even using Google's Colab. Okay. So I'll just keep it for now for backwards compatibility, let's say.

So let me create a sample data here by using the NumPy function linspace that is linearly spaced numbers starting from zero ending at 2 pi and both inclusive. I'm going to create 98 numbers in between. So the starting and the ending numbers are anyway included. So total number of numbers that I'm going to create in this array is 100, and I'm computing the sign of that x values and saving it as y. So I have my x and y. You can see the y values. Let me print the y values here. So sin 0 is 0, and sine of 2 pi, 2 pi is 360 degrees, that is also close to zero. Right? So this is supposed to be interpreted as zero. That's not a number which is a valid number. I would say sin 2 pi, which is 180 degrees mathematically, analytically, any high school book will give you zero. So this is actually zero. Okay.

So first, we will see some of the basic plot types here. Uh, we'll first start with the line plots, then the scatter plots, bar plots, histogram, box plots, and pie charts. So let's first start with the line plot. Okay.

So first, I'm creating the figure canvas. You can see on which this figure will be plotted. I'm setting the aspect ratio. So kind of 6 in x 4 in kind of I'm creating a canvas here. So the aspect ratio of the figure can be set here. 4x4, 6x6. These are squarish kind of shapes, and this will be more like a rectangular shape in which the plot would be visible.

So to create the line plot, I have the plot function. You'll have to provide the values in the x-axis, then the values in the y-axis. These are two positional arguments, and then you have all series of keyword arguments. So what kind of label you want to give to that plot in the legend? So this label will come in the legend. What should be the color of the line? Okay, by default, it will have some sort of a blue color. But if you want to explicitly mention the color, we'll see how we can change the colors. And then the line style. So line style is a dash here. Dash means you'll get a solid line. And the width of the line is given as two pixels here. Okay. So you can change the line width. Okay. So default is one. Here, the line style is dash by default, anyway. Color is some sort of a blue, not the exact blue that you'll get to see here. And by default, there are no labels. Okay. So label is none. So I have specified a label and all the custom values.

So this will be the title of the plot. The title which comes above the plot canvas. So let me execute this. You can see this is the title, "Line plot of sine waves". This is called as the y label. So what do you want to write in the Y labels? Okay, these are the Y-axis values. Similarly, you see this as the X label. Okay, so I'm providing the X labels and the Y labels here. If you want to increase the font size, you can do that by using the font size argument. So let me have a font size of maybe 12. And I'll increase the font size for the title. Let's say I want to go for 15. So you can now see these are slightly bigger in size compared to the the tick labels. These are my tick labels, and you can see these are the ticks. So 0, 1, 2, 3, 4, 5, 6. These are the tick labels. Similarly, these numbers that you see, these are the Y tick labels. Okay. So by default, they are in the font size of 10. So I made the label as 12. This label is 12, and you can see this line width is two, and this font size is 15. Okay.

You can also customize. So let's say I want to change this color to dodger blue. D O D G E R. Dodger blue. One of my favorite colors. And you can see a very nice light bluish color is there. So you can, you know, create all sort of exotic colors. You want to go with lime green. Okay. So green is one color. Lime green is another color. Okay. So let me just delete. So this is the normal green color. Uh, this was the lime green color. Okay. So where do you find these names? Okay. So let's go to Google and search for Matplotlib named colors. Okay. So click on the first link, and here you go. These are the names of the colors that you can supply. Okay. So I was using these colors, this dodger blue. Can you see? And one of the green, lime green is present somewhere. Yeah. So this is lime color, and lime green should also be there somewhere. Okay. And then you have multiple shades of pink and red. Okay. Crimson, deep pink, hot pink, magenta, and so on. Purple, dark magenta. You can provide any of these names. So coming back. So this is how you can change the name of the color.

Now, line styles. Now, instead of dash, let me give double dash, and that will give me a dotted line. Okay, you can see, sorry, a dashed line. This is a dashed line. Okay, so to get the dotted line, you can go for colon. So this gives me dotted line. You can see very well, this is a dotted line, and you can go for a dot dash as well. So dot dash should also be there. Okay. So that is dash dot. Okay. So the dash and dot. So where do we search for these line styles? So again, go to Google and search for Matplotlib line styles. Click on the first link, and you can see all the line styles available in Matplotlib. So these are the most common ones. Again, if you want to have more exotic line styles, you can experiment with these advanced options, but these are the most common ones. Dash is a solid line. You can get a dotted line, dash-dash line, and a dash-dot line. Okay. So you can see how these lines look like. Perfect.

So coming back again. So I have taught you how you can customize the font sizes, the colors, the line styles, and even you can change the line width. So if you want, you can make it three, that will become a thicker line. You can see the line thickness has increased. Okay. So you can customize this as you want. Fine.

And then this legend function will actually place this legend in the plot. So if you forget to provide the legend here, you can see what this plot represents, that is what is given by the legend. Okay. So that legend is gone because I commented that, and you can also provide the location where you want to place the legend. So the value in the legend is populated automatically by the label. So if you have multiple plots, make sure that each of those plots have a label because you are not explicitly providing any text within the legend function here. Okay. So it's advised that each of the plot must have a corresponding label so that Matplotlib is able to recognize what should be the labels for each of these plots if you have multiple plots in the same canvas. Okay.

So let's say I want to shift the lo uh location from the first quadrant, this is my first quadrant, to let's say the third quadrant. This is the second quadrant. This is the third quadrant. So what I'll do is loc = 3. So that will change the location to the third quadrant. Now you can see the location shifted from the default of first to third. So Matplotlib actually selects the best location so that it does not overlap with the plot. So that intelligence is there with Matplotlib. Okay. So it already takes care of that. But still, in some cases, if you want to specify what should be the location, you can go ahead and do that. Okay. You can provide numbers or you can provide text as well. Okay. So the location can be provided as a text. You can see it supports these strings: upper left, upper right, lower left, and lower right. And it is also supporting the center. So by default, it will take the best place. Okay. So that it does not overlap with your actual plots. Fine.

So let's go ahead. Now this time again, I'm having a bivariate plot. Let me create two sets of random numbers here, X and Y, and again creating the figure canvas, providing the X values and the Y values to the scatter function. And then what should be the color? C = means the color. So short form of color is C. So what should be the color of the marker? What should be the marker style? Okay. And what should be the opacity level of these markers? So let's see the plot first. So you can see you have the red color markers here. If I set alpha as one, I'll get dark red. And if I decrease a value to let's say 0.2, you can see these are light red. So it becomes more transparent. So lower values will lead to more transparent markers. So that if there are markers which are overlapping, you can still see them. Okay. So you can customize the opacity, title, label, and those things are same. The new thing that here is grid. Okay. So grid is your these vertical and horizontal lines. These are called as grid. Okay. In the previous plot also, we had the grid. Okay. If you don't want to display that grid, you can set it to false, and grid will not be displayed. You can see those vertical lines and the horizontal lines which help us to track the values. They are gone. So you can customize this as per your choice and use case. So plt.show() will finally display that plot. Okay. So that should be the last Matplotlib function in any block of code which contains the plots. Okay. So here also, the last function from Matplotlib is to display the plot. Okay. So first, it creates the plot internally, does all sort of customization, and finally displays it. Okay. So that is the philosophy that we are going to follow for all the other plots as well. Okay.

And here in the line plot, let's say you want to also provide markers, that is possible. Okay. So I'm going back to a solid line. And now let me provide some markers here. Okay. So here also, if you want, you can provide marker. marker = let's say I want to go with square markers. Okay. This time I'll write 's' here. S for square. Okay. Okay, let's see what I get. You can see the square markers at all those X values. Now I had 100 X values, that's why I get so many squares here. Okay, these are so many squares. So 's' is for square. Here 'o' is for circles. Okay. So there are other marker styles. So let's go to Google once again and search for Matplotlib markers. Okay. So you'll get the marker reference here in the documentation. So you can take small circles or points, you can see. So these are the actual styles that you will provide in the code, and this is how it is actually displayed in the plots. Okay. So you can have some numbers as well. Octagon, pentagon, square. So I used this square just now. You can also go for pentagon, hexagon, asterics, and so on. Okay. So these are all, you know, markers available to us. Okay. Okay.

So let's move on to the next plot. Okay, so let's see how we can create bar plots. Bar plots is for categorical data. Okay, so for numerical data, you can go with the line charts or even the scatter plots. But if you have categorical data in your x-axis, okay, so A, B, C, D will be there in your x-axis, and their count or frequency is represented by values here. Okay. So now this A, B, C, D could represent maybe the occupation of the customers in your data set. Okay. So this A can be student, B can be salaried, C can be retired, and D can be non-employed. And then this is the actual count of those customers with those respective occupations. It could be as simple as gender also. It could so A, B could be just male, female, and then the count of the male, females. Okay. So you can have any number of categories and their corresponding frequencies or counts. So that is what the category and value means here.

So again, I'm creating my figure canvas. So this time I'm going to use the bar function. Categories, that is basically the x-axis values, will be populated by categories. The y values are going to come as counts. The values, the height of the bar will be controlled by the values. Remember that. Okay. And then what should be the color for these bars? Let's have a look. So the color for these bars is definitely green because that is the color I have provided here. And then the title, label. These are the same functions which we have seen. Okay. So these are common functions for any Matplotlib plot. So what should be written in the x-axis label, y-axis label, and the title. Okay.

You can also create stacked bar charts as well. Okay. So if you have multiple categories, you want to create a, you know, bivariate, two categorical data simultaneously, you can go for stacked bar charts as well. Okay.

Again, histogram is for univariate data. Okay. Especially if the data is numerical. Okay, you want to provide or you want to visualize the distribution of the data across multiple bins of the values. Okay, so you can provide how many bins you want in the x-axis. The default number of bins would be 10, I think, and then what should be the data used for plotting the histogram, and then what should be the color of the bars in the histogram, and then the transparency level, and what should be the edge color. So even in the previous plots, you have an option to provide the edge color and so on. Okay. So I'm again creating normal distribution of random 1,000 numbers with normal distribution. Randn. This is a randn function. N represents a normal distribution. Okay. So randomly distributed numbers coming from normal distribution. Okay. So you can see the purpose. The main difference between the bar chart and the histogram is the bar charts are discrete. There will always be some spacing because these are discrete values. Okay. A, B, C, D. Okay. This is categorical data. That's why you will always have space in a bar chart. This is for a continuous data. So you will never have a space. Okay. And this is for a continuous data. You cannot mix and match. Okay? So you cannot use a bar chart for you cannot use a bar chart for continuous data, or you cannot use the histogram for categorical data. That will not make sense.

Now let's have a look at the box plots. We want to first create three NumPy arrays. Here I'm going to use the normal function to create these NumPy arrays, normally distributed values. So zero is the location, which is the location of the mean. So the data one will have a mean at zero, standard deviation at one. I'll have 100 data points. The second data array has got mean at five and the standard deviation as two. And the third one will have a mean at minus three and a standard deviation of 1.5. Okay, all three will have 100 values. Again, a figure created and a box plot. So I'm just clubbing all these three and providing some tick labels here. Let's see how the box plot comes up. Okay, so you can see these lines represent the median value. Okay, these orange lines, they represent the median value. You have the lower whisker and the upper whisker. This is the edge of the box. Lower edge, which represents the first quartile. This is the second quartile. The median value. This is the third quartile. Okay, for any box plot. So the total height of the box will be given by the interquartile range, Q3 minus Q1. Okay. Same goes for all the other two plots. Okay. So once again, this is my Q1 value, Q2 value, and Q3 value. And you have the total length of this whisker as one and a half times of the interquartile range. That is the length of the box times 1.5 is the total height here. And some of the outliers, you can see which is going beyond the upper limit. Okay. So otherwise, if there are no circles above and below, then these horizontal lines would represent the maximum value and the minimum value. So here it represents the minimum value. Again, this is the minimum value, and this is again the maximum value. So you get all the information about the distribution in terms of the box plot. Okay.

We can also go with pie charts. So I have created some slices here. At. So let's see the slice labels A, B, C, and D. And then this is basically the fraction of your circle that will be occupied by these labels. Okay. So 40% of the circle will be occupied by A. Another 25% by B. C will occupy 20, 20%. And the last 15% will be occupied by D. So make sure that the sum of this will be equal to 100. Okay. And there is something interesting here. I have set an offset for the second bit, and that's why the second piece appears to be slightly off. Okay. So that is explode. In the exploded view, that is highlighted. Basically, if you want to highlight a particular section, then you can do this explode option as well. So the main function being used here is pi. You can provide the slices, basically. Okay. And then the slice labels, explode options. If all, by default, all are zero, means you'll get a flat pie chart. So this explode option gives you a 3D effect, essentially. And then the label formatting, this is done by this. Okay. And then whether you want the shadow. So you can see the shadow here. The shadow is given. So whether you want to have the shadow effect. So that gives me a 3D effect kind of thing. And then starting angle. Okay. 140 or you can provide 0, 90, whatever angle. So from where your slices actually start. Okay. So see this angle here, from here to here, that's 140 degrees. Okay. This angle is 140 degrees. So that's where the first slice starts, and then the second, then the third, and the last one.

Now we have seen how to customize some of these things. Let's deep dive into customizing axis limits, text, grid lines, and figure size, and so on. Okay. So here I've again created a figure. I'm going to create two plots here. Plot one and plot two. You see that I have provided two different labels, sin(x) and cos(x). So make sure if you have multiple plots within the same canvas, provide different labels, otherwise Matplotlib will not know, and you will even not be able to figure out which plot corresponds to what function. Okay. So I'm using two functions. Okay, my X values are created already above, and then you can specify which colors you want and line styles you want. Okay, fine. Titles, X label, Y label, legend, everything I have already explained. Grid and show. Let's have a look. Yeah, so my sine is in solid line, and cosine is in the dash line.

Again, if you want, you can set the limits for the x-axis and the y-axis. So here I have got the same plot, XY, and the color being red, and I'm setting the limits. So 0 to 2 pi. So 2 pi would be, so pi is 3.14. So 2 pi would be 6.28. Okay. So that's the limit that I'm setting. And for the y limits, you can see these are sine values and cosine values. So by default, everyone knows that sine and cosine will have a limit from minus 1 to plus 1. That's why the minimum value is minus 1, and the maximum value is plus 1. You can change these values. So let's say I want to go for from minus 2 to. So let's say I want to go from minus 2 to plus 2. Okay. So I'm setting my limits for the y-axis. So now you can see that this is minus 2, and this is plus 2. Okay. And if you want to see the labels, minus 2 and plus 2, slightly increase it. Yeah. So now you can see the labels as well. Okay. So you can provide what should be the y limits. Again, if you want to go for a narrower x limit, that also you can do. Okay. So here I am going from 0 to 2 pi. Let's say you want to only plot from 0 to pi. That can also be done. Okay. So you'll just change this number over here. Okay.

So at what points you want to have the tick. So by default, Matplotlib decides that I want to put the ticks at these many places. Okay. And then the labels are also automatically populated. But if you want, you can provide the actual points in the x-axis where the ticks will be placed and the corresponding labels. Okay. So you see now you have ticks only at 0, pi, and 2 pi, and the labels are also in the mathematical notations that you have given here. Fine.

So you can also provide labels in this format. Let me just copy this. Okay. If you want your labels to have a scientific notation, you can also do this. So you can use a dollar sign $2 and then backslash write pi, and this would be same dollar sign backslash 2 pi. Okay. Oh, sorry, I missed the dollar sign closure. Okay, that was the error. Okay, you can see don't forget to close this dollar sign. So dollar sign start and close, again dollar sign start and close, and within that, you can provide the mathematical symbols. Okay. So here I have provided the mathematical symbol pi. So that is another way you can actually do this. So you can see pi written in mathematical notation. Okay. You can provide any other mathematical notation there. So if you want to go with sigma, let's see how that sigma appears and so on. You can go with alpha, beta, gamma, delta. All sort of mathematical symbols you can have in the labels. That's exactly how. Okay.

Now let's see how we can work with multiple plots, that is by creating multiple plots, multiple axes, and even subplots. Okay. So here I'm using the subplots function from the plt module. How many rows? I have one row and two columns. So it will come as side by side, two plots. The total size of the figure canvas is 10x4. Okay. And in the first subplot, so these are the index for the subplots. So zero means the index for the first subplot. And then this is the usual plot. And then the title for that particular subplot. Okay. Now this time, the function changes from plt.title to ax[0].set_title. That is the object-oriented programming concept that we are applying here. Same goes for the second subplot. So first, you reference to the second subplot axis, and then the usual plotting functions, and then same goes here, reference to the second subplot, and then that's the title you are providing. Okay. Suptitle is the super title. Basically, that will be the title provided at the top of both the subplots. Okay. And tight_layout will trim any extra spaces that is on each of the edges of the plots. If there are extra spaces, that will be trimmed. That's the tight_layout. Let's see. So this is how I get.

If you want it to be one below the other, then you have to go with the other notation that is, let's say I want to have two rows, I will change this to two, and I want to have one column, so two rows and one column will give you one below the other. Okay, so in this case, I might want to go with 6x14. Let's see. That's too big. So 6x10 that should be fine. Yeah. So you can see. So height is 14 and the width is six this time. Okay. So you can customize this. And then, uh, you can also create multiple figures. Okay. So this is one figure, and then another figure, or if you want to place both the codes in one box, you'll get two figures back to back. Okay, you'll get two figures back to back because we have plt.show() function two times. So both the figures are created back to back in the same canvas. So you have created the canvas for each of the figures. Okay. So this way also you can customize. You want to have it side by side, then it's better to go with a subplot, or if you want to have one above one below the other, then you can go with this option as well. Okay.

And then we have the styling. How you want to show the background of the subplots, that's the style. So by default, our language uses the ggplot option. So you do have other stylesheets like seaborn, bmh, and so on. So let's use the default ggplot which is available in our language. So ggplot stylesheet. So again, I'm creating my figure, the plots, the x-axis values, y-axis values. I'm creating a scatter plot, and this is the new thing that we have here. I'm creating an annotation. Annotation will help to pinpoint a particular point in the plot and provide a label to it. Okay, that's the annotate function. So this is the value that will come up in the label. So this is the X and Y coordinate of the arrow. You are creating an arrow essentially, and this is the label at the end of the arrow, you can say. So and this is the text where the text starts, and then the arrow properties, what color you want, and you want to shrink or expand, what length of that arrow. Some of these customizations you can look at in the documentation. Now, these are the other regular things. So you can see that I'm able to highlight a particular point in the plot by using this arrow here. So this is the label which I have provided in the annotation. Okay.

You can also have a regular text box placed anywhere in the plot. Okay. So that function is different. So that is plt.text() function. Okay. So you can see the background colors and those things are different. If you want, you can change the ggplot to let's say seaborn or bmh. So let's go ahead and look about the line stylesheets. So Matplotlib stylesheets. Okay. So we have the stylesheets reference in Matplotlib here. So this is how we use the ggplot option, and these are the other. So this is the default. This is what we have been using until now, and then we changed it to ggplot. So once you have changed the stylesheet, that stylesheet remains active for the remaining notebook. Okay. Unless you set it back to the default. So this is a classic one, solarized light, bmh. This is how the plots will look like in the bmh. You can have dark background, light background, 538, ggplot, grayscale, and then you have the seaborn v0.8 version, bright colorlind, dark pallet, everything. Okay. So yeah, and so on. So let's come back. Let me change it to bmh and see how does the plot change. Yeah. So this has the background has definitely changed. Okay. So back to ggplot. Okay. And you can revert to the default this way.

What if you want to save your plot? Then you can use the function called as savefig. Okay. And provide the figure name. You can provide the DPI. DPI is your dots per square inch. So 300 dots per square inch. By default, it is 72. So if you increase this value, you'll get a much higher resolution image. Of course, that will also take much more space. Okay. The figure size that gets saved will also be much larger in bytes, kilobytes, and again bbox_inches, that is the tight layout that we are going to create. Okay. So that ensures minimal white space around the figure. Fine. So let's create this plot here. I've created this plot, and this is not yet saved. You can see this is not yet saved. But if I want to save this, I'll uncomment this and execute this once again. Okay. And we can verify that this has got saved. So let me just go back, and you can see that this was created 11 seconds ago. Okay. So I'm running this on some cloud platform. You can download this. Otherwise, this will be saved in your local drive. If you are running this in Anaconda, this will be saved in your local drive. Okay. So this is the plot which got saved. Perfect. So pretty high resolution. You can see it's not blurring. You can see even if I'm zooming quite a lot, it's not blurring because it's a very high resolution. 300 DPI is actually a lot. Okay. So the default is, I think, 72. Okay.

So coming back, finally, some quick tips. So you can use the subplots for multiple visualizations rather than stacking multiple plots. Okay. One below the other. So you can organize your plots in a grid, 2x2 grid, 3x2, 2x3 kind of grids. Okay. Please label your axis properly and put proper labels in the legend for clarity. If you are having multiple plots within the same canvas, use styles and colors to create visually appealing plots. Okay? And make sure that your markers are clearly visible. If you have the markers along with the lines, you must have some sort of a contrast between multiple plots and multiple markers present within the same canvas. Okay. So, keep a track of the figures and axes using the interface. That is basically the name numbering of the subplots. Okay. So, we created the subplots here. So the numbering of the subplots will be decided by this function. Okay. And of course, keep checking for the advanced features in the documentation, as we have referred for the list of named colors, Matplotlib line styles, markers, and even stylesheets. Okay.

So here I have combined few plot types. So here I have got the scatter plot, you can see, and a line plot. So I have just created some synthetic data. So in one plot, I have the scatter values as well as the line chart. And clearly, by using the legend in a proper way, I am able to demonstrate that this is a line. What does that line represent? It represents a trend, you can say. And these are my observed data points. So in your real-world scenario, these would be your observations. And this line will not be a synthetic line. That would be a line that you get from a regression model. Maybe a linear regression or some some variant of that. And then you have your usual title, X label, Y label values. Okay.

So just a quick conclusion. So we have seen in this notebook how basic plotting and Matplotlib works. Different types of plot types we have covered as line chart, scatter plots, bar plots, histogram, box plots, and even pie charts. Different customizations we did with the labels, legends, styles, and markers, colors, marker styles, and everything. Multiple plots we have got. How do you save your plots to the hard disk, and we have even seen combining multiple plot types in the last example. Okay. So that's it in this code demonstration on Matplotlib.

So what is Pandas? You might be wondering. Pandas is a popular open-source library in Python used for data manipulation and analysis. Pandas provides powerful data structures, which we'll see as DataFrames and Series, and these are used to store and manipulate large datasets with ease. Using Pandas, you can load data from various sources like CSV, Excel, or even SQL databases. You can also clean your datasets. You can reshape them. You can also analyze this data very quickly using a wide range of functions which are built-in available in Pandas. So let's get started.

In case Pandas is not installed, you can just uncomment this and install Pandas. Most likely, if you are using Google Colab or an Anaconda environment, Pandas would come pre-installed. So I have one code here which instructs Pandas to ignore any warnings. Warnings are different from errors. Okay.

So first, we'll see how to import a data file which is present as a CSV. CSV is comma-separated value file. So I have got a churn prediction.csv file. I'm going to import this in Pandas. So we'll be using the read_csv function to do this. Okay. And once we have read that CSV file, I'm storing it in the form of a DataFrame and I'm displaying the first five rows of that DataFrame. So this is how it appears. So DataFrame. So DataFrame objects are basically two-dimensional, like you have few rows and you have few columns. Okay. So that is two-dimensional. Whereas each column of the DataFrame is essentially a Series in Python. Okay. So this is one Series. This is another Series. This is another Series. In fact, every row of this DataFrame. So you can take out any of the rows. Let's say this row. This row would also be considered as a Series. Any row or any single column will be a Series. So Series is essentially one-dimensional because it is either a single row or a single column. Whereas DataFrame is two-dimensional because it will have m number of rows and n number of columns.

So let's see how many rows and columns this DataFrame has. So let's see df.shape. So this DataFrame has got 15,929 rows and 21 columns. You can see all the 21 columns by scrolling. Okay.

So for our simple analysis in this notebook, okay, we'll work with a simple DataFrame that I'm going to create. Okay. I'm also going to import NumPy and random module to create some random numbers here. Okay. So I'm importing Pandas as pd, that is the alias that I'm assigning it to the Pandas module functions. And NumPy is being imported as np, that is the alias for NumPy. And I'm importing the random module. Okay. So I'm fixing the seed for this random. Okay, this ensures consistency so that every time you run the code, you get the same DataFrame. Okay.

So I have few names here and their corresponding genders, and I'm going to create some age values, eight random numbers between 21 to 25. You can change this range. I'm also going to create some marks in these three subjects. Okay. So total eight values of subjects, eight values of marks in the range of 60 to 95. Highest would be 95, lowest would be 60. And then those list comprehensions. This is a list comprehension. If you see, this is also a list comprehension. And this is also. So these lists are going to be provided to this dictionary. So this is a dictionary that I'm creating. This is the key and this is the value. So values are all lists. Okay. So names is a list here coming from. So names is a list coming from here. Genders is again a list coming from here. Age is another list coming from here. Marks is another list which is again coming from here. And finally, subjects is also a list which is coming from here. Okay. So I have populated all the five lists with these five keys. So let me execute this. I'm passing a dictionary. Ultimately, this is a dictionary from here to here, it's a dictionary, you can see this curly bracket. So this dictionary is now being converted into a DataFrame object. Okay. So this is my DataFrame. I've got the names, gender, age, marks, and subjects. Okay. So if you want to see how many rows and columns are present, we can see df.shape. So I have got eight rows and five columns. And I and have a quick statistical summary of my DataFrame. Okay.

So the statistical summary will only take the numeric columns. I have got only age and marks as the numeric columns. Uh, the other three columns, that is the name, gender, and subject. These are the three categorical columns. So that is why they are not included in this summary. Okay. So in this summary, what do I have? I have the count. How many values are present in that column? The mean of that entire column. So the mean of age values is 23.375 years. The mean marks is 78 something. The standard deviations. I have the minimum age is 21. Maximum age is 25. 50 percentile is the median value. Same goes here. 64 is the minimum marks. Maximum is 91. And the median marks is 77. These are the uh first quartiles. This is the second quartile, which is the median value, and this is the third quartile, and this is the fourth quartile, also the maximum value. If you want, you can round these values as well to, let's say, two decimal points for easy visibility.

Similarly, if you want to have the statistical summary of the categorical columns, you can use describe and include the object category. Okay. So I have eight unique values in the name, two unique values in gender, and three unique subjects. So the topmost uh, anyway, for name, it doesn't matter because all the names are anyway unique. But for the gender, I have got the most frequent value is male with a frequency of five, and the most frequent subject that I have in my DataFrame is statistics with a frequency of four.

Let's look into the quick information about this DataFrame by using info() function. So info() function gives me a quick summary of the DataFrame. Eight entries means eight rows are present. Total five columns are present. So you also get the information about the rows and columns and how many non-null values are present in each of these columns. You can also see the data type of each column. So if you see name is in the object data type, gender is also the object data type. Subject is object data type. That's why I used object here so as to get the statistical summary. Okay. The other two are present as integers. So the age and marks, these are present as integers. So you can actually have a look into the data type of each of the columns by using the info() function.

Now let's have a look how can we select and filter data. Okay. Let's say I want to select only the marks column out of my DataFrame. So this is a Series. You remember, individual columns of my DataFrame are going to be Series. Let us check the data type of the DataFrame. First, it shows pandas.core.frame.DataFrame. And if I do the same thing for my type and df.marks, you can see the data type here is Series. What essentially it means is this is basically a Series, whereas that entire DataFrame, this was my DataFrame, and that's 2D.

Now let's do some conditional indexing. Okay. So you can also select multiple columns. Okay. So let me just show you how do we select multiple columns. Here I've only selected a single column, marks, but let's say you are interested in looking at two columns. So one is name, and the other is marks. So if you notice, what I have passed is basically a list of column names. A list of column names. Okay. For a single column, you don't have to go for a list. But if you're, but if you want to pull out multiple columns from the DataFrame, you need to pass it as a list of column names.

Now let's say you want to only display those rows. This is your complete DataFrame. Now you want to display only those rows. Okay, where the marks is greater than 80. So essentially, if you want to manually check where the marks is greater than 80. So this is greater than 80. This is greater than 80. This one. Okay. So you should be getting only three rows. Let's verify. So yeah, these are the three students who have secured marks greater than 80. So this is how you can select rows based on some column names. Okay. Or based on some conditions applied on the columns.

How do we select some rows? So to select rows, you need to use the function loc and iloc. Okay, let's see. So but before we understand the concept of loc and iloc, let's understand labels. So in a DataFrame, these are my column labels. These are my column labels, and these are my row labels. So what you see in bold, they are labels. Also, the DataFrame columns and rows have indices by default. So the default index starts from zero. So this is zero. This is 1, 2, 3, and 4. So the index of subject is four. Index of age column is two. Index of name column is zero. Similarly, these are my indices. Okay. So by default, the labels and the indices for rows they are same. So right now, there is no difference between these numbers, but Python knows based on what function you are using. So loc means it is going to assume that this number that you are providing is actually a label, and iloc means this number that you are providing is an index. Okay. So 0, 1, and 2. So these are the.

So another good thing about the iloc function is you can use that iloc function to use multiple to fetch multiple rows and columns. So let's say I want to fetch the first five rows. Super colon means start from start and end one before five. So index 0, 1, 2, 3, 4. This is iloc, remember. So index location from 0 to 4. So 0 to 4 I'm getting. Yeah. So similarly, you can also fetch columns. So df.iloc. Let's say you want to fetch all the rows and the first three columns, that can be done. So all the rows and first three columns. So 0, 1, and 2 because it's a index location. So it will stop one before three, that is two. 0, 1, 2. So this is 0, 1, and 2. Okay. You can also use it with a keyword like minus one. So df.iloc. So I want to fetch the last row. So last row means minus one. So this is your last row. Similarly, if you want to fetch the last column, that can be done. So last column is df.iloc. So in that case, all rows of the last column. So this is all rows of the last column. You can see the last column is subject. Okay. So this is my subject column.

You can see the subject column here. Yeah. So let me just write it down. So this is the last row and here fetch the last column.

Okay, you can also add and modify columns. So right now, what I'm going to do is, let me show you the data frame once again. So this is my data frame. I'm going to add another column here by doubling the marks. Okay, let's see. So I have fetched my marks column and multiplied that by two and saved the value in another column name. Now, since this column name does not exist, it will be created. So you can see the new column name is double marks and it is basically the double of the original marks.

You can also use some lambda functions and apply function to update the values. So, for example, let's say I want to give five bonus marks to all the students. Okay. So I have added five marks into the original marks for each student. That's what this lambda function does. So the values of x is fetched from the marks column because you are applying that lambda function on the marks column and each value of x will be added to five and that resultant value will be replaced back into the marks column.

You can also rename columns. So now, since I have already updated my marks, it's a good idea to update the marks column to updated marks. So now you can see your marks column is no longer marks. It is updated marks. And, uh, let's say you think now that this double marks column is no longer needed. You want to drop it or you want to delete it. So how do I delete? So you can use the drop method. Okay.

So the drop method can actually be used to drop both rows and columns. Okay. So this is the list of labels, either the row labels or column labels, which you want to drop. So if you want to drop multiple columns, then this will be a list of multiple column labels. So these are my labels. Okay, these are my labels. Axis equal to one means drop columns and axis equal to zero means drop rows. So essentially, whatever list of labels you are providing, Python will think that these are the labels for the rows if you make axis equal to zero, which is anyway the default. So if you do not provide axis, it will take axis equal to zero. In that case, it will start dropping the rows. In place is basically you want to drop the rows or columns permanently. That means in place equal to true. So you can make it false if you want to see whether you are dropping the right row or the right column or not. Anyway, the in place option is by default false. So you will have to explicitly set it to true, otherwise it will not be dropped permanently. It will be dropped from the view but not from the actual data frame. Okay. So make sure that if you want to drop it permanently, you make it in place equal to true.

Now let's see how can we index some particular values. So right now, if you see, I want to fetch the values present in the name column and the first index. Okay. So this will give me the entire name column and now I am fetching the index zero. So that index zero gives me RF patail. That's exactly what I got here. So you can also slice our data frames. So this is getting the first two rows. You can actually use the iO method as I taught before or you can directly give the index of the row labels.

You can reset your index also. Okay. So suppose, uh, your original data frame has the index as 0, 1, 2, 3, 4, 5, 6, 7, 8. These are your index. How do you know that these are the index? Uh, you can check df.index. So your index starts with zero, ends at 8. So 0 to 8, in fact, it will end at seven with a step size of one. And if you want to know what all columns are present, column names, so df.columns. So these are the column names present in your data set. Okay. So these are the column names and these are the row names, basically labels. So you want to set a new name to the row labels. So what I want is this name should it. So what I want is this named column itself to be set as the new index. I want to set this as a new index. So that is exactly what we are going to do by this code. So now you can see the names of this DF. Yeah. So the index instead of 0 to 8, which it was earlier showing, now the index values are the actual names of the students. So these are my row indices. Okay.

So this is a data frame which has proper labels for the rows as well as for the columns. So these are my column labels and these are my row labels. Now the indices are still 0 to the index of the first column is now 0, 1, 2, 3. The index of the first row is still zero and going up to 7. Okay, you can just check. So the index is always going to start from zero, whatsoever be the labels. So earlier when we created this data frame, the row labels and the indices were same, but now I have changed my row labels. The index cannot be changed. Index will always be numeric, auto-incremented. Okay.

So we can also do sorting. So let me sort this entire data frame based on the updated mark. So by default, it will be in the ascending order. What do you know? So you can do a shift tab tab and it shows ascending as true. So by default, it will be in the ascending order. Okay. So if you want to be in descending order, so then you have to make ascending equal to false. Okay. So now this is in descending order. 96, 93, 86, down up to 69. So this is pretty much in the descending order.

You can also sort values based on two columns. So whenever the age values are equal, then the sorting happens based on updated marks. So if you have a look here again, this is by default ascending order. So here the age values are same. So then sorting happens based on the updated marks. Okay. So whenever the age values are same, then the next level of sorting happens based on the updated marks.

You can also sort the index. So this is interesting. Until now, the indices were not sorted. Okay. Now I have even sorted my indices. You can see in the alphabetical order this time. A to S.

You can also reset the index. So let's say you want to get your names column back. Okay. So you have got your names column back. Okay. In place equal to true means that resetting of the index is done permanently. So your data frame has changed once again permanently. This has gone back to the original.

Now this time I'm creating couple of nan values or missing values in my data frame. So some of the values, see the fourth row, essentially the fifth row, index four. So this is the row where it will make a change and minus one means the last row. Now the columns 0, 1, 2. So essentially, and this is the place where it will be replaced with null value, also referred as nan. Nan is not a number, essentially a null value. So these two are going to replace these two positions. Okay, let's have a look. Perfect. So this is one nan here, another nan here. Okay.

Another thing that you might have noticed is the original data frame had the age values as an integer. The data type here was an integer. The moment we have these missing values as nan, the data type has changed. You can see that decimal point coming up. So this is no longer integer. It is actually in the float data type now. Okay, this is in a float data type. So you can verify that by using the info. So your age is no longer an integer. It is float 64-bit precision. Okay.

So you want to know how many missing values are there. So these are the positions where the missing values are present. That is marked with true. So true shows that these positions are missing values. So counting how many missing values by each column. So name column does not have any missing value. Gender also does not have any missing value. Age column has got two missing values and the other two columns don't have any missing value. Right? So column-wise, how many missing values are present? You can use this code to get.

So now you want to fill the missing values with the average value of that column. Let's say, uh, right now the missing value is present in the age column. So you can take the average age and fill the missing values. So fill NA function fills the nan values. That's what this function does. Filling the nan values. Okay, fill NA is fill the nan values in that column. So again, by default, that is false. So you'll have to make it true so that the filling happens permanently or replace the original column, either way. So after filling, replace the original column, that is one way, or you use the in place option. So now you can see this 23.5 and this 23.5 is coming because of the fill NA function. Earlier, these were nans.

So you can check the info and if you want, you can convert it back into an integer format. So you will lose some precision. So these decimal points will be ignored then. Okay. Or rounded to the nearest whole number. So I have type-casted it back to an integer. So it depends on your use case. You might want to do that. So here the function used is as type. As type changes the data type of a column to a specified data type. So I am type-casting it back to an integer. So you can type cast it to an integer 8-bit, 16-bit, or a 32-bit precision. So based on your use case. Okay.

So I'm creating a few more missing values after creating a copy of the data frame. So I'm creating a copy and from that copy I'm dropping some more, uh, values. So creating some more missing values in the updated marks. This time you can see this is one, this is one. Uh, other thing that you can do is instead of filling the missing values, you might want to drop the rows which have missing values. So let's say I want to drop these two rows which have the missing values. So that can be done by using the drop NA function. So the drop NA function removes the missing values instead of filling it with the average or the median value that you specify. So by default, axis is zero, which means the entire row will be dropped. Okay, whichever row contains a single missing value that will be dropped. Okay, any means any single row which contains a missing value will be dropped. So if you want to drop columns, then make sure that you keep axis equal to one. So let me just drop and you can see that the row number four and the row number seven, they got dropped. Yeah, row four and row seven are missing now. Good that we created a copy. So my original data frame is still the same, unchanged.

What if you have duplicates in your data frame? So I am creating a simple data frame with duplicates here. So this entire record is duplicated two times. You can see. So I want to drop the duplicates. So you can use the function drop duplicates. So by default again, it will drop the rows and by default again, it will have the in place option which is set to false. So again, you can set in place equal to true or type cast it back to the original data frame. So after dropping, replace your df2. So that is another option, or you use the in place equal to true, either of. So I have dropped my, uh, duplicate rows and, uh, you can also do some operations on your strings. So, for example, here the subject names, I want to convert them into lowercase. So I'm type-casting the subject values into a string and then converting that string into a lowercase and then replacing the original column. So after converting the string values into lowercase, I'm replacing my original column. So that's how I get here. So the subject now is becoming in lowercase. So all these operations you can do depending on your use case.

Now let's have a look how can we merge and join data frames. I'm creating a couple of dummy data frames, DF1 and DF2. I've got two names, A and B, uh, with marks set as 90, 80 and 1817. In the other one, I've got age of these two students, A and C, as 17 and 18. So I can do merge operation. So this is my data frame on the left side, data frame on the right side. On on will use a key. On will use a key. So you can specify the column which is to be used as a key. So this is the key. How? So if you are familiar with SQL, you might be already knowing inner join, outer join, left join and right join. If you're not familiar, uh, you can go ahead and read about these kind of joins on our website analyticsia.com. There are a lot of blogs which explain the concept of SQL joins. Okay.

So here I'm using the inner join, which will match the keys in both the data frames. So here the keys are AB. Here the keys are AC. So which key is common in both? A. And that is the reason I get only A here. So the key which is common in both will appear. That is the meaning of inner join. Okay. Outer join will have the keys from both the left and the right data frames. So I'll get, uh, A, B, as well as C. Yep. So outer join will give you all three. And if you notice here, in the case of B, I don't have the age values, and in the case of C, I don't have the, uh, marks. Okay. Because in your marks data frame, there was no C, and in your age data frame, there was no B. Okay. That is why we have the missing values of marks for C and missing value of age for B. Right? A was present in both the data frames. So both marks and age were updated accordingly.

You can also do a simple concatenation of your data frames back to back. Okay. So in this case, I created another data frame, uh, DF3, and this is coming from DF1. This is my DF1 and this is simply F3. So concatenation means the data frames will be just concatenated one below the other. Why one below the other? Because by default, the axis is again zero. If you change the axis to one, then they will be concatenated side by side. Okay, remember that. So if you want this kind of concatenation, you have to make sure that the axis is set to one. So axis equal to zero will give vertical concatenation. Axis equal to one will give it column-wise, that is horizontal concatenation. So you can see by default axis is set to zero and that is why it was concatenated vertically.

You can also do group by and pivot tables. So let's do a group by quickly. So I'm grouping all the data frame columns or values by their subject in this case. Let's say, so subject-wise, I want to have the mean of the updated marks. That's what this code says. So can you give me subject-wise updated marks mean? So you can also do a sum function here. So fetch the mean marks. Okay. So this is correct. So similarly, you can go for the describe function where not only the mean value but also the other statistics are visible. Here you can see. So this 96, 85, 78 is already there in the mean column. But alongside, you have the min and max values also. So subject-wise minimum and maximum values. 96, 96. That's a single value. That's why the standard deviation is not a number. So generative AI is present only once in the entire data frame, which is why the standard deviation cannot be computed. So for Python, the standard deviation is 7.2. For statistics, 7.14. The minimum marks obtained for Python is 79, maximum is 93. And for statistics, it is 69 and 86. Seems as statistics, uh, paper was a bit difficult. Okay.

So similarly, you can create pivot tables. This is also similar to contingency tables. Okay. So here again, you can set which column is to be set as index and which column will be set as the value. So here I have the aggregation function. Now your aggregation function can be, uh, mean, it can be minimum, it can be max function, it can be sum function, or any of these. So or median. So several aggregation functions you can apply on this values. So the aggregation function is always applied on the values and you can provide index. So here goes the index. So here goes the index and the aggregation functions are applied on the values here. So you can also apply multiple aggregation functions if you want. Yeah, you can see. So this is the mean marks and the minimum marks. So multiple aggregation functions can be easily applied.

So what is the difference between pivot tables and group by then? So group by typically works for single-dimensional summaries, whereas pivot table is more flexible and ideal for multi-dimensional summaries. Let's take an example. So here I'm creating a single-dimensional summary. For each gender, I want to get the average updated marks. So the females, uh, updated marks average value is 82.33, and for males, the average is 83.6. Now, not only the average of the marks, but let's say I want to get the average marks of each gender across the three subjects. So that is where the pivot table would be a better choice. So if you see, so for each gender and each subject, so these are the three subjects. So subject one, subject two, subject. For each subject, I've got my mean aggregation function. Here is the mean. So this is the average marks for each of the three subjects, and these subjects are, uh, being put in the columns. Can you see the columns are in the subject, and index is present here in the rows. Okay. So index always refers to the rows, columns anyway in the columns, and the values for which the aggregation function is going to be applied. Okay. So this aggregation function is going to be always applied on the values. So that is how you can create more complex summaries by using pivot tables.

Finally, let's have a look into how can we use apply and map functions. So, I'm, uh, so we have already seen the lambda function being applied on a single column, right? We have already updated that. So, these are the marks. So, let me decrease the marks by 10 to get the original marks because I have already added it two times. So I've decreased the marks by 10 to get the original marks. Now, uh, here I've written a custom function which I'm going to apply. Here I have applied a simple lambda function. But instead of applying a simple lambda function, you can also apply a custom function. So what does this do? So I'm going to assign the grades based on the marks. So if the marks is greater than 85, then I'm going to return a grade. If the marks is between 70 and 85, it is going to be B grade. And if the marks is less than 70, it will be C grade. Okay. So after applying the function on this column, I'm creating another column called as grade. So I've got the grades of the students as well.

Now you can also map, um, values of a particular column to some other value. So let's say in this case, I'm mapping males to zero and females to one. This can be done for, uh, label encoding, typically used in machine learning. So I've created a new column called as gender new, in which zeros represent males and ones represent females. So how was this done? By simple mapper. So your mapper is a dictionary where the original value becomes the key and the new value becomes the value.

So you can also use pandas to handle dates and times. Let's see here. Here I've created a simple data frame, uh, with some names and dates. You can see these dates are actually strings. How do I know? Because of the data type present in the info. So the info gives me an object data type. So I can convert those strings into actual datetime objects by using the datetime function. So I'm converting the strings into a datetime. That's what the two datetime function does. And replacing the original column. So after doing that, you can see visually it still looks very similar to what it is, but here if you see the df.info, now it is actually present in a datetime format. So you can also extract columns, dates, months, uh, week of the year, date of the year, month of the year, quarter information. So you can extract all these information from a datetime column.

Now that's it for this video on this extensive coding tutorial on using pandas for data analysis. Today we are going to dive into database and SQL fundamentals. Understanding databases and how to use SQL is an essential skill for anyone working with data, especially in the world of AI and machine learning. In this module, we'll learn about databases, their types, the role of SQL, and how these skills fit into AI workflows. Let's get started.

In this video, we'll cover several key topics. We'll start by defining what a database is and explore different types of databases. We'll discuss about why databases are essential, especially in the context of AI. Then we'll introduce SQL, the structured query language, which is the standard language for interacting with relational databases. We'll go over basic SQL commands. And finally, we'll see how databases connect to AI agents.

A database is essentially an organized collection of data. Think of it as a digital filing system that allows you to store, retrieve, and manage data efficiently. Databases are made up of data, of course, but they also rely on hardware like servers, software, the database management systems, and users who would interact with the data. This organization allows for quick access and manipulation of large amounts of information, which is crucial in many applications, including AI. Think about all the apps and services you use daily, like social media, banking, shopping, etc. All of these services rely on databases to store massive amounts of information about users, transactions, products, and more.

There are mainly two types of databases: relational and non-relational. Relational databases organize data into tables with rows and columns, much like a spreadsheet, and they use SQL for querying and managing data. Examples include MySQL, PostgreSQL, and SQLite. They are great for storing data with fixed relationships. Think of customer records linked to orders. Non-relational databases, often referred to as NoSQL databases, store data in formats like documents, key-value pairs, or even graphs, allowing them more flexible schema designs. Examples of NoSQL databases include MongoDB, Redis, and Cassandra. These are often used when data doesn't fit neatly into tables or when scalability is crucial, such as in social media applications.

Databases play a critical role in data management by providing a centralized place to store and organize data. They are designed to handle large volumes of data, making them scalable solutions for growing needs. Databases ensure data integrity, meaning that the data remains accurate and consistent over time. They also support concurrency, allowing multiple users to access and modify data at the same time without conflicts. Most importantly for us, databases serve as the foundation for AI because data is the fuel that powers AI algorithms.

Now, let's talk about SQL, the structured query language. SQL is the standard language used to communicate with relational databases. It's used to create tables, insert and retrieve data, and perform updates and deletions. If you think of a database as a library, then SQL is the librarian helping you find the book you need, update information, or even add new books to the shelves. Understanding SQL gives you the power to interact with databases effectively, retrieving the exact information you need for your applications.

Let's look at some basic SQL commands. The `CREATE TABLE` command is used to create a new table in the database and define the structure of that table, including its columns and data types. `INSERT INTO` allows us to add new records or rows to a table. `UPDATE` lets us modify existing data. `DELETE` allows us to remove data from a table. And this `SELECT` command is used to retrieve data from the database. It's one of the most commonly used commands.

Databases are vital in the field of AI for several reasons. They provide efficient storage solutions for the massive data sets often required for training AI models. Databases make it easier to access and pre-process data, which is a crucial step in any AI project. They offer a reliable and consistent source of data, ensuring that models are trained on accurate information. For AI agents that need real-time data, databases can provide up-to-date information, enabling more dynamic and responsive AI systems.

To utilize databases in AI applications, we need to connect them to programming languages like Python. This is done using libraries and tools that allow us to execute SQL queries and manipulate data within our code. Python offers several libraries such as `sqlite3` for SQLite databases, `mysql.connector` for MySQL, and `psycopg2` for PostgreSQL. ORM frameworks like SQLAlchemy can simplify database interactions by allowing us to work with databases as if they were regular Python objects. Once connected, we can extract data, pre-process it, and feed it into AI algorithms for processing and decision making.

We have already discussed connecting Python to SQL databases in our previous module. But let's recap briefly. Using Python, we can connect to databases like SQLite and PostgreSQL using libraries such as `sqlite3`, `psycopg2`, and SQLAlchemy. The benefit of doing this is that it provides efficient data management, allowing Python scripts to read, update, and store structured data. This capability is crucial for building robust data-driven applications that are the backbone of AI and machine learning projects.

In this video, we'll explore how to connect Python applications to SQL databases like MySQL and PostgreSQL. We'll discuss the advantages and use cases, especially in the context of AI agents and generative AI applications, and provide code examples to illustrate the concepts. By the end of this session, you will have a foundational understanding of how to integrate databases into your Python projects, enabling more dynamic and data-driven AI solutions.

Our agenda for this video includes understanding the reasons and benefits of connecting Python to SQL databases. We'll delve into practical code examples for connecting to both MySQL and PostgreSQL databases. We'll also learn how to execute SQL queries directly from Python and pandas, and discuss how to manipulate data once it is imported into Python.

Why connect Python to SQL databases? Connecting Python to SQL databases offers numerous advantages. Databases are designed to efficiently store and retrieve large amounts of data, which is essential for AI applications that require access to extensive data sets. They provide scalability and high performance, ensuring that your application can handle growth. Real-time data access allows AI agents to make decisions based on the most current information. Additionally, integrating databases with Python enables seamless data pre-processing, which is crucial for training accurate AI models.

The use cases for connecting Python to SQL databases are vast. For instance, AI agents needing dynamic data can retrieve and process customer information on the fly. Chatbots are a great example. They need access to updated customer data to provide helpful responses. Another use case is generative AI applications that depend on stored data sets for training and inference. Imagine developing a recommendation system for a streaming platform. The data for user preferences and content metadata has to be pulled from a central repository, which will be a database. Additionally, logging and monitoring play an important role in maintaining systems. Databases are used to store user interaction logs, which can be analyzed to improve performance and user experience.

Tools for connecting Python to SQL databases. There are several tools and libraries available for connecting Python to SQL databases. For MySQL, popular Python libraries include `mysql.connector.python` and `PyMySQL` libraries. For PostgreSQL, there's `psycopg2` and `pg8000` library. For those who prefer an ORM framework that abstracts away the raw SQL code, SQLAlchemy is a powerful option that supports multiple databases. ORMs help in interacting with databases using Python classes and methods instead of writing raw SQL queries. Additionally, pandas also integrates well with SQL functions like `pandas.read_sql_query` which makes it easy to import data into a data frame, while the `to_sql` method lets you write data from a data frame to a SQL table.

Another interesting concept is in-memory databases, and SQLite is a great example. An in-memory database resides entirely in your system's RAM, making read and write operations extremely fast. However, the data is volatile, meaning it is lost when the application terminates. In-memory databases are perfect for testing and development because they provide a fresh database state for each test run. They are also ideal for applications requiring quick data access, like caching or managing session data that doesn't need long-term persistence.

Creating an in-memory SQLite database in Python is very straightforward. By passing `:memory:` to the `sqlite3.connect` function, we tell SQLite to create a temporary database in RAM. In the code example, we establish a connection, create a table, insert data, query it, and finally close the connection. All operations are performed in memory, and no data is written to the disk.

When to use in-memory databases? In-memory databases are incredibly useful in scenarios where speed is crucial because they operate in RAM. Read and write operations are much faster than disk-based databases. This makes them ideal for applications that need to process data quickly. They're also excellent for testing and development. Using an in-memory database allows developers to run tests without the need to set up a test database environment. Each test can start with a clean slate, ensuring test isolation and repeatability.

Limitations of using in-memory databases. Volatility. Since the data resides in volatile memory, it doesn't persist after the application ends. This means you shouldn't use in-memory databases for applications where data persistence is required. Memory constraints. Additionally, they are constrained by the available system memory, making them unsuitable for large data sets. Concurrency. They may also have limitations in handling concurrent connections, which can be a concern for multi-threaded applications. Understanding these trade-offs is essential when deciding whether to use an in-memory database for your project.

To overcome these limitations of in-memory databases, we have persistent databases like MySQL, PostgreSQL, MongoDB, Redis, Cassandra, etc.

Connecting to MySQL database. Let's look at how to connect to a MySQL database using the `mysql.connector.python` library. First, we

import the library. Then we attempt to establish a connection using MySQL.Connectctor.connect function, providing the necessary parameters like host, user, password, and database name. We wrap this in a try-except block to handle any connection errors gracefully. If the connection is successful, we print a confirmation message.

Connecting to a PostgreSQL database using psycopg2 follows a similar pattern. We import the psycopg2 library and then attempt to connect using the psycopg2.connect function. Again, we provide the host, user, password, and database names. Wrapping the connection attempt in a try-except block helps us to catch and handle any errors that may occur during this process.

Once we have established a connection to the database, we can execute SQL queries directly from Python and fetch the results into a pandas DataFrame. This is done using the pandas read_sql query function, which takes the SQL query and the database connection as arguments. After executing the query and storing the results in a DataFrame, it is a good practice to close the database connection. The process is similar for both MySQL and PostgreSQL, with the main difference being the library used for the connection.

Limitations of direct SQL queries. While writing raw SQL queries in Python, there are some limitations we need to consider. First, SQL queries can become complex and difficult to manage, especially for large projects. Writing and debugging long, intricate SQL queries can be time-consuming. Another issue is security. Raw queries can be vulnerable to SQL injection attacks if not handled properly. Always ensure that user inputs are sanitized and parameterized to avoid these risks. Lastly, there's maintainability. As your code application grows, maintaining raw SQL queries in Python code can become cumbersome. It's harder to refactor and keep the code clean when you have lots of embedded SQL statements.

To overcome the limitations of raw SQL queries, you can use Object Relational Mapping (ORM) frameworks like SQLAlchemy or SQLModel. ORM allows you to interact with your databases using Python classes and objects instead of raw SQL code. This offers a layer of abstraction that makes development faster and code easier to maintain. With an ORM, you can create and modify tables by simply defining classes in Python. This not only improves productivity but also makes it easier to switch between databases if needed, since the same Python code can be used with different database backends. SQLAlchemy is well-established in the Python ecosystem, providing robust features and support for various databases. SQLModel builds on top of SQLAlchemy and Pydantic, simplifying model creation with modern Python syntax and type annotations. It streamlines the process of defining database models and integrates well with FastAPI, making it ideal for building APIs and AI applications that require data persistence and validation.

Let's summarize what we have learned today. Connecting Python to SQL databases allows for efficient and scalable data handling. We explored libraries like MySQL Connector/Python and psycopg2 for database connectivity. Pandas integrates seamlessly with SQL databases for data manipulation, making it a powerful tool for data analysis. ORM frameworks like SQLAlchemy and SQLModel provide advantages over raw SQL queries, including abstraction, productivity, and easier maintenance. These concepts are fundamental for building data-driven applications, and mastering them will make your workflows much more efficient.

Hello and welcome back. In this video, we'll explore working with files and databases in Python. Understanding how to handle files and databases is crucial when building real-world data-driven applications. We'll dive into CSV and JSON file formats, how to interact with them using Python, and how to integrate them with databases. Let's get started.

Here's what we'll cover in this video. We'll start with an introduction to file handling in Python, focusing on CSV and JSON files, and then we'll provide code examples to illustrate how to read and write these file types.

So, why do we need to work with files? Data storage and exchange is one of the most common uses. Files, particularly in formats like CSV and JSON, make it easy to store, share, and exchange data between different systems and applications. CSV files are used for tabular data, while JSON files are widely used for structured data exchange between APIs. Additionally, files are often used in data pre-processing to prepare data for AI models and analytics. They provide an easy, readable way to log, store, and transfer the information needed for our machine learning workflows. Python provides built-in modules like `csv` and `json` to handle CSV and JSON files easily. Additionally, third-party libraries like pandas offer powerful data manipulation capabilities, making it easier to pre-process data for AI applications.

CSV files are commonly used for storing tabular data like spreadsheets or simple databases. Python's `csv` module allows you to read these files using the `csv.reader` function and write them using `csv.writer` function. The `csv.reader` function reads each line of the CSV file and returns it as a list. Similarly, `csv.writer` function lets you write rows of data back into a CSV format. This module is perfect for small to medium-sized datasets and for those who want to work with CSVs without needing advanced data manipulation. However, for large datasets or when you need advanced data manipulation, pandas is a better choice. It allows you to read CSV files into DataFrames, which are powerful data structures for data analysis and pre-processing.

Here's an example of how to read and write CSV files using pandas. First, we import pandas and use `pd.read_csv` function to read the data from `data.csv` into a DataFrame. We can then display the first few rows using `df.head` function. After performing some data manipulation, such as calculating a new total column, we can write the DataFrame to a new CSV file using `dataframe.to_csv` function. Setting `index=False` prevents pandas from writing the row indices into the file.

Reading and writing JSON files with Python. The `json` module is a part of Python's standard library and provides methods for reading and writing JSON data. The JSON objects are similar to Python dictionaries, making it straightforward to work with them. You can use `json.load` function to read JSON data from a file into a Python dictionary. Conversely, `json.dump` function allows you to write a dictionary into a JSON file. For handling JSON data in string format, you can use `json.loads` function and `json.dumps` function.

Working with JSON files using the `json` module. In this example, we read data from `data.json` using `json.load` function, which loads the JSON content into a Python dictionary. We can then access elements of the data structure using `data['employees']`. After modifying the data, adding a new employee in this case, we write the updated dictionary back to the file using `json.dump`. The `indent=4` parameter formats the JSON file with indentation for better readability.

In conclusion, handling files like CSV and JSON in Python is straightforward with the standard libraries and enhanced by tools like pandas. I hope this session has provided you with valuable insights into working with files and databases in Python.

So, in this coding tutorial, we'll see how to connect with SQL databases using Python. First, we'll start with in-memory databases which are created using the `sqlite3` package, and then we will connect with one MySQL database and another one which is a PostgreSQL database, both hosted on an AWS server, which is most likely going to be a production environment in any real-life scenario for any enterprise. So, let's get started.

Okay. So, you will need to install the `sqlite3` package if it is not present. So, you can do that by doing `pip install sqlite3`. I'm not running this because this package is already installed in my system. So, and then pandas would be most likely installed in your system if you are using Anaconda or Google Colab.

Here, I'm creating a SQLite database by connecting it to memory. So, this database will be created in your RAM, which is the memory I'm going to specify here. And then, once the connection has been made to that database, I'm going to create a cursor object. This cursor object will be used to execute the SQL commands. So, that has been done. Uh, now I'm going to execute a `CREATE TABLE` query, which is going to create a small table named as `students`. And this `students` table will have columns like `ID`, `name`, `age`, and `marks` for few subjects like `statistics`, `Python`, `machine learning`, and `deep learning`. So, these will be the columns in my table. The name of the table is `students`, and this `students` table will be created in your database. So, I'm passing that entire query string to `execute`, and who is going to execute? The cursor. Okay, let's see.

Okay, so the `students` table has been created. Now I'm going to insert some data into that table. Okay, so for that, I'm using the `INSERT INTO` command. So, `INSERT INTO students` and these are the column names, and there are some entries that I have specified. So, these are some names of the students, their age, then marks in statistics, then the marks in Python, marks in machine learning, and deep learning. Okay. So, these are pretty much standard SQL commands. Uh, if you are familiar, you can easily relate to it. And, uh, if SQL is a bit new to you, I would strongly recommend you to get some basics on SQL.

So, the query string is again executed, and, uh, it is committed. Execution means it will execute and insert the data into the table temporarily, and if there is any error, we can always roll back. Once there is no error, I can ask it to commit. So, commit is like a permanent operation. So, perfect. So, data has been inserted. Now you can fire some query strings directly to your table. Okay.

Here, a simple query I am passing here is `SELECT * FROM students`. That means I want to select all the rows from the `students` table. So, I'm going to use the `read_sql` function from pandas. So, I'm passing my query string and the connection object. Let's see. So, after the data is fetched, it is directly converted into a DataFrame, as you can see, right? So, this is the data which we created into my `students` table. So, you can also update some of the records, or delete the records, or filter your records based on all these SQL query commands.

So, here, `UPDATE students SET Python_marks = 95 WHERE name = 'Sophia Smith'`. So, basically, what I'm doing is, um, there is one student called as Sophia Smith. I want to update her marks from 78 to 95. Let's say there was some typo. So, that is the number which I want to change. Okay. This 78 to 95. So, `Python_marks = 95 WHERE name = 'Sophia Smith'`. So, after changing, you always should be committing your database once again. Okay, this is a good. So, for query commands, you don't need to commit, but for change commands or update commands, you should always commit. So, you can verify.

So, here I am passing my query. This query is the original query that is `SELECT * FROM students`. So, the same query I'm passing once again here with the connection object. Let's see if that has been changed. You can see that 78 has been changed to 95.

Once you are done with all the SQL query operations on your in-memory database, you can close the connection. It's a good programming practice to close the connection. So, that database is released from the memory. Okay? And also close the connection. So, it's no longer occupying any memory in your Python. Perfect.

So, next, we are going to connect a MySQL database with Python. Uh, for this, we'll be installing `PyMySQL` and `mysql-connector-python`. So, these are the two connectors, most popular connectors available to connect a MySQL database to Python. Okay. So, once the installations are done, let's first use the `PyMySQL` connector and connect with Python. So, import the `PyMySQL` library. And here is the database configuration. Right now, my database is present in AWS RDS. It's an RDS instance. Uh, this is the default port used by the MySQL databases on AWS RDS. RDS is Relational Database Service from AWS. There is an admin username for each database. There is a password for that admin user, and there is a database name. So, make sure you have all the details before proceeding. This can very well be a database hosted on your local machine. So, if you install MySQL Studio, you will be able to create a database in your local machine as well. In that case, the host will be replaced with your local machine's URL. So, port more or less would remain the same, and the other details, just fill in as it is needed.

One strong suggestion here is, we should not be hardcoding these details in a Python notebook. It is not a good programming practice. Ideally, we should be using environment variables to store these sensitive information because if you have, or rather, if you share this Python notebook with anyone else with all these details filled in, they would be able to access your database and make irreparable changes to your database. They can literally add anything or delete anything from your database. Right? So, this is not a good programming practice to have these details right inside your notebook. Okay. Later, in the next video, we will see how to circumvent this process. We'll create environment variables for all these database configurations and using user access keys. But for now, to keep things simple, let's go ahead. Okay.

So, let me execute this. And now I'm going to use the `connect` function from `PyMySQL`. Pass all the details to that `connect` function and start my connection. So, if this message is printed, it means I am already connected. There is no error here. Just as we did in the previous notebook. Okay. We are going to create a connection cursor, which is going to execute my SQL queries. So, once my connection cursor is created, I'm passing the same simple `SELECT * FROM students`. So, my `students` database currently is now on an AWS RDS instance. I've got, I think, eight records in that. So, I'm limiting to the first five. Okay. So, I'm fetching the first five rows essentially, and then fetching all those five rows, and then printing row by row. Once you are done, you can close the connections. Okay.

So, this is how it comes up. So, you can see that each row is fetched as a dictionary. Okay. Each row is fetched as a dictionary. I have `student_id`, the name of the student, the age, `statistics`, `Python`, `machine learning`, and `deep learning` marks. Right?

Now, let's see how can we use the `mysql-connector-python`. So, first, you'll have to install that. Once it is installed, I'm importing that connector and also the error class from that `mysql-connector-python`. Now, we'll try to establish a connection. It's a good programming practice to establish your connection within a `try-except` block, so that if there is any issue or error which occurs while connecting to your remote database. This error could be because of the invalid host name or the database name, the username is not correct, or the password is invalid, or the port name is wrong. It could be any of these five reasons. You will get an appropriate error. You will be able to catch that error and print it. Okay, so that's exactly what I'm doing here. So, if there is an error, I'll catch that error and print this. Okay. If everything goes well, this is what will get printed: "Connected to MySQL database." Perfect.

So, it's connected, and, uh, this part is again the same. I'm just creating a cursor to execute my SQL query and then fetching all the five rows and printing it one by one. So, the main difference here is, you get the output as a tuple. You directly get the values. Okay. In the previous case, you are getting the key and the values. The key names were the column names. So, the keys are the column names. Okay. Sorry. This is the key. This is the key. This is another key. So, key names are basically the column names, and the values are there. So, it's a key-value pair, which is a dictionary. So, each row comes up as a dictionary. However, in this case, you'll get directly the values present in each row as a tuple. So, one tuple represents one row. Okay? So, this is one difference. So, you can decide what suits your application better. Okay? Uh, okay. And this is a more graceful way of closing your connection. If it is already open, okay, if it is connected, then close it. If it is not connected, then anyway these two statements will not execute.

Now, let's see how can we connect pandas to MySQL database and execute direct queries using pandas. Okay. So, the first two steps are already done. You can actually use `mysql-connector-python` or `PyMySQL` and `SQLAlchemy`. So, you will need a `SQLAlchemy` engine to connect to Python pandas, so that you can fire the SQL commands. So, the SQL commands will be passed to the `read_sql` function once we have established the connection using `SQLAlchemy`. So, this will be needed. Okay. Plus, one of these two connectors will be needed. So, let's see. You'll need to uncomment this to install `PyMySQL` or `mysql-connector-python` along with `SQLAlchemy`. If this is already installed, you can delete this part and just install the `SQLAlchemy` part.

So, once your `SQLAlchemy` is installed, I'm just importing the `SQLAlchemy` `create_engine` function. Okay. So, you are creating a connection string first. You are passing the username, password, host name, and the database name here. Right? So, this is your connection string, which you are passing it to the `create_engine`. So, you are creating a `SQLAlchemy` engine. Let's see. Done.

So, I'm going to use that connection engine into the `read_sql` function of pandas. Okay. So, whatever your query string, as usual, and your connection engine. So, these two things will be needed to fire any SQL query and directly fetch the results as a DataFrame. So, this is the biggest difference. Here, we were fetching the details and row by row, we were printing it. Okay. We are not getting the DataFrame here. Neither we are getting the DataFrame here. So, here we are going to fetch all the details directly as a DataFrame. So, that is the main advantage here. Perfect. So, all my data is there as a DataFrame.

So, let's execute some SQL queries using the pandas `read_sql` function. So, first query is the same, reading all the data from the `students` table. Now, I'm going to filter records by some condition. Let's say I want to fetch only those students who scored more than 85 in Python. So, these are the students who have scored more than 85 in Python. You can just check their marks. These are all greater than 85. So, there are 20 students with the index 0 to 19. Out of 20 students, 16 students have scored more than 85 in Python.

Next, you can also select specific columns, for example, `name` and `statistics` in this case. So, `SELECT name, statistics FROM students`. So, now I'm getting the name and the statistics marks for all the students in this case.

You can also order the records by a specific column. So, in this case, let me retrieve all the students and order them by their marks in machine learning subject. `SELECT * FROM students ORDER BY machine_learning DESC`. `DESC` is for descending order. Let's see. So, machine learning marks, and in the descending order. Okay.

So, another SQL query which will calculate the average marks for all the students. Okay. In the statistics course. So, first, it fetches all the marks from the statistics subject, and then using the `AVG` function, it calculates the average. So, `statistics` is fetched, and then `AVG` function is calculated. So, 84.55 is the average statistics marks for all the 20 students.

Now, another SQL query which will count the total number of students in the table. So, this should be 20.

You can also group by the students by their age and calculate the average marks. So, so 84.75 is the marks obtained by all the students who have an age of 20 years. 81.4 is the marks obtained by all the students who have an age of 21 years, and so on. So, you can have a subject-wise average. You can also have an age-wise average that you can decide what you want to do.

One final note, um, is although `SQLAlchemy` engine is quite efficient because the connection is automatically closed the moment the query is executed. Here, you have passed the engine, and once the query is executed, the engine closes the connection from the database. But there could be scenarios where the connection may not be closed because there are multiple such connections open, and multiple people are accessing the same resource, the same database. So, in that case, it is a good programming practice to close the connection once you are done completely. So, if you're repeatedly using this engine and you want to ensure that all the connections are closed at the end, then you can use the `engine.dispose()` function. So, this is typically not needed on a single usage, but if there are multiple users connected to the same database and repeatedly doing a lot of SQL operations, then this might be needed once everything is done. Okay.

So, in the last part of this notebook, we are going to connect a PostgreSQL database using Python. So, as discussed in the presentation in the previous video, I'm going to use the `psycopg2` package. So, first, try to install `psycopg2` using this. So, first, try to install `psycopg2` using this command. So, just uncomment this and execute. If there is any error in the installation, then try the next one. So, depends on the machine and the server. Okay, the configurations. So, sometimes the first one would work, or else the next line would work, but try in that sequence. So, I have already installed the `psycopg2` library.

And then, as usual, you have your database configurations. So, this is the PostgreSQL database, which is again hosted on my RDS instance on AWS. So, this is the endpoint. Uh, PostgreSQL database connections typically use port 5432 by default. This is the name of my database. The admin username for my database, and the password. So, with these, let me start the connection. So, importing the `psycopg2` library and then, uh, also importing the error. It's a good programming practice to use `try-except` blocks again, as we did in the previous case. So, here I'm creating a function to create a connection. So, the advantage of doing this is, I can directly call this function, which will create the connection directly, and it will return that connection. So, I can use this connection in my pandas directly. So, the other things are pretty self-explanatory. We have been doing this. And to execute the SQL queries, again, I'm creating a function. So, it will need that connection and the query. After establishing a cursor to that connection, it will execute that query, and then finally commit, and whatever the results would be displayed.

So, "Connection to PostgreSQL database is successful." This is the query which I'm going to pass here and execute that query. So, this query was executed, and then you can close the connection. So, again, if you want to use the pandas `read_sql` function, so that all the results of the query are directly dumped into a pandas DataFrame, then you have to again go back to `SQLAlchemy` engine. So, again, the same connection string. So, this is the only thing which is different. So, the name of the database, name means the type of database. So, earlier it was MySQL, and then the name of the connector. So, here we had used the MySQL connector in the previous case. Here, I'm using the `psycopg2` connector, and then this part remains the same, and the rest of the code also is the same. You create a `SQLAlchemy` engine, and then query, define a query, and then execute that query with that connection. Okay, let's see.

Perfect. So, the same database I have been able to connect and retrieve all the records. Now, the advantage of `psycopg2` over MySQL connectors is that `psycopg2` directly also offers you a perpetual connection with your database, just like your `SQLAlchemy` engine. Okay. We use the `SQLAlchemy` engine within the `read_sql` function. The good thing about `psycopg2` is, you can actually use that connection that you are creating using `psycopg2` to fire SQL queries directly to your database. Okay. Although this is not recommended, and that is why this warning comes up. So, pandas only supports `SQLAlchemy` connectable, which is what we did previously. Uh, although it is working, but this is not safe, neither it is supported. So, this just worked, and it is also giving us a recommendation to use `SQLAlchemy` instead. Okay, which is what we did just before. So, this is not recommended. However, it works. Just in case you want to experiment.

Finally, once you are done, please do not forget to close your connection.

Hello everyone, and welcome to this video on working with APIs. In this video, we are going to explore the essential role that APIs play in software applications, especially in the context of AI agents and large language models. APIs are a crucial part of modern technology. They enable different systems to communicate with each other, allowing us to integrate powerful capabilities into our applications. Let's dive in.

Here's what we'll cover today. We'll start by defining what an API is and why it is important. Then, we'll delve into RESTful APIs and JSON data formats, which are foundational concepts for web-based APIs. We'll learn how to access APIs using Python's `requests` library, followed by best practices like handling rate limits, errors, and authentication. Finally, we'll discuss how APIs interact with LLMs and the creation of AI agents, and how to interact with these agents and LLMs hosted on other servers.

What is an API? Let's start by answering the question. API stands for Application Programming Interface, and it's essentially a set of rules that allows different software applications to communicate with each other. Think of an API like a waiter in a restaurant. You, as the customer, make an order. This is like sending a request. The waiter takes your order to the kitchen. That's the server, and then brings back your food, essentially the response from the server. APIs work in a very similar way. They take requests, process them, and return results.

How does an API work? An API is like a messenger that takes requests and tells the system what you want it to do, then returns the response back to you. It defines methods of communication between various software components. For example, when you use an app on your phone, the app connects to the internet and sends data to a server. The server then retrieves that data, interprets it, performs the necessary actions, and sends it back to your phone. The app then interprets that data and presents you with the information that you wanted in a readable way.

Why are APIs important? APIs are important for several reasons. Interoperability. They enable different systems to work together, allowing data and services to be shared seamlessly. Scalability. APIs can support multiple clients, allowing different platforms to access the same core services. Reusability. Instead of building new functionalities from scratch, you can use existing services through APIs. Innovation. APIs encourage the development of new applications by giving developers access to powerful functionalities and accessibility. They make data and services available, even those hosted externally, which would otherwise be challenging to access. For example, using the Google Maps API, developers can easily integrate mapping and geolocation features without building their own mapping systems from scratch.

Real-world examples of APIs. APIs are everywhere. When you log into a website using your Facebook or Google account, there's an API at work. When an app shows you a map or your current location, it's using an API for a mapping service. Payment gateways use APIs to process transactions securely. Even chatbots and virtual assistants use APIs to understand and respond to your user input. These are just a few examples of how APIs make our digital interactions seamless and efficient.

APIs in the context of LLM and AI agents. APIs play a significant role in AI. APIs allow developers to access pre-trained models and services like OpenAI's GPT models, Hugging Face models, and Google Cloud AI services without having to build these models from scratch. Using these APIs, you can offload the computational complexity to external servers, allowing you to integrate advanced AI capabilities into your applications seamlessly. This includes things like natural language processing, language translation, sentiment analysis, and even recommendation systems. APIs make AI accessible without requiring in-depth expertise in model development. Interacting with LLMs via APIs typically involves sending a prompt or user input to the API, which processes it using a sophisticated language model and then returns the generated response. For example, using the OpenAI API, you can send a prompt and receive a completion. While this is powerful, there are considerations like latency due to network communication, costs associated with API usage, and data privacy concerns, especially when sending sensitive information over the network.

Hello everyone, welcome to this video on accessing APIs using Python. And in this video, we are going to explore how to work with APIs in Python, focusing on practical techniques to interact with web services and extract data. APIs are the heart of many data-driven applications, and learning how to access them using Python is an essential skill for any data scientist or AI developer. Let's get started.

Before we dive into the code part, let's understand the basics of the HTTP protocol. HTTP, which stands for Hypertext Transfer Protocol, is the foundation of data communication on the web. It defines how requests and responses are structured between a client and a server. There are several HTTP methods you will use while working with APIs. `GET` is used to retrieve data. `POST` is used to submit new data to the server. `PUT` is used to update existing data. `DELETE` is used to remove data. These methods allow us to interact with data on a remote server. Understanding these HTTP methods is key to working with any API. Also, you will come across status codes. For example, 200 means success, 404 indicates that the resource was not found, and 500 represents a server error. These codes help you understand whether your requests have been processed successfully or if there were any issues.

RESTful APIs. Most modern APIs are designed according to the REST architecture. Representational State Transfer. REST is an architectural style used for designing applications. It relies on a few core principles. Statelessness. The server doesn't store any client state, meaning each request from the client contains all the information that the server needs to fulfill that request. Client-server architecture. This separates the user interface from data storage, simplifying how applications interact with APIs. Uniform interface. All API endpoints use standard methods and formats, making it easier to use and understand. Cachability. Responses can be cached for improved performance. RESTful APIs use HTTP methods like `GET`, `POST`, `PUT`, and `DELETE` for CRUD operations, which include Create, Read, Update, and Delete operations. This makes them straightforward to use once you understand these principles.

A RESTful API interaction involves several components. The endpoint or resource URL specifies where the resource is located. The HTTP method indicates what action you want to perform on the resource. Headers carry metadata information like content type or authentication tokens. This helps the server understand how to process the request or who is making the request. The body or payload contains data you are sending to the server, commonly in the JSON format. Finally, after processing your request, the server sends a response in JSON format, which also includes headers and the requested data. Understanding these components is essential for effectively working with APIs.

JSON is the most commonly used data format for APIs. It's lightweight and easy for both humans and machines to read and write. JSON structures data using key-value pairs, much like Python dictionaries. In the example shown, we have keys like `name` and `age` associated with the respective values. When working with APIs, you'll frequently send and receive data in JSON format. So, understanding it is crucial.

Python makes interacting with APIs straightforward. There are several libraries for API access, but two of the most commonly used libraries are `requests`. This library simplifies making HTTP requests and handles most of the complexity for you. `urllib` is a standard library for URL handling, though it can be a bit more cumbersome compared to `requests`. We typically use the `requests` library because it is user-friendly, concise, and supports all HTTP methods. It's perfect for making API calls, managing sessions, and even handling cookies, all in just a few lines of code.

Introduction to the `requests` library. To get started with the `requests` library, you need to install it using `pip install requests`, then you can import it into your Python script. The library supports all the HTTP methods you will need and simplifies tasks like handling query parameters and payloads. It also provides response objects that include status codes and the content returned by the API. The official documentation is a great source for learning more about its features.

Making requests. A `GET` request is used to retrieve data from an API. Let's look at an example of fetching public data from an API. You can make a `GET` request using `requests.get` function, passing the URL as a parameter. If you need to include query parameters, you can pass them as a dictionary using the `params` argument. The response object contains the status code, text content, and can be parsed as JSON if the response is in JSON format.

The response object returned by the `requests` library provides several useful attributes. You can check the status code to determine if the request was successful. A status code of 200 indicates success. The `headers` attribute contains the response headers, and `text` gives you the raw content as a string. If the response is in JSON format, you can parse it using the `response.json()` function. Always check the status code before processing the response to handle errors appropriately.

Making POST requests. `POST` requests are used to send data to a server. For example, while submitting forms or uploading data to an API. You can include data in the requests using the `data` or `json` parameters. When sending JSON data, it is common to set the `Content-Type` header to `application/json`. The `requests` library handles this for you when you use the `json` parameter.

Interacting with LLMs via APIs typically involves sending a prompt or input text to the API, which processes it using a sophisticated language model and then returns the generated response. For example, using the OpenAI API, you can send a prompt and receive a completion. When an API returns data in JSON format, you can parse it using the `response.json()` function, which converts it into a Python dictionary. You can then access individual data points using keys, just like with any dictionary. If the data is nested, you might need to access multiple levels of keys. Understanding the structure of JSON responses is essential for correctly parsing and using the data.

Let's start with the best practices for working with APIs. Read the documentation. Always make sure to thoroughly read and understand the API documentation. This will help you understand the endpoints, parameters, and authentication methods used by the API. Handle errors gracefully. Implement robust error handling to manage issues such as connectivity problems, incorrect inputs, or server errors. Respect the rate limits. Most APIs have rate limits in place to prevent abuse. Exceeding these limits can get your access temporarily or even permanently blocked. Secure your API keys. API keys are sensitive information. Store them securely. Use environment variables or a secure storage mechanism. Validate inputs and outputs. Ensure all your inputs and outputs are properly validated to maintain data integrity and prevent security vulnerabilities. And finally, use logging. Keep a log of API requests and responses. This helps in debugging issues and understanding what went wrong during interactions. Following these best practices will make your API integration much more reliable and secure.

Error handling in API requests. Error handling is a critical part of interacting with APIs. Here are some of the common HTTP status codes and what they mean. 400 Bad Request. The request was invalid, possibly due to incorrect syntax. 401 Unauthorized. Authentication is needed to access this resource. 403 Forbidden. The server understood the request but is refusing to authorize it. 404 Not Found. This means the resource you are trying to access doesn't exist. 500 Internal Server Error. This means that something went wrong on the server side itself. The best practices for handling errors include: Number one, always check the status code. This helps you understand whether the request was successful or if there's an issue. Use `raise_for_status()` method to raise exceptions for error codes automatically, which ensures that errors are handled in your Python script. Proper error handling makes your application more robust, ensuring that even if something goes wrong, your users experience minimal disruption.

API rate limiting. Now, let's talk about API rate limiting. Rate limiting involves restricting the number of requests a client can make to an API in a given period. The purpose of rate limiting is to prevent abuse and avoid overloading the server. Exceeding this limit can result in errors or being temporarily blocked. It also ensures fair usage among different clients. To handle rate limits effectively, check the headers. Many APIs include rate limit information in the response headers, such as the number of requests remaining. You can also implement delays. For this, you use the `time.sleep()` function to wait before making new requests if you are close to the limit. And finally, you can also use exponential backoff. Gradually increase the wait time after each retry if you encounter rate limit errors. It's crucial to respect the API's rate limit and monitor your application's request rate to avoid service disruptions.

Authentication is essential for securing APIs and controlling access to resources. It ensures that only authorized users can access certain data or functionalities. Common authentication methods include API keys, OAuth 2.0, and Bearer tokens. Now, let's talk about the authentication types one by one. Let's start with API keys. API keys are simple yet effective methods for authenticating requests. After registering with an API provider, you will receive a unique key that identifies you. You include this key in your requests either in the headers or as a query parameter. For security, it's better to include it in the headers. Remember to keep your API keys secure. Do not hardcode them in your scripts or share them publicly. Use environment variables or secure storage mechanisms to protect them.

Now, let's talk about OAuth authentication, which is an open standard for access delegation that allows third-party applications to access user data without sharing passwords. Platforms like Google, Facebook, and Twitter use OAuth to provide secure access. The OAuth flow typically involves the following steps. First, the authorization request is generated. The user is redirected to an authorization server to grant the access. Then, the user authenticates and authorizes the access. In the next step, the server returns an authorization code, which the application can exchange for an access token. And finally, this access token is used to make API requests on behalf of the user. OAuth is more complex to implement but offers enhanced security, making it ideal for situations where users need to grant access securely to their data.

Let's summarize the learnings from this module. First, we have learned that APIs are essential tools for enabling communication between different software systems. Understanding the basics of RESTful APIs and JSON data formats is crucial. Python's `requests` library provides a straightforward way to interact with APIs. Adhering to best practices ensures that your applications are robust and secure. APIs play a significant role in working with LLMs and building agents, providing access to advanced functionalities and data.

Hello and welcome to the video on building APIs with Flask and FastAPI. In this video, we'll explore how to create RESTful APIs using two popular Python frameworks, Flask and FastAPI. We'll cover the basics of setting up the endpoints, handling `GET` and `POST` requests, and also testing our APIs using tools like Postman. We'll also compare Flask and FastAPI to help you understand which might be a better choice for your projects.

Flask is a microframework written in Python. It is called a microframework because it doesn't require exotic tools or libraries, making it lightweight and easy to get started with. When we say Flask is used for building web applications and APIs, we mean it can handle both the front end (that is, what we see in our browsers) as well as the back end (which is the server-side logic and data processing part of web development). Flask provides the essential components to build web applications and APIs, such as routing, request handling, and a development server. Its simplicity and flexibility make it a popular choice for developers who want more control over their applications.

Let's break down the process of creating endpoints with Flask. First, you need to import Flask and create an instance of the Flask application. This is typically done with `app = Flask(__name__)`. Second step in Flask, we use decorators to define routes. A decorator is a special Python syntax that starts with the `@` symbol. It's a way to modify or enhance Python functions. The routes are the URLs of your API, for example, `/hello` or `/users`. The HTTP methods like `GET` and `POST` define what kind of operation we are performing. `GET` is usually for retrieving data, while `POST` is for sending data to be processed. Finally, we can run the app by using the command `python app.py` as you see here in the terminal or the command prompt. The endpoint we created can be accessed from the browser window as shown here.

Now, let's see the process of creating endpoints which uses the `GET` method for user requests. The first step, we will import the modules `request` for accessing the request data and `json` for JSON responses. So, then we would define the routes by using the `methods` parameter to specify the HTTP methods. As you can see in line number six, we are defining the route and `methods=['GET']`. Then in line number eight, we are accessing the data. You can also do this by `request.get_json()` in case the payload comes as a JSON format. And finally, we are going to return the response by using the `jsonify` function in JSON format if you want. Once you are done with the app creation, you can run the app, and we can access the app from the browser window. We have to go to the `localhost`, which can be accessed at port 5000 `/get_data`. That's the name of the endpoint we created here.

Here is another endpoint which uses the `GET` method for requesting inputs from the user. So, the `@app.route('/square', methods=['GET'])` is the name of the endpoint, and `methods` is equal to `GET` again. So, here this `/square` endpoint needs a number from the user, and by default, that number is given to be zero. In case the user is not providing any input, we'll be using zero. So, what we are returning is the square of that number. So, this is a dictionary, mostly this is the same format as of your JSON. Finally, we can run this app and access it from the browser window. You can see, uh, we have the `localhost:port_number` (5000)/`square`. So, `/square` is your endpoint, and `?` and beyond that, we have the name of the parameter, which is the `number`, which the user has to provide, equals to this is the actual number which the user has to provide, 5, 10, 15, 500, whatever that number is, for which we want to calculate the square.

Now, in this example, we have two endpoints: one for the `GET` and another for handling the `POST` request. The `/get_data` route responds to the `GET` request by returning a simple JSON message. The `/post_data` route handles the `POST` requests. It retrieves the JSON data sent in the request body using the `request.get_json()` function and returns a response that includes the received data. So, by specifying the `methods` parameter in the `@app.route` decorator, we define which HTTP methods each endpoint accepts.

Now, there are several ways to access and test your API endpoints. First method is the browser address bar. This works only for `GET` requests. Simply type the full URL of your endpoint and press Enter. The second method is using command-line tools. You can use `curl`, which is a command-line tool for transferring data, or you can use `wget`, which is another command-line tool. API testing tools like Postman can also be used. These provide a user-friendly interface for sending various types of HTTP requests and viewing the responses.

So, to test our endpoint, we can simply open a web browser and navigate to `http://localhost:5000` (which is the typical port accessed by Flask applications) and followed by a slash and then the name of the endpoint. The browser will display the JSON response from the server. Alternatively, we can use the `curl` command in the terminal to make the same `GET` request. However, since browsers don't natively allow us to send a `POST` request with a JSON body format from the web browser's address bar, we can use `curl` to send a `POST` request with a JSON payload, or you can also access the `POST` endpoints via Python code. So, this command must include the headers to specify the content type and then also uses the `-d` flag to include that to include the data.

Testing APIs using Postman. So, Postman is a very widely used tool for API development and testing. It provides an intuitive interface where you can configure requests with different methods, headers, and bodies. To test our API, we can create a new request in Postman. Specify the endpoint URL and then select the appropriate HTTP method. You can download Postman for free from this URL. To test our API in Postman, we can create a new request in Postman. Specify the endpoint URL here and select the appropriate HTTP method from this pull-down (GET, POST, and so on). For `POST` requests, we need to include a JSON payload in the body of the request. And then we finally click Send. Then Postman will display the response from the server, allowing us to verify that our API is working as expected. Uh, Postman makes it

very easy to test different scenarios and debug our APIs. Fast API is another modern web framework for building APIs with Python. It is designed to be fast and very easy to use. It leverages Python's type hints to provide automatic data validation, serialization, and interactive documentation with asynchronous support built-in. Fast API enables the development of high performance applications. Its focus on developer experience makes it a popular choice for creating robust and scalable APIs quickly.

So, comparison between flask and fast API. Both Flask and Fast API are powerful frameworks, but they have different strengths. Flask is simple and straightforward, making it a good choice for beginners and small projects. However, it may require additional plugins or manual coding for features like data validation and documentation. On the other hand, fast API provides a high performance and modern features like asynchronous request handling and automatic documentation generation. It uses Python type hints extensively which can enforce data validation and improve code quality. Finally, while fast API is slightly steeper in terms of learning curve due to its reliance on type hints, it can significantly speed up development of APIs that require robust features and scalability.

That's all in this video. See you in the next video where we'll deep dive into the hands-on of API creation and accessing these APIs using Flask framework.

So welcome back to this another coding tutorial on flask. We'll be creating and accessing endpoints in this tutorial video. So let us first start by creating a simple flask app. So in this case I'm creating a simple app which will greet the user hello world. Let's see the structure of the flask app. First I'm importing the flask class from the flask library. So by default if you are using Anaconda or even Google Collab Flask would come pre-installed but in case flask is not installed in your machine you should go ahead and do pip install flask before running these codes. Okay.

So since I already have flask we'll go ahead. So this is the flask app I'm creating and uh this is how we initialize the flask app. app equal to flask un_ame fine. This is the place where we are creating the endpoint. So this is my root endpoint. Uh against each endpoint we have to define a function. So basically it tells the user when the user accesses this endpoint what should be performed, what activity should be performed that is defined within this function. Okay. So this is the decorator which is decorating this function essentially. Okay. So whenever I'm going to access this endpoint, this is the function which will get triggered and this function is doing nothing but simply returning uh hello world back to the user. Okay.

So from this code uh I'm just going to run the flask app. So ideally this entire code should be run using a py file. This is not a good practice to run flask applications or endpoints from uh Jupiter notebook. Okay. But for the sake of simplicity and learning, we are doing it here. But ideally, this should be wrapped into a python.py file or app.py file. And that app.py file should be run using your terminal. Okay.

So, let me run this. It says that I'm serving this flask app uh with a debug mode off. And this is the URL where it is running. So, let me click on this URL. And there you go. You can see that the flask app has returned me hello world. Okay. So this is being served whenever I'm accessing that endpoint. This was very easy. You can also access this URL by using curl command. So I'll just copy this. So curl commands can be accessed from your command prompt. So I'm running my command prompt cmd. Here I can directly paste my curl command and enter. You can see the output here as hello world. Right? Okay.

So I'll have to stop this. And how do I stop this? There is a interrupt terminal or interrupt kernel button over there. So I'll click on this interrupt the kernel to interrupt the or to stop the flash server. Done.

Now let's move on to the next uh route. So here we are going to create an app routt which takes a user input and does some computation. So we'll have two app roots here. One is to greet the user with some name and this name will be provided by the user. Okay. So if I'm providing prashant, it will greet hello prashant in H1 heading. So this is the HTML code that I am wrapping it up with. Okay. And secondly, there is another endpoint created called as /square which expects some integer as a number. Okay. So when the user provides that number uh I'm going to square that number and wrap it up within this HTML string and display it back on the browser window. Let's see.

So this flask uh app is running. Okay. Now if I click on this, it says URL not found because there is nothing defined in the root URL or the root app. Okay. The root endpoint. Okay. The only endpoints defined in this flask app is greet and square. Okay. So I should be accessing the greet and the square endpoints only. So what I'm going to do is let me copy my greet URL. Okay. I'll just copy and paste it here with my name. So now you can see hello Prashant. So if I just change the name in the URL okay to ammy you can see the response comes as hello ammy okay so greet/greet is my endpoint and beyond that the user input that's exactly what this endpoint does it takes the user input as the name and displays hello name the other one is the square so you can access it here so I'll just copy this and uh replace it so I can provide some number here let's Okay 100. So this is the number which was provided and this is the square of that number. Okay.

If you want you can also access this URL on or by using curl command. So all you have to do is just type curl and then the endpoint URL along with the user input. This is very important. If you forget the user input, you'll get some error. Okay. So let's see what happens if I forget the user input. Okay. It says 404 not found. And this error is primarily because the user input was not provided. So I'm trying to access the endpoint without providing the user input. And this error message is also logged in here. Okay. 404. Okay. So 200 means the status okay means the request was successful and the response was generated successfully and served back to the user. So 200 is good. 404 is not good. Okay.

Now there might happen that you want to actually provide a name. So for example, let me go back. Uh my full name is Prashant Sahu. So if I try to give a space, okay, if you see here in the address bar, it automatically appends an um percentage sign. This is the URL encoding for the whites space which I provided. Okay. So percentage 20 that is the complete URL encoding. So if you want to use some special characters in the user input make sure that the URL is properly encoded. So you can actually check the uh W3 schools um URL for a complete reference on URL encoding. Also there is a link which will help you to generate the URL encodings freely online. So once I type my name let's say Prashant space Sahu you can see that automatically that space was encoded with percentage 20. So that should be the complete user input. So let's say tomorrow I want to go for another name um John Doe. So this is what I get automatically. So this is to be copied and now I can paste it where I want to access the URL. So greet and uh delete this. Paste it. Press enter. Super. So this makes the API calls very safe.

Also you can uh use error handling which I have done in other routes. So in the third example I'm actually going to do some error handling as just mentioned. Okay. So in this I'm going to actually use the gu get method explicitly. In fact, if you noticed behind the scenes in the first two you in the first two endpoints also we were able to access the endpoints using the get method. So get method is enabled by default even though even though you did not mention the list of methods here. Okay. In the next one I'm going to explicitly mention which methods are allowed for that endpoint. So here the get method is the only method which is allowed for this square root endpoint. So this square root endpoint is being programmed here to take two numbers as the input. I'm going to divide these two numbers and the resultant number I'm going to fade it to the square root function. So I'm going to calculate the square root of the division result. So two numbers are going to be taken as a input. These two numbers would be divided and the resultant division result will be used for square root. that square root will be result that square root will be returned back to the user that square root will be returned back to the user along with the division result. Okay. So there are two outputs this time. One is the division result and the other is the square root.

So if you look into the uh definition more closely I have enclosed the entire code within a try accept block. So these two numbers will be requested from the user num one and num two. And if either of the numbers are not provided then it will throw this error. If the second number is zero then we'll end up with a zero division error. Otherwise if everything is right we'll calculate the division result. And if one of the numbers is negative okay either the numerator or denominator. If one of them is negative then the square root operation cannot be uh done because square root of a negative number is complex. So that is a bit difficult to handle here. So I'm just throwing that error because one of the numbers is negative. And if all the conditions are satisfied, then this operation is conducted. Okay.

So let's execute this. Okay. Now my app is running. Oh, I think I forgot to stop my previous app. Okay, that is why I did not get the response here. Okay, I would have got response like serving flask app main. So let me stop it. Okay. Make sure that it is stopped and run this again. Okay. Yeah. So now it is serving. So I can go to the one of the URLs here. I have already given these URLs. You can access by directly clicking or you can copy paste. Okay. So first number is 16. The second number is four. So the division result is four and the square root of four is two. Okay. That is good.

Now here I'm getting an error because I have only provided number one and not the number two in the request. So the request is only giving the first number and not the second number. Since the second number is missing, I'm getting this error. Okay, the next request the second number is zero. If you see which is why the zero division error is coming up here and finally uh here the first number is negative which is why uh this last error is coming. We cannot calculate the square root of a negative number.

You can also access all these endpoints via curl. So let me show you one of them. Let's say you copy this from here to here and then go back to your command prompt and just paste the URL. Yeah. So here I get the response. So the entire response is in a HTML format as you see. Okay. Within the H1 heading. Perfect.

So in the next example I'm going to demonstrate the post method and some of its variants. Okay. So here I am enabling both the get and post method. So post method is typically used when the user wants to post a data to the server either as a HTML uh form or some JSON payload. Okay. Get method is typically used when the user wants to request some data from the server or downloading some data from the server. So in this case the form is expecting message field. Okay. By default if no message is given by the uh user it will take no message as the input. Fine. So here I'm creating a form which will be rendered. Okay. So this form uh action is sub uh here you'll you are creating a text field and then a submit button a simple form with a text field here and a submit button. Okay. So once the button is submitted then the request.post method gets accepted. Okay. And in this case uh this is what is returned. Okay. So the same message is returned along with this string.

So let's display this form. Okay. Again I have to stop the previous. Okay, now it's serving. So I can access this form here. So you can see on the top left I've got a small form and uh uh let me provide some message here. So hello world this is flask app and hit submit. So this is what you submitted to the server right. Perfect. So this works and you can also access via this curl. Okay. So I'm just stopping it.

Now what if you want to access the same post this form basically you want to access it via Python code. Okay. You want to submit a post request to the server using a Python code. Okay. You don't want to fill this form. Essentially you don't want to fill this form. Okay. You refresh this and it will again give you the form. So let's say uh you don't want to fill this form. So then this is the code that you will need. Okay. So what I'm going to do is copy this code. Run this flask cap. Okay. Run this flask cap. You cannot run this code in the same notebook because you cannot run multiple codes in the same notebook. So I'm going to create a temporary notebook here where I'll paste that code just to run it. Okay. So this code is going to access your flask app with the post method. So this is the URL where I'm going to place the request. Okay. So this is the URL and this is the message which I was supposed to type in the front end. Right? So this is let me just refresh. Okay. So yeah here. So whatever you want to type here you just want to submit that from the back end here. Okay. So the same message you are passing it to the server using this data dictionary. Okay. So you can use the post method pass the URL and also the data. Okay. And then collect the response from the server and extract the text part from the response. Let's see. Super. So this is the response that I've got from the server. And there are many more things in the response. Let's quickly have a look. The first one is headers. So you can check the headers. Okay. So the response header has the date and time of the request when it was given to the server. Okay. The content type, the character set, the content length 43 characters essentially and whether the connection is closed currently or not. And this is the work zug server. This is the flask server essentially. Okay. Otherwise the URL of the actual server would be displayed here once the application is hosted on a real server. Right? Uh you can also see response dot tab. You can see the other things like history and the JSON format of the response. Let's go for the status. Okay, this is something status code. So if everything is successful, they should return 200. So status code 200 means the request was successfully fulfilled by the server without any error. Okay. And there are many more things you can explore. Fine.

So coming back to the app, let me stop this. I have created a small variation of this flask app. So in this uh this flask endpoint is being modified to accept JSON data now. Okay. So I'll not be using the dictionary uh input. In fact, if you see here, the main difference is the get from form data. Instead of getting the form data as a string, it will get it in the JSON format. So request.get JSON. Okay. So this is the field it will be interested in to extract the message and from there the response is also JSONified. So here the response was directly given as a HTML format but here now I'm JSONifying the entire response. So the response will come in a JSON format. So this part of the code is same. Okay. Let me run this and again to access this I'm going to copy this code. So if you are accessing this from the front end, it will not be much different from here. Let's see. Okay. So the JSON data that I'm going to parse here is message. So this this is important because this field will be uh searched by the server. Okay. To process your request. Okay. So this is the URL which you are hitting and this JSON data is parsed to the JSON argument here. And now from the response.json JSON you are able to fetch your entire output. So response.json is what you are interested in right perfect.

So let me stop this and finally we are going to create a almost end to end application with a front end and a back end kind of. Okay. It's a pretty advanced app with HTML templating and uh supported by Ginga library and CSS rendering. Okay. So can be a bit tricky for a beginner to understand this entire codebase. I would strongly suggest you to copy this, paste it in Chad JPD and ask Chad JPD to explain you line by line that will be very easy. I'm assuming that you are fairly comfortable with Python coding and for that reason I'll just explain in high level what is happening. So this is very much the same code which I used here. So which I had used for the get method. Okay, just converted that into a post method now. Okay, just converted that into a post. So the first difference is here I have added a home route base route. Okay. So which you can access. This is important. Okay. Earlier we had not done that. So this is a root route. So where this HTML page will be rendered. So there are three HTML pages which will be used in this Flask app. So one is the homepage. Second is the form page. So this is this form uh that we were you know creating this form on the fly using a Python code. Instead of creating this form on the fly, I have already created this form and saved it separately as a HTML file. Right? So that form HTML has been separately created. So I'm not going to go ahead and open that HTML code and explain there. Please feel free yourself and uh let me know if you have any questions or doubts on that. you can take help of uh charg to explain the form uh HTML code because that is lot of HTML in that okay so these two things are clear one is the home HTML the form html will create a form where the user is expected to give two numbers as the input and finally if everything goes well the computation happens and the result is displayed in another HTML format okay so that result HTML is rendered along with your actual calculation result. If anything goes wrong here, it goes into the accept block and uh that is handled gracefully.

So let me execute this. Okay. So I'll just click here simply. It goes to the home.html page. So this is essentially home.html. Um I'm clicking on this to go to my calculator app. I'll provide two numbers here. So let's say 1 2 3 4 5 is the first number. Second number is let's say 25. So first thing that will be done is this number will be divided by this number and the resultant number is 493.8 and square root of that is 22.22. Right? So click on this and it goes back to the homepage. Right? So this entire thing once again let me show you the homepage was this the app root base. Click on this. It goes to which endpoint? It goes to the square root endpoint where the form.html is rendered. And once I have given some numbers, okay. And click on the submit button. This is my submit button. It goes to the results.html actually. So this is coming from the results HTML. Perfect.

And if you notice something, there is 304 304 which never came before in any of the previous runs. 304 never came here. So this is something new. So 304 is not a error here. It's actually telling you that I'm caching the CSS and the HTML uh files in the server and the cached version is actually rendered to the user. So 304 not modified is a HTTP status code indicating that the requested resource. What are resources are we requesting? Three HTML files and one CSS file cascading stylesheet CSS file we are accessing. So these are not modified since last time it was requested. So that caching helps the server to load these files very fast and also reduces the server overload and the data transfer is also reduced.

One important thing that you must note is the folder organization of these HTML files that you need to have in your local so that the HTML files are correctly rendered when you run the flask app. So this is the Jupyter notebook which runs the flask app. This could be a regular Python file which you are using to run your flask app. This is the one which will render your flask server. Within this folder itself, I have got two more folders. One is called as templates. So the templates folder has the HTML templates. Uh the form and the results of the form which will be displayed and the home. So these are all HTML files which we are using in our flask cap. There is one more folder called a static. This static folder contains the CSS files the cascading stylesheet files. Now this CSS file is attached or linked to the all the HTML files here. Okay. So when I open any of these HTML file you can see that this stylesheet has been you can see that this stylesheet has been this has been referenced within the static folder. So that is the reason why you need to place your CSS file within the static folder. The CSS file is the file which will style the front end of your web page. Okay. It contains the styling information. What color, what background, what how should the buttons look like? How should the text fields look like? What should be the size, font size, font face, what should be in a normal weight, bold, italic. So all the look and feel of the web pages of the forms and the results are coded within a CSS file. And this is the raw code of the HTML files that you can see.

Hello and welcome back. In this module, we'll do some hands-on using chat GPT, Google Germany and hugging face APIs. So, let's get started. So, in the first video, we'll use JPT and Google Germany with Python for doing some real world tasks using their APIs. So, first we need to install OpenAI. You can unccommen this line of code and execute. Similarly, we will need to install the Google's generative AI package.

Let's see how to create OpenAI keys which will be needed to access the chat GPT models. So, we will navigate to OpenAI login page. If you have never created an account, please create an account. If you already have an account, we'll go ahead and do the login. You might need to enter the verification code sent to your in inbox. Upon authentication, it will take you to the open AI platform page. You can see the different projects that you have created. So if you have never created a project, let it be the default project right now. So if you see on this top right side you have the API reference. So this will give you a quick idea about how the OpenAI API works. First about the authentication how we can make the requests what kind of endpoints are present for audio chat embeddings finetunings batch uh requests file upload and so on. So all these are present in this API reference page. So please feel free to explore this documentation. Then we can navigate to the dashboard. So from the dashboard on the left side you can see your API keys. So if you have never created an API key this field will appear empty. So I'll go ahead and create a new secret key by clicking button on the top right. If you want you can provide some name to this key. Let's call it as AI agents. I'll create the secret key. Please make sure you copy this and uh you can open a notepad and keep it safely. We'll be needing this API key in the Jupyter notebooks. So save this file and then we are done. So let's get back.

So execute this cell. It will ask you for the API key. So the API key we just created and copied. I'll just paste it here and press enter. We'll add the open API key to the environment variable here. In a similar way, we'll have to create the Germany API credentials. So there is a link here. You can get the API key by clicking here. So you will have to first login into your Google account. And from there on the top left you can see the get API key button. Here also you can see the get API key button. You can click on this. uh if you have never created a project you will have to create a project first or else you can select one of the existing projects and then click API key in this existing project. So similarly uh you can copy this key and uh save it. So once we have saved the Google Germany API key, we'll have to enter it here and then press enter. Let's execute this. So I have set my Google API key in the environment variable and we are good to go.

So here I'm defining a function which will take the user prompt and the model. I'll be experimenting with GPT4 model and Germany pro models. Jiny pro model is from Google and GPT40 model is from OpenAI. So if the model is GPT40 uh this is how we would be making the API call to the chat GPT API and this is how we are going to make the API call to the Google Germany API. Okay. So we need to define the uh role and the prompt. So prompt will be taken as the input from the user and the actual function is chat.comp completions.create. So we are going to pass the model and the messages as a list. Here we can also set up the temperature. So higher the temperature, more is the degree of randomness in the model's response. And finally once we are passing all these three parameters to the uh chat completion API, we'll get the response and from the response I'm going to extract the message and the content of that message. So this function will return the final response of the message with the actual content.

Now if the model name is Germany pro uh we are going to use the generative model class uh and initialize and finally pass the user prompt to that generate content built-in function of that model and fetch the response from the uh model and response. We have the text field. So I'm going to extract that text from the response and return in this function call. If you are mentioning any other model name uh different from GPT4 or geminy pro then then this function will return this error message basically saying that the LLM is not configured and the reason for this is each model from a different vendor will have a different way of making an API call. So there is no generic method to make API calls to all possible models, right? So we need to be sure which model we are accessing.

So we'll take a couple of exercises in this notebook. So in the first exercise, we'll generate some text with both charg uh Google get completion function we created above. This is the user prompt that I'm passing and by default the model is configured to use GPT4 anyway. So I don't need to do anything more. I'll just execute this. It might take few seconds to get the response from the server based on your internet speed and the load on the server. So write a short story about a college student. I have got story about a student named Lena and uh about her uh college days. Okay. And a similar response I'm getting from Germany pro model. Okay. So here the user prompt is exactly the same. Uh the model name has been changed to Germany pro. Let's see what story Google Germany makes. So the persona identified by Google Germany is about I think Emily Carter here. So I think the story is about Emily Carter in this case and you can take your time reading this story.

Now let's try out uh zeroshot prompting basically we'll ask some questions to the model and we will expect the replies. So this function display and markdown is being imported to make a prettier display of the model's response. So again using the get completion function I'm passing the prompt as explain generative AI in two bullet points very short and crisp. So let's fetch the response and display that response in a nice manner. Okay. So these are the two bullet points I have received from the GPT4 model. Generative VI refers to algorithms based on machine learning models like neural networks that can create new content such as text, images, music or code and so on and so on. Let's see what is the response from Germany model. I've changed the model name here. So Germany model is giving me even more crisp response. Genative vi is a type of AI that can create new data or content from scratch just such as generating images, text or music. Super.

So we'll take more examples of uh prompting or basically using the uh APIs from different different models in the next notebook. Okay.

Hi and welcome back. So in this hands-on exercise, we will do some prompt engineering with open-source large language models. Uh you using hugging face serverless APIs. Basically hugging phase is a repository of various large language models. We can either download these models from hugging face repository or we can directly call these models which are hosted on the hugging face servers. So in this notebook we will only use the inference APIs for these models which are already hosted and running on the hugging face server. In the next notebook we will actually download these models using the hugging face repository and then use those models locally. Right. So in this notebook we'll learn how to run any open source model via hugging phase inference API and you can run this entire notebook on Google collab as well. Right now I'm running it on my local machine. So hugging phase has made something called as inference API free to use with some basic rate limits. So under the fair usage policy if you are not exceeding the rate limits uh you can directly access these models hosted on the hugging face server and directly get the response. Okay. So don't try to access this API just too often and reach the rate limits otherwise you will end up making uh you know blocked out by the uh hugging face. Okay.

So we are going to experiment two models in this uh notebook. One is the Mistl 7B instruct model which is a 7 billion parameter model and this transformer model has been built by the French company called as Mistral AI and it's a instruct fine-tune model. Uh and you can just follow this link. This is the base model which has been used to instruct fine-tune. So this is what we call as the model card. So if you have never created an account on hugging face, you will have to first create an account. And I have already logged in. You can see I've already logged in into my hugging face account. Okay. And the second model we are going to use is the Gemma 2B IT model. It is for instruction tuned again. So this is again a open-source model by Google. It's a two billion parameter model and it is fine-tuned for instruction following tasks and that enables us to handle a different wide variety of complex natural language processing tasks. Okay. So of course you will need internet connection because we are going to access the API calls from hugging face servers. You will also need the hugging face account and the API key.

So first let's see how we can create the hugging face key. So you can go to this link which will take you to the account page from where you can get the access tokens. Okay. On the left side you can see the access tokens. So I already have created a couple of keys but let me create uh a new key. So click on this button and there will be three options fine grain, read and write. For most of the task you will be doing uh on the hugging face platform you will be needing the read token type. This way you will be able to make API calls to all the publicly available models on hugging face. all the models which are available for inference APIs using the right you will be able to actually upload your own model to the hugging phase repository if you want to do that and in the fine grain you can provide fine grained access to your repo for read and write access okay so we'll go with the read only token type which is the most common token type you are going to use in this course so I can provide some name let's say AI agents Okay. And then I'll provide so this is the token and uh this is the token name you can say and this has a readonly access to all your organization resources and we'll be able to make inference calls on your behalf and that's exactly what we are looking for. So I'll go ahead and create token. I'll be presented with some key here. So don't forget to copy this and save it exactly the way we saved the openi access key and then we'll come back to the notebook.

So here let's execute this. You will be asked to paste that key and press enter. So we'll be using the request library to make uh inference API calls to the hugging face serverless APIs. So first we'll need to create headers just as we did in the last module and then we are going to create a payload and the model API URLs as the inputs to the query function. So this uh userdefined query function will take the payload and the model API which we want to hit. Okay. So we are making a post request to this model API URL. So different models will have different API URLs and we'll also pass the uh headers which will have my authentication uh token and we'll define a JSON payload. Finally, we'll get the response from the server after the post request is made and from there we'll access the JSON part of the response. So let me create this function.

So there are a lot of settings uh you can find the detailed documentation by clicking on this link. But uh the most common configurations that we are going to need and you should be knowing is uh we can restrict the model to create some specific token limits. So if you give some max new tokens value here let's say 200 or 500. So the model will create maximum that many tokens. Okay, so this is the number of tokens you want your model to create in the response. Temperature again the value typically lies between 0 to one. So value closer to zero means more deterministic response. The same response will be obtained every time you hit the model API. So there will be less creativity and the model's response would be more realistic, lesser hallucinations. However, if the temperature value is set to be higher, closer to one, then the model's response would be more creative and you are allowing for more variation in the responses. And uh here I am creating the API URL. So this API URL you can get it from the page. So so this is the model card page. This is what we call as the model card page. So there is a button on the right side which says use this model. So by using the transformer API. So once you click on that it will give you the a short code which you can copy and start using. So here you can see this is the model link. However for now we'll directly use the model API URL. Okay. So let's execute this. Similarly this is the API URL for the GMA 2B it model that we are going to use. So there are some standard parameters. For example, wait for the model's response. Yes. and um we don't want to return the full text. The maximum tokens we want to return is only 1,000 and so on.

So let's start uh prompting our models. Uh first we'll take a basic Q&A uh question and answering. This is zero short prompting to the model. So here the prompt that I am passing to the model is can you explain what is quantum computing to a fifth grader. So this is the complete prompt. Uh let me print the prompt and this is the model's API URL. Okay. And uh when you run this code you'll get the response from the model. So here I'm passing the parameters for the Mistl model and the model API URL I'm using is for the Mistl. So basically this response was obtained from Mistl model. Now there might be a possibility that you may get an error. So if you are getting an error, most likely reason is you will have to go ahead and accept the uh license. So click on this link. This will take you to the model card page. And uh you might get a window here to accept. So since I have already accepted, I have been granted access to this model. So this is a gated model. So you will have to accept the license agreement for this model. Yeah. So for example if I click on Gemma 2B it okay so this is the kind of window you will get even for the Mistl model the first time you try to access it. So the moment I acknowledge the license I will be granted access to this model. So let me authorize this model. So this is a one-time activity. So this model will be now accessible to your account. So again the message says you have been granted access to this model. So this is one time activity you might have to do for few models which are gated models not very freely publicly accessible. So coming back now that we have been granted access to the model we can go ahead and start querying this model. So I have got the model's response from Mistl. Now let's see the similar response from the Gemma model. So this is the response code. 200 means successfully retrived the model's response. So this is no error. 200 is success message basically. So the prompt was again the same which is can you explain what is quantum computing to a fifth grader. So this time GMA model has replied that imagine you have a coin regular coin has got two sides head and tail but this is a special type of a coin which is called as a quantum coin and this can have both heads and tails at the same time I mean so on right so if you want to pify your response you can use the uh display and markdown functions from IPython display so this is slightly better okay let me take another example here I've got some uh big paragraph u you can say a report in this case I'm carving a prompt template in this case so the prompt template says summarize the following report delimited by triple back text on generative vi in maximum five lines so basically uh I'm passing this report as a placeholder so this report will go along with this prompt this report will go along with this prompt okay and I am asking the generative AI model to summarize in maximum five lines. Okay. So let's first print the prompt and you can see the prompt the report went along with the instruction. Okay. So this is the complete prompt that you are passing to the model. Now again the same function I'm using. So the payload here is the same prompt which I created this entire thing and I'm going to use the mist model this time and I'm retrieving the generated text from that mistral model response and now going to display that. Okay. So my M myst model says that genetic VI is a technology that can so and so. So this is the summarized version of this entire text. Okay. And if you want you can count this should not be more than five lines. Similarly for the Gemma model the only difference is the Gemma tob it API URL is being used instead of the Mistl model. Let's see the response here. I've got only a single line response in this case. Okay. Not expected. This was supposed to be five lines at least like this. Anyway, let's take another example. I've got a review here uh about So here the review says, I recently worked with real estate company to purchase my first home and the experience was outstanding. So the agent was knowledgeable, patient, incredibly responsive and so on. They guided me through every step of the process picking what could have been a stressful experience. So basically this is a review about a real estate company. Fine. So act as a customer review analyst and given the following customer review text do the following tasks. So first you'll have to first I'm asking the model to find the sentiment either positive, negative or neutral. And then I'm asking the model to extract five key topics or phrases from this review. Okay. and then passing that entire review text as a placeholder just the way we did in the above case again using the same query function this time I'm using the Mistl model first let me fetch the response from the Mistl model and see okay so the overall sentiment uh the model has figured out to be positive and these are five key topics or phrases this review is talking about so outstanding experience knowledgeable agent patient smooth and enjoyable process process and guidance through every step. Super. Let's see what the Gemma model says. Again, the only downside was the high price of the property. Uh the overall I would rate my experience with this four out of five stars. Yeah. Okay. The overall sentiment is positive. That's the thing. And uh the customer is expressing satisfaction it seems. And these are the five key topics. So this is a completion. And if you notice this is the completion that I'm getting from the Gemma model. Okay. So JMA model thought that the previous response is not uh complete. So it first completed that response not the response but the review. So the review first gets completed although this was not expected but yeah so the review gets completed and then further the sentiment and the key topics were extract. Okay perfect.

Hi, welcome back. So in this notebook we will see how we can download the opensource large language models using the hugging face repository. So I strongly suggest you run this model on a machine which has access to GPU. If your local laptop does not have access to GPU, you can upload this notebook to Google's collab and we can use the free GPU provided by Google Collab. Okay. So this is the model that we'll be using once again. So how do you open this notebook on Google Collab? So let's go to Google Collab first. So collab.ressearch.google.com. You might need to sign in into your Google account first. So I have already signed in. And from here you can upload this notebook. So I already have the notebook in my local machine. So this is the third notebook which I'm uploading to Google Collab. So I want to run that notebook on Google Collab. Okay. So this is the model. So here we have successfully uploaded our Jupyter notebook to Google Collab. We are again going to use the same Gemma 2B in uh 2 billion parameter model which is instruction fine-tuned on Google's Gemma 2 billion LLM model. This is the base model and this is the instruction tune model. The recurrent gemma is a family of open language models which is built on top of the recurrent architecture which has been developed by Google and this particular model has been pre-trained and also later on instruction tuned to follow natural language instructions in English.

First things first on the top right side you will have to first select a kernel. So I strongly suggest you to select the uh kernel which is GPU one. So connect to a hosted runtime. In this case it is a T4. So you can see all the runtime types available by clicking on the change runtime type. Um so you can take the hardware accelerator to be T4 GPU. So, make sure you are using the T4 GPU and click save. Once it is saved, click on the connect button and wait for few seconds until your collab notebook is connecting to the Google's GPU server. So, a server has been allocated and now I have been allocated the GPU. So, you can see roughly 13 GB of RAM is available to us and around 112 GB of disk space. So we need at least 5GB of GPU memory. So that should be sufficient to run the inference with recurrent Gemma. Okay. So this model which will be downloaded, this will occupy roughly 5GB of GPU memory. Okay. So let's go ahead. So

If you're running this on Google Colab or any server or your local machine where you have GPU, then only you run this code. Otherwise, this will throw you an error. So, since I have already connected to GPU, I can run this code.

So, what this does is it will check the GPU memory available. How much GPU memory is available currently. So, it says currently there are no processes on my GPU memory. So, utilization is also 0%. If you see, currently what I have is the Nvidia version number, the driver version installed within the GPU. The GPU will support CUDA language. So, this is the version of CUDA available on my GPU. Some name which has been provided to the GPU persistence and this and that. Okay.

So, main thing is the memory usage here. You can see this is almost zero MB as of now because we haven't started using this. Uh, you might need to install the transformers library. So, let's go ahead and do that. Using the, see, transformers library has been created by Hugging Face. So, if you want to use the Hugging Face, you'll have to install transformers library. Fine. So, I've installed it now and let's go ahead. You might need to restart the runtime to make sure that all the installed libraries are ready. Okay, so let's do that. Click on the runtime and restart session. Okay, so click yes. Wait for a few seconds. I think it is already done. So, we can again check the Nvidia SMI. Okay, this looks good.

Now we can again start by importing the Hugging Face API key. So, when I run this code, it will ask me to log in. So, I am registering this Jupyter notebook on Hugging Face. It is asking me to provide the token. So, you have to paste the Hugging Face token that we created some time back. So, you can uncheck this and click login. So, login is successful. So, I now have access to Hugging Face from this Jupyter notebook running on Colab.

So, this is the model ID. You can check the model ID card by clicking on this link. So, this is the model card. Okay. And how to use this model? Instructions are given on the top right using the transformers library. So, some basic code is already given as a starter code. Okay. So, how to use this as a pipeline and how do you call it? So, everything is there as a starter code. So, we'll go back to our Colab and let's run this.

So, you can see that this entire model is now being downloaded into Google Colab. So, the process is still running. You can see when the model is being downloaded, it is downloading lot of other stuff like tokenizers, configuration. This is the tokenizer model. Now, this tokenizer model is 4.2 MB. The tokenizer.json file. So, this contains all the tokens information. Some special token information if you have in this JSON file and then other configurations are there. So, the main thing is this model itself. Can you see the safe tensors corresponds to the weights of this model. So, essentially what you are downloading is the model weights which is corresponding to roughly 4.97 GB. Okay. So, my model has been downloaded to Google Colab server. You can see almost 5.5 GB of space is being consumed already.

So, let's try out some basic prompting. So, first I have to create a message. So, setting the user as a role and content to be the actual prompt. So, I am going to prompt my model to explain what is AI in three bullet points. So, that is the chat which I'm passing to the apply chat template function from the tokenizer and uh, this will tokenize and print the prompt. Okay. So, basically what I'm doing is uh, how this prompt is interpreted by the model. Okay. So, BOS is basically the beginning of sentence. And these are special tags, you can say, which are passed to the model so that the model knows what is the system message and what is the user's actual prompt. Okay. So, these are all like system message, you can say. These are tags, special tags for that. And within the user, this is where the actual prompt lies, right?

So, let's see the model.device. This should give me CUDA index zero. So, this is CUDA is referring to the GPU. So, right now my model has been mounted on the GPU. Uh, let's encode this prompt. So, when the prompt gets encoded, basically it will give the token IDs and those token IDs are used as input by the model to generate the outputs, right? So, so the input and outputs would be token IDs essentially. Okay. Now, I'm using the output token IDs to decode the actual English words from these token IDs. Okay. So, token IDs would be numbers essentially. I am using the decode function to convert it back into an English language, you can say, right? So, this is the decoded output from the tokenizer. Okay. So, this is the final output, you can say, right? So, artificial intelligence is the ability of and so on.

So, if this looks a bit complicated and overwhelming, that's the reason Hugging Face has provided a pipeline which you can use much easily. Okay. And that pipeline is what you can also see from the model card page. So, I'll give you some examples of how we can use that pipeline. So, pipeline makes it easy. We don't have to go for the tokenizer input, output, decode functions and so on. In fact, there are auto tokenizers available for each model which will handle these tasks easily implicitly. Okay. So, again, these parameters we have already discussed in the previous video. So, temperature controls the creativity of the model. Value closer to zero means more deterministic, less creative. Value closer to one means you are allowing the model to be more creative, which means more variation in the outputs are expected when you rerun the same code many times. Right?

So, here I am using model.generate, generate function to actually generate the response and fetch the output. But again, the outputs are in the format of token IDs. Using those token IDs, I am decoding and generating the English text. So, that's the same thing. Okay. So, pipelines make it easier to send the prompts. So, you don't need to encode and decode every time. Uh, that's what the main value proposition here is. So, transformer.pipeline, pipeline here, you are using the text generation endpoint, you can say. So, each model will have different endpoints for different, different tasks. So, I'm using the text generation endpoint. I'm passing the model URL or the inference API URL and the tokenizer information. So, here the tokenizer is what we imported. So, all this tokenizer information we are passing here and just mapping it to CUDA. So, basically this model inference task will be run on CUDA. Okay.

So, I've created this pipeline. I'm just printing my prompt once again, which will be this. And then let's pass this prompt to the pipeline and directly generate the response. Okay. I'm restricting the maximum tokens to 150 and I'm allowing a bit balanced response. So, here is the response I receive. So, this is the complete response. Okay. And from here, I have to extract the generated text part. Okay. So, let me print the extracted generated text. So, this is how it comes up. If you want a more, more prettier response output, you can again import the display and markdown functions from the IPython. Use that markdown function to print this in a markdown format. Okay. Yeah. So, that looks much better.

Now, let's see how much GPU has already been used. So, if you see, this is 5.4 GB of memory has already been used. So, it uses more than 5 GB of memory. That's what exactly we mentioned at the start of the notebook. Okay. So, you can also print the pipeline. I'll take a few more examples here. So, in this example, we are going to do again zero-shot question answering from the model. So, we'll pass a prompt to a locally loaded LLM and ask it to perform some of the NLP tasks using prompting. So, the prompt text will be the user's input. Rest is default, system default. Um, fine. So, here the prompt text from the user is, "Can you explain what is mortgage?" So, this goes as the input. So, this input will be mapped here in the content part and this entire payload will be given to the apply chat template function from the tokenizer. Okay. And let's see this prompt. So, this is the complete prompt that I get from the tokenizer. And the tokenized prompt is now given to the pipeline for generating the output. So, the generated output text is what I will be printing now. Okay.

So, this is the complete response that I have received. Lot of information is there. So, can we display it in a proper format? So, I'm again using the markdown function to display it in a markdown format. You can see now it is very well formatted and much easily readable. I have another example here where I am going to share a detailed report on generative AI and I'm going to ask the model to summarize it. Okay. So, this is a kind of a prompt template I'm creating. So, in that prompt template, I'll be attaching this report. This is the entire report which I'll attach along with this prompt template. Okay. So, create jamma prompt. There's a complete prompt text and that prompt text is nothing but this entire prompt template which I'll be passing it to the pipeline. So, let's see this time. So, the maximum tokens is 500. So, I expect that the summary is less than or equal to 500 tokens. So, let me print it. Okay, perfect. So, I have exactly got five lines. That is what exactly I had asked. So, give me a summary in exactly five lines. So, I have got the five line summary of this entire text. Perfect. Let me format it if you want. So, these dashes become bullet points here.

One last example. We'll take it for a basic sentiment analysis. So, here I've got some review and I'll attach that review on this prompt template. So, what does this prompt template do? It is asking the model to act as a customer review analyst and asking the model to do two tasks. One is to find the sentiment for this entire review and second to extract five key topics. That entire prompt template is being passed as a prompt now. So, this will give me a tokenized prompt and that tokenized prompt is going to the pipeline and finally the output is generated. Fine. So, let me display that response. So, overall sentiment looks to be positive. Let's read the review. "I recently worked with this real estate company and so on. The experience was outstanding, knowledgeable, patient, guided me through every step." Yeah. So, definitely this looks quite positive response. The reviewer has expressed satisfaction and these are the key phrases. So, can I prettify it and display? Perfect. So, this is a prettyified display.

So, that's all in this coding hands-on tutorial on using the large language models on your local machine by downloading it from the Hugging Face repository. Hi and welcome back. As AI has evolved, so have the frameworks and tools designed to help us build more advanced and autonomous systems. Now, instead of starting from scratch, we have powerful tools that make it easier to create, manage, and deploy AI agents. I'll introduce you to some of these tools in this video. These tools will be used more comprehensively later in the further courses of this program. In this session, we'll explore four key tools in AI development: Langchain, Langgraph, CrewAI, and Autogen. We'll talk about what each tool does best, some practical use cases, and a comparison to help you decide which one might fit your needs. Let's get started.

But first, why do we need specialized tools to build AI agents? Well, let's consider some of the challenges. First, building these agents is complex. Second, integration can be a real challenge. Each agent might need to pull data from multiple sources, talk to different systems, and work well with other models. And third, scalability. As your agent grows in capabilities, it needs to handle more tasks, more users, and often needs to perform consistently as it scales. So, what are the solutions? We are talking about AI that can understand language, make decisions, and even take action. So, it's a lot to handle, especially when you want to develop something quickly and make it flexible. These tools make the process smoother and help integrate everything into a single workflow. Tools like Langchain and Autogen simplify all these tasks, reducing development time and complexity. Let's see what each of these tools bring to the table.

Let's start with Langchain. Langchain is an open-source framework that's become a go-to for developers using large language models like OpenAI's ChatGPT. It's designed to simplify the entire process of building applications with these models. What makes Langchain unique? For starters, it has extensive integrations. This tool works with everything from cloud storage providers like Amazon and Google to APIs and web scraping tools. So, it can gather information from almost anywhere. Another standout feature is its document handling. Langchain can process over 50 different document types, which is super helpful for applications that need to handle a lot of diverse data. Langchain has impressive tools for coding and debugging, making it easier for developers to find and fix issues as they arise. Plus, it includes built-in memory functions, whether it's short-term memory or long-term memory. Langchain lets agents remember important details across interactions, helping them stay relevant and contextually aware.

Let's look at some of the specific ways you might use Langchain. Langchain really shines when it comes to a few specific applications. Chatbots are one of the most popular. Langchain helps developers create chatbots that can answer questions, engage in conversations, and retrieve information from multiple sources, making for a more dynamic and responsive experience. React agents are a type of intelligent system designed to combine reasoning and action, allowing them to adapt and respond dynamically to complex environments. By integrating planning with real-time feedback, React agents can solve problems efficiently and improve decision-making processes in uncertain contexts. For those working in software, Langchain can also help with code analysis. Developers can use it to generate, analyze, and even debug code. It's like having an AI assistant that understands code and can make suggestions.

Next up is Langgraph. Another tool that's a bit different from Langchain. Langgraph is all about workflow management. It lets you design workflows using something called as directed acyclic graphs or DAGs, where each part of the workflow is represented by a node. What's really cool about Langgraph is its advanced memory capabilities. Not only can it remember important details, but it's also capable of handling errors and even allowing human intervention if needed. This makes it robust and ideal for more complex projects. Another unique feature is that Langgraph integrates seamlessly with Langchain. So, you get the best of both worlds: that is Langchain's resources and Langgraph's workflow structure. Lastly, Langgraph has caching. So, you can save the progress of a workflow and return to it later, which is incredibly useful for complex applications. So, where would you use Langgraph? It's perfect for managing complex workflows with multiple steps that need to interact with each other. Imagine a data processing pipeline where one step relies on the output of the other. Langgraph can handle all that while keeping everything organized. Agentic RAGs combine generative models with reasoning capabilities, allowing agents to autonomously generate responses based on both learned knowledge and logical inference. So, in multi-agent systems, multiple such agents collaborate or compete, sharing information, coordinating actions, and solving complex tasks through collective reasoning and communication, enhancing scalability and adaptability in dynamic environments. All this can be very easily and efficiently handled by Langgraph.

Moving on to CrewAI. CrewAI has a unique approach by organizing AI agents around specific roles. This means each agent is assigned to a particular role or function, making it easy to design systems where multiple agents work together. One of the CrewAI's unique features is dynamic task allocation. Agents can autonomously assign tasks to each other based on need, which is perfect for projects that require real-time decision-making. CrewAI is also built on top of Langchain, which gives it access to all of Langchain's integrations and tools. And if you are a developer who likes monitoring and tracking, CrewAI provides detailed insights into each agent's performance and task progress. CrewAI is excellent in creating multi-agent research teams. You can build agents that collaborate to analyze data, conduct experiments, and report findings. Basically automating much of all the research processes. In customer support, CrewAI's agents can interact with customers, answer questions, and even escalate issues to human agents if needed. This frees up human resources for more complex cases. It's also an ideal solution for education. Imagine personalized learning agents that adapt to a student's pace and provide tailored feedback. Lastly, CrewAI can help in software development by assisting in coding tasks, debugging, and even testing applications.

Now, let's talk about Autogen, Microsoft's conversational agent tool. Autogen stands out because it treats workflows as conversations. This makes it very intuitive, especially for anyone familiar with chatbots or other interactive systems. Autogen's modular design means you can easily extend its capabilities by adding new components and workflows. You can even execute code with these workflows, adding a lot of versatility. It also supports cross-language operations and asynchronous messaging, which is ideal if you are building a distributed system with agents that need to operate independently and communicate as needed. It also supports cross-language operations and asynchronous messaging, which is ideal if you are building a distributed system with agents that need to operate independently and communicate as needed. Autogen has advanced capabilities too. It supports containerized code execution, meaning agents can write and run code in isolated environments. This is great for security and flexibility. It also works with multiple programming languages, currently supporting Python and .NET with more languages in the pipeline. And if you're working on a large project, asynchronous messaging lets agents communicate without waiting, which is ideal for scalable distributed systems.

To wrap things up, Langchain, Langgraph, CrewAI, and Autogen each offer different features and capabilities that fit different types of projects. Thank you for joining, and I hope this session has given you a clear understanding of what's possible with these tools.