Transcription
If you've read a fair amount of Python code, then you've probably seen this `__init__.py` file pop up quite a few times. It's especially common in larger Python projects. Well, in today's video, I'm going to break down exactly what this file is, when you should use it, and exactly how it works, so stay tuned.
So, first things first, I've actually just removed that file and I'm going to explain to you something that's a prerequisite, which is known as Python modules. Now, a Python module is really just some Python code or Python file that contains code that you'll import from another module. Now, technically, any Python file is a module, but we refer to something as a module usually when we're importing it from another script. So, for example, I have my `string_util` module, I have my `math_util` module, and we can imagine that this contains some code that we might want to use in a larger program. So, maybe what I have is something like a `main.py` file, and from `main.py`, I want to import some of this code just to make my, uh, file a little bit cleaner, right? So I can say something like `from string_util import capitalize` and `from math_util import add`, and now I have access to these functions from my utilities and I can use them directly in my file. If I run the code, you see that I don't get any errors, and this is perfectly fine.
So, this is quite common and it's a great way to organize your scripts to make things easier to read and understand, where you don't have all of your logic in a single file. And you can imagine as the project is larger and larger, you're going to have more and more modules, and what's going to come is the need for a package. Now, a package is simply a directory that contains Python modules. So I can do something like `utils` here, and then I'm going to take my two utilities and put them inside of this newly created package. Now, again, a package is just a directory that contains Python scripts or Python modules, and previously, in older versions of Python, you needed to create this `__init__.py` file in order to mark that this directory was a package. But in newer versions of Python, like all of the modern ones you're using today, that's no longer necessary, and any directory that contains Python scripts is able to be considered a package.
Okay, so all of that's great, but now if we go to `main.py`, notice that our imports are no longer working. It's saying we can't resolve `string_util` and `math_util`, and that's because they're inside of this package. So now, if I wanted to import these various functions, I would need to change this to reference the package. So I would have to say `from utils.string_util import capitalize` and `from utils.math_util import add`, and all of that's fine. And if I run the code now, you can see that all of this works. And by the way, if you're wondering why these `__pycache__` things keep popping up, anytime you import a module from a script in Python, it will automatically be cached inside of `__pycache__`, so you don't need to keep recompiling all of these module scripts, especially because a lot of times they're not changing. Don't worry too much about that, but just a fun fact in case you were interested in that.
Okay, so now that we've got all of this, it's time to finally talk about `__init__.py`. So I'm going to go ahead and create this `__init__.py` file, which is used inside of a Python package. Now, even though it's not necessary, it's still often used because of its role. Now, what the `__init__.py` file does is simply initialize the package by running some code whenever it's imported. Super quick pause here, I want to share with you something that's super valuable and completely free, and that is my coding newsletter. In the coding newsletter, I send you coding challenges, obviously the solution to those challenges, cool project ideas, stories, insights from me as a software engineer with over 10 years of experience, and as a lead magnet for this, I'll even send you an entire 14-page PDF breaking down all of the ways that you can make money from coding, which you can see right here. That's completely free. You can sign up from the link in the description or techwith.net/newsletter. And quickly, if any of you guys are interested in some private mentorship from me, I do have a program. It's called Dev Launch. I've just opened a few more slots. If you want to access, you can apply and see if you're a good fit from the link in the description. Just ask you to fill out a short form just to make sure that I can actually help you, and there's a video here that shares some great insights. Anyways, thanks for listening. Now let's go back to the video.
So, to see the simplest example of this, I'm just going to make a print and I'm going to say `print("Imported utils")` like that, and I'm going to go back to my `main.py` file and show you what happens in the terminal when I run this. Notice when I do that, that it says "Imported utils", and notice that it only says that one time. So again, what the `__init__.py` file does is it will run some code to initialize the package the first time this package is imported. So here, it's imported twice, right? Because we're importing `string_util` and `math_util`, both from the `utils` package. So the first time that we import this package and we initialize it, we run this `__init__.py` file. That's the use case.
So if you were importing a larger package, so imagine importing something like TensorFlow, you would probably have some code that you'd want to run before that import is successful, and that's why sometimes when you import code, it takes a second before it actually loads in your script. Well, there's other reasons as well, because usually it's running some kind of setup operation. So you can imagine that if you're writing a Python package, you would utilize this file to run some kind of setup or config before the package is fully ready or loaded, and then it can be used by your Python script.
Awesome. Now, as well as just doing something like, you know, printing some message or initializing something, you can simplify the imports of different components or dependencies in your package with this `__init__.py` file. So right now, if we look at `main.py`, the imports can be a little bit annoying and kind of weird to have to write out, and you'd have to know to use the `string_util` and `math_util` library or module, sorry, in order to find `capitalize` and `add`. If you just wanted these functions to be available by default, then inside of your `__init__.py` file, you can actually write them out. So I can say `from .math_util import add` and I could say `from .string_util import capitalize`, and then from my `main.py` file, I would be able to just directly import these from `utils`. So I could say `from utils import add, capitalize`. That's because they're defined directly in the `__init__.py`, and anything that's defined in the `__init__.py`, even something like a variable, for example, like `c`, can be directly imported when you import the package. So here I can now import `add`, `capitalize`, and `c`.
So let's run the code and I'm going to show you actually an error that pops up and how we can fix it. So notice that when I run this, we actually get an error, and this is very common. I cannot tell you how many times this has happened to me and people that I know. It says `ModuleNotFoundError: No module named 'math_util'`. Now, that's a little bit weird. Why didn't that work previously when I wrote that import inside of `main.py`? Everything was working fine, but now when it's in `__init__.py`, it's not working. Well, the reason it's not working is because we've run this package from outside of the package. We have this `main.py` file, this is the entry point to our script. Now, when this entry point runs `__init__.py`, by default, Python's going to be looking in the directory where our entry point file was for these modules, `math_util` and `string_util`. So it's going to be looking outside of the `utils` package. We need to tell Python to look inside of `utils` to find this code, and in order to do that, we need to use a relative import. We can also use an absolute import, but typically use a relative import.
Now, a relative import is simply this single dot, or actually, it can be multiple dots, but the single dot references that you're looking in the current package. So if I say `from .math_util import add`, that means from the `utils.math_util` module, because we can have modules that have the same name that live in different directories or packages, right? Same thing with `from .string_util import capitalize` means look in the current package, which in this case is `utils`, for this `string_util` module. So now, if I go ahead and run my code, you'll see that this works fine and it prints out "Imported utils".
However, I'm going to throw a spin on this again. What if I now go into `__init__.py` and I directly execute the `__init__.py` file? Well, when I run this, notice now that we get an error, and it says `ImportError: attempted relative import with no known parent package`. The reason we now get this error is because we're trying to import `math_util` and `string_util`, but we're not currently inside of a package. The reason we're not currently inside of a package is because we've run `__init__.py` directly, and it just sees that it has these Python scripts beside it. It doesn't see any directories that it's currently inside of, and it doesn't know that it currently belongs to a package. It would only be able to reference packages if we had a directory inside of this directory or alongside the `__init__.py`. So this is why things get a little bit confusing depending on where you run the code, the import is going to work differently.
Okay, so I'm hoping this is making a little bit of sense. When I run `main.py`, it imports this package. So then when we run the imports here, we have to use relative imports because they're inside of the package. Whereas when we run `__init__.py`, which is not usually designed to be run, what happens is it doesn't know we're inside of a package, so it doesn't know where to look for these modules, and then we get this error saying we can't do a relative import if we don't know what the name of the parent package is. So there are ways to fix this, but it involves changing the imports based on how the Python code is executed, where you can use something like the `if __name__ == "__main__":` convention. We're going to talk about that in a different YouTube video, so stay tuned. But the point is, you need to know how you're executing this code and where the entry point is, because that's going to define how these imports work.
Now, what I'm going to do is I'm quickly going to set up a more advanced example with multiple packages that are nested inside of each other, and you can see some more advanced import patterns, and then that will wrap up the video. So I've updated our Python code now to include a few nested packages. You can see that rather than just having `utils`, I've now defined an `m_util` and an `s_util` package inside of this package. So now I've adjusted the imports inside of my `__init__.py` to utilize these nested packages, and you can see that if I want to import the `add` function, I now need to say `from .m_util.math_util import add`, right? So I'm kind of walking through the package structure to know what it is that I'm importing.
Now, the question that I want to pose to you, and that I'll answer obviously, is that from this `math_util` file, if I wanted to import something like the `general_util` function from my `general` file, how would I go about doing that? Well, this is exactly where I can use an absolute import or a relative import that looks up a directory level. Sorry, so from `math_util`, if I want to import this `general_util` function, what I can do is the following. I can say `import` or I can say actually, sorry, `from ..general import general_util`. Now, this is a relative import, and I'm looking two levels up, or one level up the parent directory or the parent package. So rather than looking inside of `m_util`, which is my current package, I'm looking inside of the `utils` package, which is its parent package, where I then find the `general` module and I import `general_util`. So if I run my code now from `main.py`, you can see that I don't get any errors and everything is completely fine.
Now, a slightly different thing. Let's say from my `string_util`, I want to import my `math_util.py`, which is in the same level but in a different package. Now it gets a little bit more complicated, but what I can do is say something like `from ..m_util.math_util import add`. Okay, so if I want to import from a package that's at the same level, sorry, I say, okay, I want to go up to my parent package, which is `utils`. Inside of `utils`, I know I have the `m_util` package, and then inside of `m_util`, sorry, I have `math_util`. I should have named this something better. And then I can import `add`. And then same thing. If I go back to `main.py` and I run this, notice we don't get any errors and the imports work properly.
Okay, so that's it for this video. I wanted to cover all of the advanced imports and how you use Python packages with `__init__.py`. If you found this helpful, make sure to leave a like, subscribe to the channel, and I will see you in the next one. [Music]