Creating Local Python Packages with __init__.py

Packages for code reuse and distribution

Photo by boris misevic on Unsplash

When it comes to choosing a coding language for a project, I generally select from a few options available to me at work. If it’s only a few lines, I tend to use Bash. If it’s going to require software architecting, I tend to use C++ or Java (I really want to get into Rust, but there’s very limited support for that at my current job). For everything in between I use Python.

Python is a perfect “intermediate” language for those quick one-off scripts, and it works for larger projects, but my small Python scripts tend to grow over time. I need my code to be relatively portable, but building everything into its own module as the code grows leads to duplication, unnecessary bloat, and even code rot.

Virtually every language has a way to build and load common functionality. In Python, the feature is known as a “package”. Packages can be built for large distributions, but here I want to talk about creating simple package directories that you can use to promote code reuse among your own scripts or when you need to send a tarball of a script off to a friend or co-worker. Those don’t need to be as rigidly defined.

I find myself frequently looking up the options in __init__.py and confusing the options. Here's a quick primer in how to build packages you can send off with your scripts. These should work across most versions of Python (tested in 2.7 and 3.6).

I’m going to focus on the __init__.py packaging and not namespace packaging here. I find this method to be the easiest to use and useful just about everywhere in the size project I tend to work in. I’m also not going to talk about setup.py and what else can go in __init__.py (eg: version, author, etc). For small scripts that just need code reuse, I generally don’t bother. If for no other reason than I don’t need my coworkers hunting me down to fix code they borrowed from me in the first place!

I find it's most helpful to have a project to test with, so let's build one that looks like this:

The purpose of this isn't to make anything "useful", but to demonstrate the concept. So let's make some really simple functionality in each module.

time_pkg/PrettyTime.py

Three methods here.

dtToPrettyTime takes a datetime and returns the pretty printed time string. This is called by the other two methods to standardize the output.

ssepToPrettyTime takes an ssep (Seconds Since Epoch, the Unix Timestamp date) and returns the pretty printed time.

getPrettyTime just returns the current time as a pretty printed string.

time_pks/ssep.py

Similar to PrettyTime.py.

dtToSsep is the helper method that takes a datetime and returns the ssep.

prettyTimeToSsep takes in the pretty time string and returns the ssep.

getSsep gets the current ssep.

print_pkg/printer.py

Just two functions to help illustrate the output.

hello prints “Hello World”.

output stringifies and prints any string you give to it.

To turn those directories into packages, all you have to do is add an empty __init__.py file under the print_pkg and time_pkg directories. To do that, just do “touch print_pkg/__init__.py” and “touch time_pkg/__init__.py”.

Now it’s time to look at example.py and see how we bring in these packages.

example.py - empty __init__.py

Used this way, you have to explicitly import each file and call each method

There are a few things to point out here.

We imported the module prettyTime by using dot notation. “import time_pkg.prettyTime” imports the entire prettyTime module, but also requires you to call the methods with dot notation, so you’ll see “time_pkg.prettyTime.getPrettyTime().

We then import the ssep module with the “from” syntax, calling “from time_pkg import ssep”. This lets us shorten the calling method name to ssep.getSsep(), for example.

Finally, we can import the methods directly like we did with hello and output, and then we can call those methods directly.

Note that we can directly import the files in time_pkg and print_pkg without telling Python where they are or using system paths if they’re in the same directory as the script calling them. Python uses the location of the script as a part of the system path when determining where packages are to import. Even if example.py is called from a different directory or from a softlink, putting the packages at the same level as example.py will let python find them.

This is pretty useful since we can even include packages within each other (though it makes dependencies tougher). Let’s edit our print_pkg/printer.py to also print the time when we call hello():

And that’s it. We’ve imported the prettyTime module from time_pkg and can use its methods.

This is relatively easy, but it’s not the best. dtToPrettyTime and dtToSsep really are helper functions that don’t need to be exposed.

example.py - __init__.py with specific imports

In the __init__.py for time_pkg, we can explicitly call out what we want to import and that’s what will be available for export if someone imports time_pkg. Note the “.” at the beginning of the import. This is telling Python to look in the current directory for the modules. We could also use “from import time_pkg.prettyTime import ssepToPrettyTime”. I tend not to do that because someone might rename my package (top-level package directory). I prefer to assume I control only what’s inside my package directory.

To use it, just import the package itself (or use the “from” syntax to pick and choose):

An argument against this is that it can dirty the namespace a bit. It makes the methods available at two locations. If we wanted to call “getSsep”, we could now call it by calling “time_pkg.getSsep()” or by calling “time_pkg.ssep.getSsep()”. It’s in two places.

An argument for this is that it makes clear what’s supposed to be exposed to the user, and it’s at the top level of the package instead of making the user go searching for the right module and method to use.

I prefer this method because it acts like an API. Bigger projects and professional packages might definitely go another route.

example.py - __init__.py exporting “all”

We can go one step further to clearly define our intent in __init__.py by creating the __all__ variable. Take a look at how we might rewrite the __init__.py in our print_pkg:

Now we can import from our print_pkg directly into our script’s namespace just like if we’d called “from print_pkg.printer import hello” and “from print_pkg.printer import output” like this:

Note that we can still call output and hello directly.

__all__ is used as a way to let users know exactly what the package contains. You can still use print_pkg.hello() and print_pkg.output if you just call it with “import print_pkg” as well, so this really leaves it up to the end-user how they want to call your package within their script. Please give your package methods unique names.

None of this will hide methods from package users, they’ll just have to use dot notation to call it directly. __all__ is often used as a kind of declaration of what will be supported, though, as you continue to develop your package. If an end-user goes off-script and calls something not in __all__, all bets are off in future releases.

That’s all folks!

I hope that’s helped explain the basics of what you can put in __init.py__ at a very high level. A lot more can be done there, __init__.py is a full module in and of itself of course! This is simply how I use it, and the methods described are safe across most common versions of Python.

Citizen Upgrade is a community of experts covering technology, society, and personal development. Visit us at our website, on Facebook, or on Twitter.