(Relative) imports in python

.

Realtive imports in python

I sometimes prioritize delivery speed in my work and may overlook certain concepts that I can work around. One such concept is relative imports in python. I am certainly not the first developer to encounter the following error when executing a random Python script:

ImportError: attempted relative import with no known parent package

So here is my attempt to explain to myself what is going on.

Difference between a package and a module

Let's start with the fundamentals. A module refers to a file containing Python code, while a package is a folder that houses Python code/modules. A package can include both modules and other packages.

A package can be perceived as a specialized form of a module that encompasses other modules.

In the following directory structure, the names clarify the distinction between modules and packages:

imports_project
├── main.py
├── module1.py
└── package1
    ├── __init__.py
    ├── module1.py
    ├── module2.py
    package2
        ├── __init__.py
        ├── module1.py
        └── module2.py

Observe how different packages may contain modules with identical names.

Relative imports

Relative imports are imports that start with a . and are used to import modules or packages relative to the current module.

So in package2.module1 I can say:

# package1/package2/module1

from . import module2
from .. import module1

and these should be equivalent to:

# package1/package2/module1

from package1.package2 import module2
from package1 import module1

Every time you use a ., you are going one level up from the current file.

You can obviously also go down a level by using the name of the module or package you want to import. So you could say

# package2/module1.py
from ..package2 import module2

this would be equivalent to:

# package2/module1.py
from . import module2

But why would you?

Of course, this is the same as saying:

# package2/module1.py
import module2

but that's not a relative import.

To the actual problem

Now if you tried to go up 3 levels in package2.module1:

# package1/package2/module1

from ... import module1

you end up with the infamous error:

ImportError: attempted relative import with no known parent package

This is because relative imports require a parent package to be defined. Explicitly they need the variable __package__ to be not empty. You can verify this by adding a simple print to each file:

print(f"in {__file__} the variable __package__ is {__package__}")

And you will see when __package__ is empty (in main.py) and it is not (in files under package1 and package2).

Similarly, in main.py you cannot import module1 using relative imports, because main.py is not part of a package.

So you can do

# main.py
import module1

but you cannot do

# main.py
from . import module1

So to summarise:

  • if the file you are referring to is not part of a package, you cannot load it using relative import.
  • outside a package you cannot use relative imports.

So what is the point of relative imports?

Relative imports can be handy when creating packages. They are less verbose as you don't have to define the full route. So you can say

from ... import module1

but in absolute terms you might have to say:

from package1.package2.package3 import module1

However, notice there is a readability cost as the reader will have to look at the hireachy to understand what is being imported.

Moreover, without relative imports, debugging can be easier. For example, remember you cannot use relative imports when executing a module directly, i.e. when __name__ == "__main__".

Conclusion

  • Utilize relative imports during package development, but avoid them in scripts.
  • Be mindful of the impact on code readability when using relative imports.
  • Relative imports cannot be used at the main entry point of your Python program.

bonus 1: a note on __all__ variable

If the __all__ variable inside __init__.py is explicitly defined inside package1/__init__.py, then modules listed in __all__ will be imported when you do from package1 import *, otherwise none will be imported. You can also add other methods, etc to the __all__ variable to allow import directly from the package, so you can say from package1 import method1 instead of from package1.module1 import method1.

bonus 2: packages with - in their name

I was developing my first package last week and I named it wb-data. Doh. This is the first time I noticed that there aren't any packages in python that have - in the name. PEP8 has some good advice here:

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

It is funny how you don't notice rules until you actually break them.

bonus 3: developing python packages locally

I am using gitlab private registry to deploy a Pypi package. Pipenv allows you to add additional sources that you can reference using index in your Pipfile:

wb_data = {version="~=0.0.2", index="gitlab"}

However, I wanted to use the package locally without having to deploy it every time to the registry. Turns out there is handy pip argument --editable for this:

pip install -e ../local_copy_of_wb_data

This will just save a reference to the local copy of the package in your site-packages folder. You can then import it as usual.