Relative import problems in Python 3

Question

Python imports drive me crazy (my experience with python imports sometime doesn't correspond at all to idiom 'Explicit is better than implicit' :( ):

[app]
    start.py
        from package1 import module1
    [package1]
        __init__.py
            print('Init package1')
        module1.py
            print('Init package1.module1')
            from . import module2
        module2.py
            print('Init package1.module2')
            import sys, pprint
            pprint.pprint(sys.modules)
            from . import module1

I get:

vic@ubuntu:~/Desktop/app2$ python3 start.py 
Init package1
Init package1.module1
Init package1.module2
{'__main__': <module '__main__' from 'start.py'>,
 ...
 'package1': <module 'package1' from '/home/vic/Desktop/app2/package1/__init__.py'>,
 'package1.module1': <module 'package1.module1' from '/home/vic/Desktop/app2/package1/module1.py'>,
 'package1.module2': <module 'package1.module2' from '/home/vic/Desktop/app2/package1/module2.py'>,
 ...
Traceback (most recent call last):
  File "start.py", line 3, in <module>
    from package1 import module1
  File "/home/vic/Desktop/app2/package1/module1.py", line 3, in <module>
    from . import module2
  File "/home/vic/Desktop/app2/package1/module2.py", line 5, in <module>
    from . import module1
ImportError: cannot import name module1
vic@ubuntu:~/Desktop/app2$ 

import package1.module1 works, but i want to use from . import module1 because i want to make package1 portable for my other applications, that's why i want to use relative paths.

I am using python 3.

I need circular imports. A function in module1 asserts that one of its parameter is instance of a class defined in module2 and viceversa.

In other words:

sys.modules contains 'package1.module1': <module 'package1.module1' from '/home/vic/Desktop/app2/package1/module1.py'>. I want to get a reference to it in form from . import module1, but it tries to get a name, not a package like in case import package1.module1 (which works fine). I tried import .module1 as m1 - but that's a syntax error.

Also, from . import module2 in module1 works fine, but from . import module1 in module2 doesn't work...

UPDATE:

This hack works (but i am looking for the 'official' way):

print('Init package1.module2')

import sys, pprint
pprint.pprint(sys.modules)

#from . import module1
parent_module_name = __name__.rpartition('.')[0]
module1 = sys.modules[parent_module_name + '.module1']

Answer 1

Circular imports should be generally avoided, see also this answer to a related question, or this article on effbot.org.

In this case the problem is that you import from . where . is the current package. So all your from . import X imports go through the package’s __init__.py.

You can make your problem a bit more visible, if you explicitely import your modules in the __init__.py and give them another name (and adjust the other imports to use those names of course):

print('Init package1')
from . import module1 as m1
from . import module2 as m2

Now when you are importing m1 in start.py, the package first initializes m1 and comes to the from . import m2 line. At that point, there is no m2 known in __init__.py so you get an import error. If you switch the import statements in __init__.py around (so you load m2 first), then in m2 it finds the from . import m1 line, which fails for the same reason as before.

If you don’t explicitely import the modules in __init__.py something similar still happens in the background. The difference is that you get a less flat structure (as the imports are no longer started from the package only). As such both module1 and module2 get “started” and you get the respective initialization prints.

To make it work, you could do an absolute import in module2. That way you could avoid that the package needs to resolve everything first, and make it reuse the import from start.py (as it has the same import path).

Or even better, you get rid of the circular import at all. It’s generally a sign that your application structure is not so good, if you have circular references.

(I hope my explanation makes any sense, I already had difficulties writing it, but the general idea should be clear, I hope…)

edit

In response to your update; what you are doing there is that you use the full package name to get the reference to the module. This is equivalent (but much more complicated) to the first possible option to make it work; you use an absolute import using the same import path as in start.py.