Relative imports in Python 3

Question

I want to import a function from another file in the same directory.

Sometimes it works for me with from .mymodule import myfunction but sometimes I get a:

SystemError: Parent module '' not loaded, cannot perform relative import

Sometimes it works with from mymodule import myfunction, but sometimes I also get a:

SystemError: Parent module '' not loaded, cannot perform relative import

I don't understand the logic here, and I couldn't find any explanation. This looks completely random.

Could someone explain to me what's the logic behind all this?

Answer 1

unfortunately, this module needs to be inside the package, and it also needs to be runnable as a script, sometimes. Any idea how I could achieve that?

It's quite common to have a layout like this...

main.py
mypackage/
    __init__.py
    mymodule.py
    myothermodule.py

...with a mymodule.py like this...

#!/usr/bin/env python3

# Exported function
def as_int(a):
    return int(a)

# Test function for module  
def _test():
    assert as_int('1') == 1

if __name__ == '__main__':
    _test()

...a myothermodule.py like this...

#!/usr/bin/env python3

from .mymodule import as_int

# Exported function
def add(a, b):
    return as_int(a) + as_int(b)

# Test function for module  
def _test():
    assert add('1', '1') == 2

if __name__ == '__main__':
    _test()

...and a main.py like this...

#!/usr/bin/env python3

from mypackage.myothermodule import add

def main():
    print(add('1', '1'))

if __name__ == '__main__':
    main()

...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...

from .mymodule import as_int

The way you're supposed to run it is...

python3 -m mypackage.myothermodule

...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.

The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...

from mymodule import as_int

...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...

from mypackage.mymodule import as_int

...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...

import sys
import os

PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))

from mypackage.mymodule import as_int

It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...

I'm -1 on this and on any other proposed twiddlings of the __main__ machinery. The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern. To make me change my mind you'd have to convince me that it isn't.

Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.