Attempted relative import beyond toplevel package

Question

Here is my folder structure:

Mopy/ # no init.py !
   bash/
     __init__.py
     bash.py # <--- Edit: yep there is such a module too
     bass.py
     bosh/
       __init__.py # contains from .. import bass
       bsa_files.py
     ...
   test_bash\
     __init__.py # code below
     test_bosh\
       __init__.py
       test_bsa_files.py

In test_bash\__init__.py I have:

import sys
from os.path import dirname, abspath, join, sep
mopy = dirname(dirname(abspath(__file__)))
assert mopy.split(sep)[-1].lower() == 'mopy'
sys.path.append(mopy)
print 'Mopy folder appended to path: ', mopy

while in test_bsa_files.py:

import unittest
from unittest import TestCase

import bosh

class TestBSAHeader(TestCase):
    def test_read_header(self):
        bosh.bsa_files.Header.read_header()

if __name__ == '__main__':
    unittest.main()

Now when I issue:

python.exe "C:\_\JetBrains\PyCharm 2016.2.2\helpers\pycharm\utrunner.py" C:\path\to\Mopy\test_bash\test_bosh\test_bsa_files.py true

I get:

Traceback (most recent call last):
  File "C:\_\JetBrains\PyCharm 2016.2.2\helpers\pycharm\utrunner.py", line 124, in <module>
    modules = [loadSource(a[0])]
  File "C:\_\JetBrains\PyCharm 2016.2.2\helpers\pycharm\utrunner.py", line 43, in loadSource
    module = imp.load_source(moduleName, fileName)
  File "C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\test_bash\test_bosh\test_bsa_files.py", line 4, in <module>
    import bosh
  File "C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\bash\bosh\__init__.py", line 50, in <module>
    from .. import bass
ValueError: Attempted relative import beyond toplevel package

Since 'Mopy" is in the sys.path and bosh\__init__.py is correctly resolved why it complains about relative import above the top level package ? Which is the top level package ?

Incidentally this is my attempt to add tests to a legacy project - had asked in Python test package layout but was closed as a duplicate of Where do the Python unit tests go?. Comments on my current test package layout are much appreciated !

---

Well the answer below does not work in my case:

The module bash.py is the entry point to the application containing:

if __name__ == '__main__':
    main()

When I use import bash.bosh or from bash import bosh I get:

C:\_\Python27\python.exe "C:\_\JetBrains\PyCharm 2016.2.2\helpers\pycharm\utrunner.py" C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\test_bash\test_bosh\test_bsa_files.py true
Testing started at 3:45 PM ...
usage: utrunner.py [-h] [-o OBLIVIONPATH] [-p PERSONALPATH] [-u USERPATH]
                   [-l LOCALAPPDATAPATH] [-b] [-r] [-f FILENAME] [-q] [-i]
                   [-I] [-g GAMENAME] [-d] [-C] [-P] [--no-uac] [--uac]
                   [--bashmon] [-L LANGUAGE]
utrunner.py: error: unrecognized arguments: C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\test_bash\test_bosh\test_bsa_files.py true

Process finished with exit code 2

This usage message is from the main() in bash.

Answer 1

TLDR: Do

import bash.bosh

or

from bash import bosh

Avoid modifying sys.path, as this duplicates modules.

---

When you do

import bosh

it will import the module bosh. This means Mopy/bash is in your sys.path, python finds the file bosh there, and imports it. The module is now globally known by the name bosh. Whether bosh is itself a module or package doesn't matter for this, it only changes whether bosh.py or bosh/__init__.py is used.

Now, when bosh tries to do

from .. import bass

this is not a file system operation ("one directory up, file bass") but a module name operation. It means "one package level up, module bass". bosh wasn't imported from its package, but on its own, though. So going up one package is not possible - you end up at the package '', which is not valid.

Let's look at what happens when you do

import bash.bosh

instead. First, the package bash is imported. Then, bosh is imported as a module of that package - it is globally know as bash.bosh, even if you used from bash import bosh.

When bosh does

from .. import bass

that one works now: going one level up from bash.bosh gets you to bash. From there, bass is imported as bash.bass.