Running Luigi task from cmd - "No module named tasks"

Question

I am having problems running a Luigi task through the Windows cmd. Here are the facts:

• Running Anaconda installed in C:\ProgramData\Anaconda2 (Python 2.7)

• Anaconda has added its paths to the PATH variable but there is no PYTHONPATH variable

• The task i am trying to run is located in C:\....\tasks.py

• Trying to run it as follows:

  C:\.... luigi --module tasks MyTask --dt 20170316
  ImportError: No module named tasks

I tried creating a PYTHONPATH variable and adding the exact path to the directory containing my tasks.py file but it didn't work. Another problem I am having, which may be related is when I launch the luigi scheduler through cmd using:

luigid

it works fine but whenever I try to start it using:

luigid --background

I get the following error:

No module named pwd

It seems like there is something wrong with my setup overall, any help would be appreciated.

Answer 1

I had experienced same issue and solved it. Module which you want to schedule or execute must be located in any folder listed in in sys.path One of the way to achive this in WINDOWS TERMINAL/CMD/ is to navigate to folder that you have python module and execute command:

set PYTHONPATH=%cd%;%PYTHONPATH%

That command will add temporary your current directory to existing PYTHONPATH. If you do not have PYTHONPATH variable in system just skip part after semicolon.
In the same terminal window issue luigi command.

luigi --module tasks MyTask --local-scheduler

If you still experience issues please add your PYTHONPATH to PATH variable using:

set PATH=%PYTHONPATH%;%PATH%

For me addidng current folder to PYTHONPATH works and it is easy to execute using batch files. Alternatively you can add this variable permanently in Windows.