PyInstaller: a module is not included into --onefile, but works fine with --onedir

Question

I'm using PyInstaller to bundle my application into one .exe file. The problem is that it works fine with --onedir option, but can't find a module when built with --onefile.

Both --onedir and --onefile say during the building process:

<...>
INFO: Analyzing hidden import 'sklearn.utils.sparsetools._graph_validation'
<...>

Running the instance created with --onedir works fine, but the instance produced by --onefile dies:

<...>
  File "_min_spanning_tree.pyx", line 8, in init sklearn.utils.mst._min_spanning
_tree (sklearn\utils\sparsetools\_min_spanning_tree.c:4754)
ImportError: No module named _graph_validation

Here are my .spec files

onedir.spec

# -*- mode: python -*-
a = Analysis(['../../brainactivity.py'],
             hiddenimports=['greenlet', 'sklearn.utils.sparsetools._graph_validation', 'sklearn.utils.sparsetools._graph_tools', 'scipy.special._ufuncs_cxx', 'sklearn.utils.lgamma', 'sklearn.utils.weight_vector'],
             hookspath=None,
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='brainactivity.exe',
          debug=False,
          strip=None,
          upx=True,
          console=True,)
coll = COLLECT(exe,
               a.binaries,
               [('./data/201305182224-DF-facial-3-420.csv', '../../data/201305182224-DF-facial-3-420.csv', 'DATA')],
               [('./model/brain_20k_colored_properly.obj', '../../model/brain_20k_colored_properly.obj', 'DATA')],
               [('brain_fragment_shader.glsl', '../../brain_fragment_shader.glsl', 'DATA')],
               [('brain_vertex_shader.glsl', '../../brain_vertex_shader.glsl', 'DATA')],
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='brainactivity')

onefile.spec

# -*- mode: python -*-
a = Analysis(['../../brainactivity.py'],
             hiddenimports=['greenlet', 'sklearn.utils.sparsetools._graph_validation', 'sklearn.utils.sparsetools._graph_tools', 'scipy.special._ufuncs_cxx', 'sklearn.utils.lgamma', 'sklearn.utils.weight_vector'],
             hookspath='.',
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          [('./data/201305182224-DF-facial-3-420.csv', '../../data/201305182224-DF-facial-3-420.csv', 'DATA')],
          [('./model/brain_20k_colored_properly.obj', '../../model/brain_20k_colored_properly.obj', 'DATA')],
          [('brain_fragment_shader.glsl', '../../brain_fragment_shader.glsl', 'DATA')],
          [('brain_vertex_shader.glsl', '../../brain_vertex_shader.glsl', 'DATA')],
          a.zipfiles,
          a.datas,
          name='brainactivity.exe',
          debug=False,
          strip=None,
          upx=True,
          console=True )

Answer 1

I had the same error. The solution is to create a hook for sklearn. Generally, u need to create a hook file like this

hiddenimports = ['sklearn.utils.sparsetools._graph_validation'] 

and save this in a file with name hook-modulename.py in the same folder. But this will import only _graph_validation. This might lead to error for another module. Best to import all the submodules in a package by

from hookutils import collect_submodules
hiddenimports = collect_submodules('sklearn') 

and save it to a hook file in the same folder. For me, I had to create 2 hook file. one for sklearn and one for scipy.

from hookutils import collect_submodules
hiddenimports = collect_submodules('scipy') 

after saving them I used below command to run

pyinstaller --additional-hooks-dir=. myfile.py

for better understanding follow this link.