Problem with Pyinstaller and Scikit-learn

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with Pyinstaller and Scikit-learn

jonathan.kauffman
Hello All,

I am using Pyinstaller to generate an executable for a Python program that uses the scikit-learn library.  I am able to create this executable and run it without error on my Windows 7 machine, but when I run the same executable in a VirtualBox Windows 7 virtual machine (in order to determine if the executable will run on a "fresh" Windows installation) I receive the following error:

C:\Users\CleanInstall\Desktop\packaged-deployed-driver>deployed-driver.exe
LOADER: executable is C:\Users\CLEANI~1\Desktop\PACKAG~1\deployed-driver.exe
LOADER: homepath is C:\Users\CLEANI~1\Desktop\PACKAG~1
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\Users\CLEANI~1\Desktop\PACKAG~1\deployed-driver.exe
LOADER: Extracting binaries
LOADER: Executing self as child
LOADER: Setting up to run child
LOADER: Creating child process
LOADER: Waiting for child process to finish...
LOADER: executable is C:\Users\CLEANI~1\Desktop\PACKAG~1\deployed-driver.exe
LOADER: homepath is C:\Users\CLEANI~1\Desktop\PACKAG~1
LOADER: _MEIPASS2 is C:\Users\CLEANI~1\AppData\Local\Temp\_MEI24282
LOADER: archivename is C:\Users\CLEANI~1\Desktop\PACKAG~1\deployed-driver.exe
LOADER: Already in the child - running user's code.
LOADER: manifestpath: C:\Users\CLEANI~1\AppData\Local\Temp\_MEI24282\deployed-dr
iver.exe.manifest
LOADER: Activation context created
LOADER: Activation context activated
LOADER: Python library: C:\Users\CLEANI~1\AppData\Local\Temp\_MEI24282\python27.
dll
LOADER: Manipulating evironment
LOADER: PYTHONPATH=C:\Users\CLEANI~1\AppData\Local\Temp\_MEI24282
LOADER: PYTHONHOME=C:\Users\CLEANI~1\AppData\Local\Temp\_MEI24282
LOADER: Manipulating Python's sys.path
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: extracted pyi_os_path
LOADER: extracted pyi_archive
LOADER: extracted pyi_importers
LOADER: Installing import hooks
LOADER: out00-PYZ.pyz
LOADER: Running scripts
Traceback (most recent call last):
  File "<string>", line 24, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\Detectors.MultiDetector", line 6, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn", line 31, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.__check_build", line 47, in <module>
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.__check_build", line 26, in raise_build_error
WindowsError: [Error 3] The system cannot find the path specified: 'C:\\Users\\C
LEANI~1\\AppData\\Local\\Temp\\_MEI24282\\sklearn\\__check_build/*.*'
LOADER: RC: -1 from deployed-driver
LOADER: OK.
LOADER: Deactivating activation context
LOADER: Releasing activation context
LOADER: Done
LOADER: Cleaning up Python interpreter.
LOADER: Back to parent
LOADER: Doing cleanup
LOADER: Freeing archive status for C:\Users\CLEANI~1\Desktop\PACKAG~1\deployed-d
river.exe

My development machine is running Windows 7 64-bit, Python 2.7.3 32-bit, the latest development version of Pyinstaller (pulled today), and the latest development version of scikit-learn (also pulled today).  I was able to reproduce this error by creating an executable from a Python script with the following line:

from sklearn.utils import weight_vector

I am executing the following command in order to create the executable:

python pyinstaller.py --onefile --debug path\to\code\deployed-driver.py

Can anyone suggest a reason why my executable is not running properly in the virtual machine?  Let me know if you need any additional information to help identify the problem.

Thanks,

Jonathan

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Pyinstaller and Scikit-learn

Martin Zibricky
[hidden email] píše v Út 05. 03. 2013 v 18:01 -0800:
> python pyinstaller.py --onefile --debug path\to\code
> \deployed-driver.py

> Can anyone suggest a reason why my executable is not running properly
> in the virtual machine?  Let me know if you need any additional
> information to help identify the problem.

First try --onedir mode and ensure all files are bundled properly.

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|

Re: Problem with Pyinstaller and Scikit-learn

jonathan.kauffman
Thanks for the quick response.

I tried --onedir mode as you suggested for both my complete program and the minimal example that reproduces the error message.  After creating the executable in --onedir mode for the complete program I received the following error when running the executable:

LOADER: executable is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe
LOADER: homepath is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe
LOADER: No need to extract files to run; setting extractionpath to homepath
LOADER: Already in the child - running user's code.
LOADER: manifestpath: C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe.man
ifest
LOADER: Activation context created
LOADER: Activation context activated
LOADER: Python library: C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\python27.dll
LOADER: Manipulating evironment
LOADER: PYTHONPATH=C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1
LOADER: PYTHONHOME=C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1
LOADER: Manipulating Python's sys.path
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: extracted pyi_os_path
LOADER: extracted pyi_archive
LOADER: extracted pyi_importers
LOADER: Installing import hooks
LOADER: out00-PYZ.pyz
LOADER: Running scripts
Traceback (most recent call last):
  File "<string>", line 24, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\Detectors.MultiDetector", line 6, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.svm", line 13, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.svm.classes", line 3, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.linear_model", line 15, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.linear_model.least_angle", line 24, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.cross_validation", line 25, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.metrics", line 32, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.metrics.scorer", line 25, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.metrics.cluster", line 8, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i
n load_module
    exec(bytecode, module.__dict__)
  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o
ut00-PYZ.pyz\sklearn.metrics.cluster.supervised", line 19, in <module>
  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 404, i
n load_module
    module = imp.load_module(fullname, fp, filename, self._c_ext_tuple)
  File "expected_mutual_info_fast.pyx", line 1, in init sklearn.metrics.cluster.
expected_mutual_info_fast (sklearn\metrics\cluster\expected_mutual_info_fast.c:4
869)
ImportError: No module named lgamma
LOADER: RC: -1 from deployed-driver
LOADER: OK.
LOADER: Deactivating activation context
LOADER: Releasing activation context
LOADER: Done
LOADER: Cleaning up Python interpreter.

I was able to resolve this by adding an explicit import for lgamma to deployed-driver.py.  However, when I transferred the directory produced by Pyinstaller to my virtual machine and attempted to run the executable, I received the same error message that was included in my original post.

I also ran Pyinstaller in --onedir mode for my minimal example and did not have any difficulty running the executable in my build environment, but I received the same WindowsError after transferring the directory produced by Pyinstaller to the virtual machine and attempting to run the executable.

Here are the contents of the dist\ directory:

 Directory of C:\pyinstaller-develop\SklearnMinimalExample\dist\SklearnMinimalEx
ample

03/06/2013  07:37 AM    <DIR>          .
03/06/2013  07:37 AM    <DIR>          ..
04/10/2012  11:31 PM            59,904 bz2.pyd
03/06/2013  07:37 AM    <DIR>          include
03/01/2013  08:28 AM             1,857 Microsoft.VC90.CRT.manifest
03/01/2013  08:28 AM           224,768 msvcm90.dll
03/01/2013  08:28 AM           568,832 msvcp90.dll
03/01/2013  08:28 AM           655,872 msvcr90.dll
02/09/2013  06:05 PM         1,313,415 numpy.core.multiarray.pyd
02/09/2013  06:05 PM           174,793 numpy.core.scalarmath.pyd
02/09/2013  06:05 PM           410,944 numpy.core.umath.pyd
02/09/2013  06:05 PM         2,222,455 numpy.core._dotblas.pyd
02/09/2013  06:06 PM            46,383 numpy.fft.fftpack_lite.pyd
02/09/2013  06:06 PM            41,019 numpy.lib._compiled_base.pyd
02/09/2013  06:06 PM         2,382,083 numpy.linalg.lapack_lite.pyd
02/09/2013  06:06 PM           521,885 numpy.random.mtrand.pyd
04/10/2012  11:31 PM           103,424 pyexpat.pyd
04/10/2012  11:31 PM         2,303,488 python27.dll
10/27/2012  09:20 AM           110,080 pywintypes27.dll
02/16/2013  08:55 AM           180,355 scipy.io.matlab.mio5_utils.pyd
02/16/2013  08:55 AM            48,516 scipy.io.matlab.mio_utils.pyd
02/16/2013  08:55 AM            75,662 scipy.io.matlab.streams.pyd
02/16/2013  08:56 AM           108,705 scipy.linalg.calc_lwork.pyd
02/16/2013  08:56 AM            59,065 scipy.linalg._cblas.pyd
02/16/2013  08:56 AM         2,485,538 scipy.linalg._clapack.pyd
02/16/2013  08:56 AM         2,720,377 scipy.linalg._fblas.pyd
02/16/2013  08:56 AM         6,897,971 scipy.linalg._flapack.pyd
02/16/2013  08:56 AM         2,208,558 scipy.linalg._flinalg.pyd
02/16/2013  09:01 AM            89,564 scipy.sparse.csgraph._min_spanning_tree.p
yd
02/16/2013  09:01 AM           233,527 scipy.sparse.csgraph._shortest_path.pyd
02/16/2013  09:01 AM           145,501 scipy.sparse.csgraph._tools.pyd
02/16/2013  09:01 AM           168,478 scipy.sparse.csgraph._traversal.pyd
02/16/2013  08:57 AM         2,822,178 scipy.sparse.linalg.dsolve._superlu.pyd
02/16/2013  08:58 AM         4,012,276 scipy.sparse.linalg.eigen.arpack._arpack.
pyd
02/16/2013  08:57 AM           928,767 scipy.sparse.linalg.isolve._iterative.pyd

02/16/2013  09:00 AM         1,641,587 scipy.sparse.sparsetools._bsr.pyd
02/16/2013  08:59 AM           675,572 scipy.sparse.sparsetools._coo.pyd
02/16/2013  08:59 AM         1,131,421 scipy.sparse.sparsetools._csc.pyd
02/16/2013  09:00 AM           538,950 scipy.sparse.sparsetools._csgraph.pyd
02/16/2013  08:59 AM         1,503,879 scipy.sparse.sparsetools._csr.pyd
02/16/2013  09:00 AM           571,548 scipy.sparse.sparsetools._dia.pyd
02/16/2013  09:01 AM           229,202 scipy.spatial.ckdtree.pyd
02/16/2013  09:01 AM         1,315,873 scipy.spatial.qhull.pyd
02/16/2013  09:01 AM            46,248 scipy.spatial._distance_wrap.pyd
02/16/2013  09:02 AM         1,483,700 scipy.special.specfun.pyd
02/16/2013  09:02 AM         2,551,131 scipy.special._ufuncs.pyd
02/16/2013  09:02 AM           137,879 scipy.special._ufuncs_cxx.pyd
04/10/2012  11:31 PM             9,728 select.pyd
03/05/2013  08:26 PM            55,808 sklearn.metrics.cluster.expected_mutual_i
nfo_fast.pyd
03/05/2013  08:25 PM           131,072 sklearn.metrics.pairwise_fast.pyd
03/05/2013  08:25 PM            14,336 sklearn.utils.lgamma.pyd
03/05/2013  08:25 PM            56,832 sklearn.utils.murmurhash.pyd
03/05/2013  08:25 PM            66,048 sklearn.utils.sparsefuncs.pyd
03/05/2013  08:25 PM            37,888 sklearn.utils.weight_vector.pyd
03/05/2013  08:24 PM            10,752 sklearn.__check_build._check_build.pyd
03/06/2013  07:37 AM         3,287,022 SklearnMinimalExample.exe
03/06/2013  07:37 AM               488 SklearnMinimalExample.exe.manifest
04/10/2012  11:31 PM           686,592 unicodedata.pyd
10/27/2012  09:21 AM            98,816 win32api.pyd
10/27/2012  09:20 AM            25,600 win32pdh.pyd
10/27/2012  09:20 AM            24,064 win32pipe.pyd
04/10/2012  11:31 PM            74,240 _ctypes.pyd
04/10/2012  11:31 PM           285,184 _hashlib.pyd
04/10/2012  11:31 PM            23,552 _multiprocessing.pyd
04/10/2012  11:31 PM            40,960 _socket.pyd
04/10/2012  11:31 PM           721,920 _ssl.pyd
              63 File(s)     51,804,132 bytes

 Directory of C:\pyinstaller-develop\SklearnMinimalExample\dist\SklearnMinimalEx
ample\include

03/06/2013  07:37 AM    <DIR>          .
03/06/2013  07:37 AM    <DIR>          ..
04/10/2012  11:18 PM            21,529 pyconfig.h
               1 File(s)         21,529 bytes

     Total Files Listed:
              64 File(s)     51,825,661 bytes
               5 Dir(s)  101,922,615,296 bytes free

As far as I can tell there aren't any missing files.

On Wednesday, March 6, 2013 5:05:31 AM UTC-5, Martin Z wrote:
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="yzlKcZiSRswJ">jonathan...@... píše v Út 05. 03. 2013 v 18:01 -0800:
> python pyinstaller.py --onefile --debug path\to\code
> \deployed-driver.py

> Can anyone suggest a reason why my executable is not running properly
> in the virtual machine?  Let me know if you need any additional
> information to help identify the problem.

First try --onedir mode and ensure all files are bundled properly.

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Pyinstaller and Scikit-learn

Martin Zibricky
In reply to this post by jonathan.kauffman
[hidden email] píše v Út 05. 03. 2013 v 18:01 -0800:
> ut00-PYZ.pyz\sklearn.__check_build", line 26, in raise_build_error
> WindowsError: [Error 3] The system cannot find the path specified: 'C:
> \\Users\\C LEANI~1\\AppData\\Local\\Temp\\_MEI24282\\sklearn\
> \__check_build/*.*'

This is definitely an issue with scikit-learn. It expects some
directories to exist.

Look at file ./sklearn/__check_build/__init__.py and other files there.

scikit-learn package is trying to do 'os.listdir()' for
directory .\sklearn\__check_build that does not exist. However, it does
this already when trying to raise an exception. You could closer at how
sklearn is trying to verify the build!

You could try to create this directory in --onedir mode and if necessary
put there some files.



--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.