5

My first post on here, bear with me.....

I'm working on bundling a tkinter application using PyInstaller, and I've come across an issue when I use the inspect library. Basically, I can call the source code of a class, but not a function, that I have written. I've made this contrived case to demonstrate:

I have a folder structure like this:

experiment/
---experiment.py
---init.py
---resources/
    /---resources.py
    /---init.py
---loader/
    /---loader.py
    /---init.py

resources.py defines one function an one class:

def foo(a):
    print(str(a) + ' is what you entered.')

class Bar:
    def __init__(self, b):
        self.b = b

loader.py imports that function and that class and defines a function to print their source code:

import inspect

from resources import resources

def testfunc():
    source_Bar = inspect.getsource(resources.Bar)
    print(source_Bar)

    source_foo = inspect.getsource(resources.foo)
    print(source_foo)

and experiment.py loads that printing function from loader.py and calls it:

from loader.loader import testfunc

testfunc()

I can run experiment.py in a Python console and get the expected output (source code for foo and Bar).

I then use PyInstaller to create an executable from experiment.py (from a virtual environment with just PyInstaller added). The spec file is untouched (I can share it), but I do copy/paste the loader and resources directories to dist/experiment so that the executable can find them. If I run experiment.exe from the Command Prompt, I get this output:

C:\Users\earne\Desktop\experiment\dist\experiment>experiment.exe
class Bar:
    def __init__(self, b):
        self.b = b

Traceback (most recent call last):
  File "experiment\experiment.py", line 3, in <module>
  File "experiment\loader\loader.py", line 9, in testfunc
  File "inspect.py", line 973, in getsource
  File "inspect.py", line 955, in getsourcelines
  File "inspect.py", line 786, in findsource
OSError: could not get source code
[14188] Failed to execute script experiment

So the code for Bar is found and printed, but the code for foo cannot be found! Note that line 9 is where foo is inspected.

The real context is a graphing program where I want to return the code used to make a plot, in case users want to make tweaks. So foo is actually a lot of matplotlib code, loader is a module for formatting the plot code, and experiment is the tkinter application. As in this case, the inspect works fine from an IDE but breaks after building the exe.

More about my setup:

  • the virtualenv was created with Anaconda Prompt; I used conda create to make a new environment and then pip installed PyInstaller
  • Python is 3.7.7
  • everything in the environment, basically what it comes with:
altgraph       0.17
certifi        2020.4.5.1
future         0.18.2
pefile         2019.4.18
pip            20.0.2
pyinstaller    4.0.dev0+03d42a2a25
pywin32-ctypes 0.2.0
setuptools     46.1.3.post20200330
wheel          0.34.2
wincertstore   0.2
  • Windows 10 64bit

What I've tried:

  • a lot of switching the import method (e.g. explicitly importing the function by name, using import *, importing only the module and using module.function)
  • messing around with the .spec file, like adding my modules to the datas argument of Analysis. I've seen this issue, and I've tried to add my modules to a.pure as htgoebel commented, but I'm not sure if I'm doing this right, and it seems like maybe not the root issue since the code for the class Bar can be found from the same file
  • I've used both the most current PyInstaller version (3.6) and the current developer version from https://github.com/pyinstaller/pyinstaller/archive/develop.zip

Overall the weirdest part seems that the class source code can be found but the function cannot, when they are in the same file - but maybe PyInstaller is making it more complicated than that in a way I don't understand? Please let me know if you have any suggestions or want me to try anything else. Happy to provide any other information/tests that I can. Cheers!

Tom
  • 8,310
  • 2
  • 16
  • 36

1 Answers1

1

I've got a solution after doing a little more poking around - feels not ideal though.

To loader I added statements to print the module and file of both foo and Bar (the "mymod" is unnecessary):

import inspect
from resources import resources as mymod

def testfunc():
    print(inspect.getfile(mymod.Bar))
    print(inspect.getmodule(mymod.Bar))
    source_Bar = inspect.getsource(mymod.Bar)
    print(source_Bar)

    print(inspect.getfile(mymod.foo))
    print(inspect.getmodule(mymod.foo))
    source_foo = inspect.getsource(mymod.foo)
    print(source_foo)

Creating the program again, the output of experiment.exe points to the discrepancy between foo and Bar. The print statements for mymod.Bar are:

C:\Users\...\dist\experiment\resources\resources.pyc
<module 'resources.resources' from 'C:\\Users\...\dist\\experiment\\resources\\resources.pyc'>
class Bar:
    def __init__(self, b):
        self.b = b

and for mymod.foo:

experiment\resources\resources.py
<module 'resources.resources' from 'C:\\Users\...\dist\\experiment\\resources\\resources.pyc'>
Traceback (most recent call last):
... #same error as before

So it seems that Python can recognize that they are from the same module, but not the actual file; there's an absolute path pointing to the .pyc file for Bar, while only a relative path to the .py file for foo.

So I tried to change the imports/be more explicit, and got to using importlib:

from importlib import import_module
mymod = import_module('resources.resources')

This fixed experiment.exe to give the right output:

C:\Users\...\dist\experiment\resources\resources.py
<module 'resources.resources' from 'C:\\Users\...\dist\\experiment\\resources\\resources.py'>
class Bar:
    def __init__(self, b):
        self.b = b

C:\Users\...\experiment\resources\resources.py
<module 'resources.resources' from 'C:\\Users\...\dist\\experiment\\resources\\resources.py'>
def foo(a):
    print(str(a) + ' is what you entered.')

So even for this, I don't understand why changing the import is needed. Moreover, this fix didn't work in the context of my GUI - I still couldn't get the function source code but the class continued to work. I ended up having to follow this example and do:

import importlib.util
homedir = os.path.dirname(os.path.dirname(__file__)
resources_dir = os.path.join(homedir, 'resources/resources.py')
spec = importlib.util.spec_from_file_location("resources.resources", resources_dir)
mymod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mymod)
#I do os stuff to get a relative path to resources from loader

And this works in the application - seems overly verbose and confusing, but ultimately it doesn't apparently affect the functioning of the application, either as an .exe or as a Python script.

Would still love to know what is going on here, if anyone feels like giving a shout. Hope this helps someone if not.

Tom
  • 8,310
  • 2
  • 16
  • 36