My question is similar to How to import a module given the full path? however I'm not importing a .py source file, I'm importing a package with a .pyd.
During runtime I am creating new package modules from some dynamically generated c code. I successfully generate package Foo with a __init__.py file and a mod.pyd:
/a/temp/dir/foo/
    __init__.py
    bar.pyd
The example code I'm working with is
import importlib.util
spec = importlib.util.spec_from_file_location("foo.bar", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
foo.bar()
If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/__init__.py') the module bar from the pyd fails to load.
If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/') then spec_from_file_location returns None.
If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/bar.pyd') I get the following error stack:
File "<frozen importlib._bootstrap>", line 577, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 903, in create_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
An alternative to the recommended 4 line importlib solution is to create concrete classes for MetaPathFinder and SourceLoader. My following MetaPathFinder works fine, however, I am unsure how to implement the Loader correctly. get_data return value expects some Unicode representation of python code. How do I return the package structure or the pyd contents in the loader?
class MyLoader(importlib.abc.SourceLoader):
    def __init__(self, fullname, path, pyd):
        self.path = path
        self.fullname = fullname
        self.pyd = pyd
    def get_data(self, path):
        print('*** get_data', path)
        with open(os.path.join(self.path, self.pyd), 'r') as f:
            return f.read()
    def get_filename(self, fullname):
        print('*** get_filename', fullname)
        return os.path.join(self.path, '__init__.py')
    def is_package(self):
        return True
class MyFinder(MetaPathFinder):
    def __init__(self, this_module, workdir, pyd):
        self._this_module = this_module
        self._workdir = workdir
        self._pyd = pyd
    def find_spec(self, fullname, path, target=None):
        print('find_spec', fullname, path, target, self._this_module)
        if fullname != self._this_module:
            return
        filename = os.path.join(self._workdir, self._this_module)
        spec = ModuleSpec(
            name = fullname,
            loader = MyLoader(fullname, filename, self._pyd),
            origin = filename,
            loader_state = 1234,
            is_package = True,
        )
        return spec
    def is_package(self):
        return True
 
    