Library import in Python does not work as expected

Question

I am having a strange error executing this Python code.

import tensorflow as tf
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(2,)),
  tf.keras.layers.Dense(10, activation='sigmoid'),
])
model.compile(optimizer='adam',
              loss='mae',
              metrics=['mse'])
model.summary()

resulting in the following error:

ModuleNotFoundError: No module named 'keras'

( I already tried the same code on google Colab and it works ).

The strange thing for me is that this code instead works without errors on the same machine ( this means that all libraries are there ):

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, Dense

model = Sequential([
  Flatten(input_shape=(28, 28)),
  Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

and, if in a jupyter notebook I execute first the second version and than the first one, I do not encounter errors.

What can be the cause of such behavior and how to fix it ? I am using Python 3.9.7 with miniconda.

UPDATE 1: this behavior seems to go away when downgrading from tensorflow 2.6.0 to tensorflow 2.4.1 .

UPDATE 2: complete traceback as requested:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/tmp/ipykernel_19355/718649126.py in <module>
----> 1 model = tf.keras.models.Sequential([
      2   tf.keras.layers.Flatten(input_shape=(28, 28)),
      3   tf.keras.layers.Dense(10, activation='softmax')
      4 ])
      5 

~/miniconda3/envs/tf/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py in __getattr__(self, item)
     60 
     61   def __getattr__(self, item):
---> 62     module = self._load()
     63     return getattr(module, item)
     64 

~/miniconda3/envs/tf/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py in _load(self)
     43     """Load the module and insert it into the parent's globals."""
     44     # Import the target module and insert it into the parent's namespace
---> 45     module = importlib.import_module(self.__name__)
     46     self._parent_module_globals[self._local_name] = module
     47 

~/miniconda3/envs/tf/lib/python3.9/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load(name, import_)

~/miniconda3/envs/tf/lib/python3.9/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'keras'

Answer 1

It seems that there's a bug in the current version of tensorflow, particularly with the _load() method in the LazyLoader class defined in the module tensorflow.python.util.lazy_loader. I think we could just leave it until being fixed. I'd like to dig into another question raised here: how come your second segment of codes worked yet the first one failed?

The reason comes from the import system as well as the dot operator in Python.

In the statement import m1.m2.m3 or from m1.m2.m3 import n1, the dot notation in m1.m2.m3 is used to separate names of modules(packages) and submodules(subpackages). It's not a real dot operator. When processing this statement, Python will first import the module(package) m1, then m1.m2, then m1.m2.m3. In each step of loading the modules, two notable side-effects occur(among lots of other actions):

• the to-be-imported package's __init__.py file is executed.
• a binding is placed in the parent module’s namespace to the submodule object. For this example, m1.__dict__['m2'] is set to be the module object m1.m2, and m1.m2.__dict__['m3'] is set to be the module object m1.m2.m3.

And for the real dot operator, for example an expression obj1.attr1, the evaluation order will be:

1. evaluate obj1.__getattribute__('attr1') and return the value
2. if no such an attribute found in the first step, then evaluate odj1.__getattr__('attr1')

where the search of obj1.__dict__['attr1'] is launched in the first step.

Back to tensorflow, there's such a statement in __init__.py under the package tensorflow:

keras = _LazyLoader("keras", globals(), _keras_module)

Here's a simplified "second version" of your sample as below:

from tensorflow.keras.models import Sequential 
# tensorflow.__dict__['keras'] is set to be the module tensorflow.keras
import tensorflow as tf # make the module name tensorflow useable

print(type(tf.keras)) # tf.keras is evaluated to the submodule tensorflow.keras
# output: <class 'module'>

It turns out the name binding to the submodule object overwrites the binding(assignment) in __init__.py.

But if we don't import the submodule tensorflow.keras, then the binding in __init__.py works:

import tensorflow as tf

print(type(tf.keras))
# output: <class 'tensorflow.python.util.lazy_loader.LazyLoader'>

That explains why your second method worked but the first one failed.

---

Reference:

For import mechanism:

• 7.11. The import statement
• 5. The import system

For dot operator:

• Descriptor HowTo Guide - Invocation from an instance

Answer 2

You can use from tensorflow import keras and try again executing the same code.

if it shows the same error again then run below code in jupyter notebook:

import keras
print(keras.__version__)
import tensorflow
print(tensorflow.__version__)

if you find some differences in versions, then execute below code in jupyter notebook which will automatically install upgraded and compatible version of tensorflow and keras along with its other required libraries.

!pip insatll tensorflow
import tensorflow
print(tensorflow.__version__)

!pip install keras
import keras
print(keras.__version__)

Let us know if issue still persists.