1

Hi I'm trying to use systemml for keras parallelization in IBMcloud but when I run the code:

sysml_model = Keras2DML(spark, keras_Model, input_shape=(1,1), weights='weights_dir', batch_size=batch_size, max_iter=max_iter, test_interval=0, display=10)

I get fowlloing error, can you help me with any sloution:

<ipython-input-20-772087af6a08> in <module>()
      4 samples = train.count()
      5 max_iter = int(epochs*math.ceil(samples/batch_size))
----> 6 sysml_model = Keras2DML(spark, keras_Model, input_shape=(1,1), weights='weights_dir', batch_size=batch_size, max_iter=max_iter, test_interval=0, display=10)
      7 sysml_model.fit(train[1], train[3])

/home/spark/shared/user-libs/python3.6/systemml/mllearn/estimators.py in __init__(self, sparkSession, keras_model, input_shape, transferUsingDF, load_keras_weights, weights, labels, batch_size, max_iter, test_iter, test_interval, display, lr_policy, weight_decay, regularization_type)
   1033         regularization_type: regularization type (default: "L2")
   1034         """
-> 1035         from .keras2caffe import convertKerasToCaffeNetwork, convertKerasToCaffeSolver, convertKerasToSystemMLModel
   1036         import tempfile, keras
   1037         if keras.backend.image_data_format() != 'channels_first':

/home/spark/shared/user-libs/python3.6/systemml/mllearn/keras2caffe.py in <module>()
     26 import os
     27 import math
---> 28 from itertools import chain, imap
     29 from ..converters import *
     30 from ..classloader import *

ImportError: cannot import name 'imap'
Vaibhav
  • 2,527
  • 1
  • 27
  • 31
Farhad
  • 147
  • 2
  • 9
  • 1
    I'm facing the same problem. Seems SystemML is not compatible with Python3 yet. imap is a py2 function. I even tried installing systemml 1.3.0. No go. – Vaibhav Mar 30 '20 at 15:47
  • Is there any way to ignore this warning in code? or force to use map? if you find out let me know please – Farhad Mar 31 '20 at 07:16
  • I edited the script file throwing the error and replaced the import line with: https://stackoverflow.com/a/30271777/1578274 which fixed this issue. But then encountered other Py3 incompatibilities in SystemML. – Vaibhav Apr 01 '20 at 17:29

1 Answers1

0

imap is not present anymore in the itertools module of Python 3. What you need to do is use the 2to3 package that comes built-in with Python 3. So all you need to do is 2to3 estimators.py in the terminal. This would show you all the changes you need to make to the estimators.py file so that it is compatible with Python 3. If you would like to write all these changes to the file, just do 2to3 -w estimators.py.

abinezer
  • 1
  • 1