0

I study this tutorial https://www.tensorflow.org/tutorials/generative/cyclegan and the completed model is well worked on windows. Then I convert this tf model to mlmodel, but the model's output(MultyArray) has empty shape. How do i solve this problem...? (This model's EPOCH is 1)

computer::: windows10 / tensorflow and -gpu 2.2 / tfcoreml 1.1

this is convert code

import tfcoreml
import coremltools
from tensorflow import keras

saved_model = keras.models.load_model('saved_model')
# get input, output node names for the TF graph from the Keras model
input_name = (saved_model.inputs[0].name.split(':')[0])[0:7]
keras_output_node_name = saved_model.outputs[0].name.split(':')[0]
graph_output_node_name = keras_output_node_name.split('/')[-1]

# Saving the Core ML model to a file.
model = tfcoreml.convert('saved_model',
                         image_input_names=input_name,
                         input_name_shape_dict={input_name: [1, 540, 540, 3]},
                         output_feature_names=[graph_output_node_name],
                         minimum_ios_deployment_target='13',
                         red_bias=-123.68,
                         green_bias=-116.78,
                         blue_bias=-103.94)
model.save('./saved_mlmodel/saved_model.mlmodel')

this is saved_model.summary

Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, None, None,  0                                            
__________________________________________________________________________________________________
sequential_15 (Sequential)      (None, None, None, 6 3072        input_2[0][0]                    
__________________________________________________________________________________________________
sequential_16 (Sequential)      (None, None, None, 1 131328      sequential_15[0][0]              
__________________________________________________________________________________________________
sequential_17 (Sequential)      (None, None, None, 2 524800      sequential_16[0][0]              
__________________________________________________________________________________________________
sequential_18 (Sequential)      (None, None, None, 5 2098176     sequential_17[0][0]              
__________________________________________________________________________________________________
sequential_19 (Sequential)      (None, None, None, 5 4195328     sequential_18[0][0]              
__________________________________________________________________________________________________
sequential_20 (Sequential)      (None, None, None, 5 4195328     sequential_19[0][0]              
__________________________________________________________________________________________________
sequential_21 (Sequential)      (None, None, None, 5 4195328     sequential_20[0][0]              
__________________________________________________________________________________________________
sequential_22 (Sequential)      (None, None, None, 5 4195328     sequential_21[0][0]              
__________________________________________________________________________________________________
sequential_23 (Sequential)      (None, None, None, 5 4195328     sequential_22[0][0]              
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     multiple             0           sequential_23[0][0]              
                                                                 sequential_21[0][0]              
                                                                 sequential_24[0][0]              
                                                                 sequential_20[0][0]              
                                                                 sequential_25[0][0]              
                                                                 sequential_19[0][0]              
                                                                 sequential_26[0][0]              
                                                                 sequential_18[0][0]              
                                                                 sequential_27[0][0]              
                                                                 sequential_17[0][0]              
                                                                 sequential_28[0][0]              
                                                                 sequential_16[0][0]              
                                                                 sequential_29[0][0]              
                                                                 sequential_15[0][0]              
__________________________________________________________________________________________________
sequential_24 (Sequential)      (None, None, None, 5 8389632     concatenate_1[0][0]              
__________________________________________________________________________________________________
sequential_25 (Sequential)      (None, None, None, 5 8389632     concatenate_1[1][0]              
__________________________________________________________________________________________________
sequential_26 (Sequential)      (None, None, None, 5 8389632     concatenate_1[2][0]              
__________________________________________________________________________________________________
sequential_27 (Sequential)      (None, None, None, 2 4194816     concatenate_1[3][0]              
__________________________________________________________________________________________________
sequential_28 (Sequential)      (None, None, None, 1 1048832     concatenate_1[4][0]              
__________________________________________________________________________________________________
sequential_29 (Sequential)      (None, None, None, 6 262272      concatenate_1[5][0]              
__________________________________________________________________________________________________
conv2d_transpose_15 (Conv2DTran (None, None, None, 3 6147        concatenate_1[6][0]              
==================================================================================================
Total params: 54,414,979
Trainable params: 54,414,979
Non-trainable params: 0

this is mlmodel.spec description

input {
  name: "input_2"
  type {
    imageType {
      width: 540
      height: 540
      colorSpace: RGB
    }
  }
}
output {
  name: "Identity"
  type {
    multiArrayType {
      dataType: FLOAT32
    }
  }
}
metadata {
  userDefined {
    key: "coremltoolsVersion"
    value: "3.4"
  }
}
Running
  • 3
  • 1

1 Answers1

0

This is usually not a problem. When you run the model, you still get a multi-array of the correct shape.

If you already know the shape, you can fill it in so that it shows up in the mlmodel file, but this is more for documentation purposes than anything else.

Matthijs Hollemans
  • 7,706
  • 2
  • 16
  • 23
  • Thank you for your help!! I got predicted result on mlmodel, then how to convert it to image..? can i get useful site for this?? by the way Is there a difference between MLMultiarray and Multiarray on coreml?? – Running Jun 12 '20 at 11:11
  • MLMultiArray is the class name used by Core ML, but it's called MultiArray in other places. It's the same thing. There are several answers on how to convert the output to an image here on StackOverflow (plus you can check out my book). The best way is to tell the mlmodel that the output should be an image instead of a MultiArray. – Matthijs Hollemans Jun 12 '20 at 11:52
  • I changed my mlmodel's output description responds to input description. :: width: 540, height: 540, colorSpace: RGB but error occurs on xcode.. Error Domain=com.apple.CoreML Code=1 "Failed to convert output Identity to image" UserInfo={NSLocalizedDescription=Failed to convert output Identity to image, NSUnderlyingError=0x28324a310 {Error Domain=com.apple.CoreML Code=0 "Invalid array shape ( 768, 768, 3 ) for converting to gray image" UserInfo={NSLocalizedDescription=Invalid array shape ( 768, 768, 3 ) for converting to gray image}}} error – Running Jun 12 '20 at 22:45
  • The clue is in the error message: the output is 768x768. – Matthijs Hollemans Jun 13 '20 at 10:06
  • There was an error before the model ended....in: let result = try model.prediction(input_2: buffer(from: inputImage!)!).... buffer func is https://stackoverflow.com/questions/44462087/how-to-convert-a-uiimage-to-a-cvpixelbuffer func buffer..... what's wrong... – Running Jun 15 '20 at 04:08