Keras Model Network 3D Input to 2D Output

Question

I have a 3D Input (Samples, Steps, Features). So each sample has a chain of steps that have different features. Now, I want a 2D Output (Samples, Steps) where I have samples and at each step of the sample a 0 or a 1 calculated from the model.

So I think it is a sequential binary classification problem. I have some difficult to define the model, especially the output layer.

Here are the shapes of numpy arrays:

x_train.shape 
# (200, 1657, 669)

x_test.shape
# (41, 1657, 669)

y_train.shape
# (200, 1657)

y_test.shape
# (41, 1657)

I tried this model but the output was not the one I was expecting

n_residues, n_features, n_outputs = trainX.shape[1], trainX.shape[2], trainy.shape[1]
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_residues,n_features)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.1))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(n_outputs, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(trainX, trainy, epochs=epochs, batch_size=batch_size, verbose=verbose)

m_classes = model.predict_classes(x_test, verbose=0)
print(m_classes)
[  36   36   59   32   16   32   36  804 1047   16   16   36   32   36
   36   36   16   16   16   16   16   16   36   16   36   36   36   16
   59   36   36   36   16   16   16  804   16   16   16   36   36]

The output is a 41 long vector for the 41 samples in the test set with classes 0 -1657 I assume.

My desired output for the test set would be 41 binary vectors that are 1657 long.

Thanks!

what was your output? can you edit the question with current output the model is predicting and the expected output? — Madhi, Apr 22 '19 at 07:58
Why is this not the model you are expecting? Please elaborate. — rayryeng, Apr 22 '19 at 08:01
Correct me if I understood your question wrong. You test set has a total of 41 records. It's a classification model range from 0 to 1657. You are expecting an output of binary models with a vector of length 1657. ie.., if the model is predicting 16 then the output should print like this [0,0,.....,1,0,...0] at array index 15th position it should print as 1 all other should print 0. — Madhi, Apr 22 '19 at 08:13
Close! Indeed my test set has 41 records and each record has 1657 steps. Each step has 669 features. The model should recognize when a step is significant and when not from the features. So it could also print multiple 1s like [0,0,1,1,0,...,0,1]. — tantan88, Apr 22 '19 at 08:19

score 2 · Answer 1 · answered Apr 22 '19 at 08:51

When you are dealing with Conv1D, RNN or sequence models the output can be one to one, many to one, (or) many to many. In this case, the model is acting many to one. Generally, In Keras, there is a return_sequence or stateful parameter. If these parameters are False then your model behaves like many to one. (ie.., the output shape is (batch_size, unit_length). In this case, unit length is an output). To make a model many to many then the output should be like (batch_size, time_step, unit_length). Just initialize the stateful==True will help you to solve this problem.

some of the helpful links to understand about sequence output data
Understanding LSTM
Feeding one prediction after another

check the conv1D initialization official documentation and set the parameter stateful=True. This is the Theoretical Idea behind LSTM or Conv1D. This links will help you to get architecture Idea behind LSTM's.

Keras Model Network 3D Input to 2D Output

1 Answers1