How to generate more than 1 output per input in LSTM?

Question

Assume this is my model:

_________________________________________________________________ 
Layer (type)                 Output Shape              Param #    ================================================================= 
embedding_16 (Embedding)     (None, 10, 500)          71500      _________________________________________________________________ 
lstm_31 (LSTM)               (None, 10, 500)          2002000    _________________________________________________________________ 
dropout_15 (Dropout)         (None, 10, 500)          0          _________________________________________________________________ 
time_distributed_16          (None, 10, 500)          250500    _________________________________________________________________ 
softmax (Activation)         (None, 10, 500)           0     =================================================================

But I want to have in my last layer:

softmax (Activation)         (None, 100, 1000)           0

I have been trying to do this for hours. I don't know if this is possible or not. I don't think you can change output size of LSTM (looking at its model) but is there a layer that i can add so that it generates , say, 10 ouputs per input?

In simple words, assume I want to my model to generate 10 words for each word i put in. I hope I am able to explain.

score 1 · Accepted Answer · answered Aug 07 '18 at 21:05

There are different ways of looking at the "multiple output" here (and by "here" I take a guess that you are using keras library - it seems so from the printout).

In a simple case, having e.g. Dense(10) layer would solve it. The "secret sauce" in using TimeDistributed layer wrapper as explained in this SO post.

The other approach requires using functional API of keras. How to get multiple out is explained in the docs.

How to generate more than 1 output per input in LSTM?

1 Answers1