1

I am working on a custom Recurrent layer in Keras, and I need to reset a specific weight after each sequence.

For example:

  • The entry has shape (nb_sequences, sequence_size, value_size),
  • My network has 2 sets of weights, say self.A and self.B
  • self.A is trainable, self.B is not

The output is computed with both self.A and self.B

I would like my model to have a clean reseted self.B at the begining of each sequence while still training self.A like all model.

In this model, self.A acts as the controler, and self.B acts like a readable/writable memory. So during the sequence, self.A will write and read in self.B. But I want the memory to be empty at the begining of each sequence

I saw that you can reset a whole model with save_weights and load_weights like presented in This Question, and I think I can adapt this to reset a specific weight in a layer, but the hard point is to do this at the end of each sequence.

This Keras Documentation explains how to do specific actions at the begining/end of each Train, Epoch or Batch but I can't find how to do things at the begining of each sequence...

I also thought of using the states variables sent at each step to reset self.B at the begining of each sequence but I can't figure how to use this...

Any ideas ?

Pantoofle
  • 336
  • 1
  • 13
  • I don't know how to do this in keras, but usually when you do not want to train some weights, you specify to the optimizer to minimize the cost function only with respect to some specific variables, rather than minimize with respect to all of them and later restore the weights you do not want to modify... – Pietro Tortella Jun 16 '17 at 14:27
  • @Gyldhas It is unclear to me why `self.B` would ever change as it is not trainable. Why would you reset it if never changes? – nemo Jun 16 '17 at 15:10
  • @nemo In fact, `self.B` acts like a memory. During the sequence, the network reads and writes from/to `self.B`. So it changes. But I want a clean memory at the begining of each sequence – Pantoofle Jun 16 '17 at 15:56
  • @PietroTortella `self.B` is already noted as `trainable=False` during the build stage of the layer, so Keras should not try to optimize these variables – Pantoofle Jun 16 '17 at 15:57

0 Answers0