TF - in model_fn pass global_step to seed

Question

In Tensorflow Estimator API (tf.estimator) is there a way to use the current session within model_fn to evaluate a tensor and pass the value to python? I want to have a seed in dropout dependent on the value of global_step, but as the former requires int the latter is a tensor.

Answer 1

I don't see any way to access the session inside model_fn to get the current value of global_step.

Even if it was possible, changing the seed for tf.nn.dropout at each step would create a new Graph operation at each step with a different seed, which will make the graph grow bigger and bigger. Even without a tf.estimator, I don't know how you can implement this?

---

I think what you want is to make sure that you get the same randomness between two runs. Setting a graph-level random seed with tf.set_random_seed() or just using a normal seed in the dropout should create a reproducible sequence of masks. Here is an example with code:

x = tf.ones(10)
y = tf.nn.dropout(x, 0.5, seed=42)

sess1 = tf.Session()
y1 = sess1.run(y)
y2 = sess1.run(y)

sess2 = tf.Session()
y3 = sess2.run(y)
y4 = sess2.run(y)

assert (y1 == y3).all()  # y1 and y3 are the same
assert (y2 == y4).all()  # y2 and y4 are the same

The answer here provides more details on how to make the graph randomness reproducible.