How to reuse variables in tensorflow 2.0?

Question

When using tensorflow 2.0, I find something weird about tf.Variable? There are two cases bellow.

The first one

x1 = tf.Variable(12., name='x')
x2 = tf.Variable(12., name='x')
print(x1 is x2)
x1.assign(1.)
print(x1)
print(x2)

The output is

False
<tf.Variable 'x:0' shape=() dtype=float32, numpy=1.0>
<tf.Variable 'x:0' shape=() dtype=float32, numpy=12.0>

which means variables with the same name don't share the same memory.

The second one

x = tf.Variable(12., name='x')
print(x)
y = x.assign(5.)
print(y)
print(x is y)

x.assign(3.)
print(x)
print(y)

The output is

<tf.Variable 'x:0' shape=() dtype=float32, numpy=12.0>
<tf.Variable 'UnreadVariable' shape=() dtype=float32, numpy=5.0>
False
<tf.Variable 'x:0' shape=() dtype=float32, numpy=3.0>
<tf.Variable 'UnreadVariable' shape=() dtype=float32, numpy=3.0>

The result is unexpected, variables x and y with different names share the same memory, but id(x) is not equal to id(y).

Therefore, the name of variable can't distinguish whether variables are identical(share the same memory). And how can I reuse variables in tensorflow 2.0, like with tf.variable_scope("scope", reuse=True) tf.get_variable(...) in tensorflow 1.0?

Do you have a concrete example on how you would like to reuse variables? — AloneTogether, Feb 05 '22 at 13:00
For example, the variable `x` is defined in the net A, I want to create the net B to reuse `x`. — Gauss, Feb 05 '22 at 13:08
What is stopping you from directly using `A.x` in `B`? Maybe something like this?https://stackoverflow.com/questions/56201185/how-to-find-a-variable-by-name-in-tensorflow2-0 — AloneTogether, Feb 05 '22 at 13:11

NeoZoom.lua · Answer 1 · 2022-07-19T16:30:10.030

Quoted from your question:

The result is unexpected, variables x and y with different names share the same memory, but id(x) is not equal to id(y).

No, this is incorrect. From the docs of tf.Variable.assign, where read_value is default to True:

read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op.

Here "something" should be a new operation, which isn't x, but is evaluated to the value of x.

To reuse x, just access x:

y = x
print(y is x) # True

Finally, regarding:

which means variables with the same name don't share the same memory.

Therefore, the name of variable can't distinguish whether [...]

You have to create different(thus distinguishable) names yourself, regarding your first example. You might want to take a look at the comments of this accepted answer https://stackoverflow.com/a/73024334/5290519

How to reuse variables in tensorflow 2.0?

1 Answers1