I have met a snippet of code of tensorflow 2.0, which is used for calculating the loss. The total loss is composed of two parts: 1) regularization loss, 2) prediction loss. My question is why model.losses is regularization loss? model here is an instance of tf.keras.Model. I'm kind of confused by the tensorflow official API documentation. tf.keras.Model, it says
Losses which are associated with this Layer.
Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a
tf.GradientTapewill propagate gradients back to the corresponding variables.
Why could we get regularization loss via accessing losses property? Also, what is eager safe? If losses property is returning regularization loss, why is it named losses instead of regularization_loss?
with tf.GradientTape() as tape:
outputs = model(images, training=True)
regularization_loss = tf.reduce_sum(model.losses)
pred_loss = ...
total_loss = pred_loss + regularization_loss