1

Why doesn't torch.autograd compute the gradient in this case?

import torch
x = torch.tensor([1., 2., ], requires_grad=True)
y = torch.tensor([x[0] + x[1], x[1] - x[0], ], requires_grad=True)
z = y[0] + y[1]
z.backward()
x.grad

Output is a blank line (None). The same occurs for x[0].grad. Why?

PS: In retrospect, I realize the motivation for making y a tensor with requires_grad was so I could examine its own gradient. I learned that one must use retain_grad for that here: Why does autograd not produce gradient for intermediate variables?

user118967
  • 4,895
  • 5
  • 33
  • 54

1 Answers1

2

When you use torch.tensor for y, it just uses the values of x to initialize the tensor, the gradient chain is lost.

This works:

x = torch.tensor([1., 2., ], requires_grad=True)
y = [x[0] + x[1], x[1] - x[0], ]
z = y[0] + y[1]
z.backward()
x.grad

The result is tensor([0., 2.])

Sergii Dymchenko
  • 6,890
  • 1
  • 21
  • 46
  • 1
    Thanks. In retrospect, I realize the motivation for making y a tensor with requires_grad was so I could examine its own gradient. I learned that one must use retain_grad for that here: https://stackoverflow.com/questions/45988168/why-does-autograd-not-produce-gradient-for-intermediate-variables – user118967 Apr 14 '20 at 22:00