What does data.norm() < 1000 do in PyTorch?

Question

I am following the PyTorch tutorial here. It says that

x = torch.randn(3, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
    y = y * 2

print(y)

Out:    
tensor([-590.4467,   97.6760,  921.0221])

Could someone explain what data.norm() does here? When I change .randn to .ones its output is tensor([ 1024., 1024., 1024.]).

Answer 1

It's simply the L2 norm (a.k.a Euclidean norm) of the tensor. Below is a reproducible illustration:

In [15]: x = torch.randn(3, requires_grad=True)

In [16]: y = x * 2

In [17]: y.data
Out[17]: tensor([-1.2510, -0.6302,  1.2898])

In [18]: y.data.norm()
Out[18]: tensor(1.9041)

# computing the norm using elementary operations
In [19]: torch.sqrt(torch.sum(torch.pow(y, 2)))
Out[19]: tensor(1.9041)

Explanation: First, it takes a square of every element in the input tensor x, then it sums them together, and finally it takes a square root of the resulting sum. All in all, these operations compute the so-called L2 or Euclidean norm