Concatenate Two Tensors in Pytorch

Question

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 2. Got 32 and 71 in dimension 0 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:87

I have a tensor of shape [71 32 1].

I want to make it of shape [100 32 1] by padding zero vectors.

I tried by concatenating a padding vector of zeros of shape [29 32 1]. I get the error above.

I try with a padding vector of zeros of shape [29 32 1], I still get an error.

How could I create the required tensor?

Answer 1

In order to help you better, you need to post the code that caused the error, without it we are just guessing here...

Guessing from the error message you got:

1.

Sizes of tensors must match except in dimension 2

pytorch tries to concat along the 2nd dimension, whereas you try to concat along the first.

2.

Got 32 and 71 in dimension 0

It seems like the dimensions of the tensor you want to concat are not as you expect, you have one with size (72, ...) while the other is (32, ...).
You need to check this as well.

Working code

Here's an example of concat

import torch

x = torch.rand((71, 32, 1))
# x.shape = torch.Size([71, 32, 1])
px = torch.cat((torch.zeros(29, 32, 1, dtype=x.dtype, device=x.device), x), dim=0)
# px.shape = torch.Size([100, 32, 1])

Alternatively, you can use functional.pad:

from torch.nn import functional as F

px = F.pad(x, (0, 0, 0, 0, 29, 0))