2

This is a minimally working/reproducible example:

import torch
import torch.nn as nn
from torchsummary import summary

class Network(nn.Module): 
    def __init__(self, channels_img, features_d, num_classes, img_size): 
        super(Network, self).__init__()
        self.img_size = img_size
        self.disc = nn.Conv2d(
            in_channels = channels_img + 1, 
            out_channels = features_d, 
            kernel_size = (4,4)
        )

        # ConditionalGan: 
        self.embed = nn.Embedding(
            num_embeddings = num_classes, 
            embedding_dim = img_size * img_size
        )

   def forward(self, x, labels): 
        embedding = self.embed(labels).view(labels.shape[0], 1, self.img_size, self.img_size)
        x = torch.cat([x, embedding], dim = 1)
        return self.disc(x) 
    
# device: 
device = torch.device("cpu")

# hyperparameter: 
batch_size = 64

# Initialize model: 
model = Network(
    channels_img = 1, 
    features_d = 16, 
    num_classes = 10, 
    img_size = 28).to(device) 

# Print model summary: 
summary(
    model, 
    input_size = [(1, 28, 28), (1, 28, 28)], # MNIST
    batch_size = batch_size
)

The error message I get is (for the line with summary(...)):

Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)

I saw in this post, that .to(torch.int64) is supposed to help, but I honestly don't know where to write it.

Thank you!

Hulio Almedo
  • 67
  • 6
  • I think the problem is about `labels`, so cast it to `long` here: `self.embed(labels.long())...`. This should clear this particular error. – Mustafa Aydın Mar 27 '21 at 18:08
  • 1
    Please feel free to make this an answer, and I will mark it as accepted as soon as I see it! It turns out, you pointed exactly to the right point, I had experimented with `x.to(torch.int64)` at the right place.. What works for me now is `self.embed(labels.to(torch.int64))`. :) – Hulio Almedo Mar 27 '21 at 19:59
  • *If* you might also provide an answer on WHY one has to write `self.embed(labels.to(torch.int64))` instead of `self.embed(labels)`, I would really appreciate that. – Hulio Almedo Mar 27 '21 at 20:02
  • Tried to give an answer, good luck with your work! – Mustafa Aydın Mar 27 '21 at 20:33

1 Answers1

0

The problem lies here:

self.embed(labels)...

An embedding layer is kind of a mapping between discrete indices and continuous values, as stated here. That is, its inputs should be integers and it will give you back floats. In your case, for example, you are embedding class labels of the MNIST which range from 0 to 9, to a contiuum (for some reason that I don't know as i'm not familiar with GANs :)). But in short, that embedding layer will give a transformation of 10 -> 784 for you and those 10 numbers should be integers, PyTorch says.

A fancy name for an integer type is "long", so you need to make sure the data type of what goes into self.embed is of that type. There are some ways to do that:

self.embed(labels.long())

or

self.embed(labels.to(torch.long))

or

self.embed(labels.to(torch.int64))

Long datatype is really an 64 bit integer (you may see here), so all these work.

Mustafa Aydın
  • 17,645
  • 4
  • 15
  • 38