1

Background: I'm working on an adversarial detector method which requires to access the outputs from each hidden layer. I loaded a pretrained VGG16 from torchvision.models.

To access the output from each hidden layer, I put it into a sequential model:

vgg16 = models.vgg16(pretrained=True)

vgg16_seq = nn.Sequential(*(
    list(list(vgg16.children())[0]) + 
    [nn.AdaptiveAvgPool2d((7, 7)), nn.Flatten()] + 
    list(list(vgg16.children())[2])))

Without nn.Flatten(), the forward method will complaint about dimensions don't match between mat1 and mat2.

I looked into the torchvision VGG implementation, it uses the [feature..., AvgPool, flatten, classifier...] structure. Since AdaptiveAvgPool2d layer and Flatten layer have no parameters, I assume this should work, but I have different outputs.

output1 = vgg16(X_small)
print(output1.size())
output2 = vgg16_seq(X_small)
print(output2.size())
torch.equal(output1, output2)

Problem: They are in the same dimension but different outputs.

torch.Size([32, 1000])
torch.Size([32, 1000])
False

I tested the outputs right after the AdaptiveAvgPool2d layer, the outputs are equal:

output1 = nn.Sequential(*list(vgg16.children())[:2])(X_small)
print(output1.size())
output2 = nn.Sequential(*list(vgg16_seq)[:32])(X_small)
print(output2.size())
torch.equal(output1, output2)

torch.Size([32, 512, 7, 7])
torch.Size([32, 512, 7, 7])
True

Can someone point out what went wrong? Thank you

FisNaN
  • 2,517
  • 2
  • 24
  • 39

1 Answers1

1

You need to call the eval mode before doing inference.

i.e.

vgg16.eval()
vgg16_seq.eval()
kHarshit
  • 11,362
  • 10
  • 52
  • 71
  • Thank you. It works. Could you explain why? If the mode is wrong, why `nn.Sequential(*list(vgg16.children())[:2])(X_small)` and `nn.Sequential(*list(vgg16_seq)[:32])(X_small)` get the same outputs? – FisNaN Dec 07 '20 at 05:34
  • 1
    I think it's most probably due to dropout (in layer 35 and 38), as by default it's enabled in training mode, but we don't want it in inference (disabled in eval mode). Read more [here](https://stackoverflow.com/questions/60018578/what-does-model-eval-do-in-pytorch). – kHarshit Dec 07 '20 at 06:12