0

I tried to save the output feature maps in the “forward” function.

the function is

 def forward(self, x):
        x = self.forward_features(x)
        x = self.head(x)

        return x

what i tried is

 y=x[-1].cpu().numpy()
        with open('res.pkl', 'wb') as df_file:
             pickle.dump(y, file = df_file)

but got tensor for one image , how can i got for all images?

Does what i made right ?

Tensor like that

tensor([[0.8981, 0.4641, 0.5740,  ..., 0.4593, 0.0317, 0.2112],
        [0.2477, 0.8282, 0.1286,  ..., 0.9769, 0.5973, 0.4928],
        [0.1353, 0.3044, 0.8599,  ..., 0.9786, 0.2175, 0.0837],
        ...,
        [0.1227, 0.1812, 0.0287,  ..., 0.3544, 0.9123, 0.4494],
        [0.9301, 0.2900, 0.1742,  ..., 0.0343, 0.6522, 0.9424],
        [0.4368, 0.0536, 0.2494,  ..., 0.5699, 0.3439, 0.6585]])
User
  • 23
  • 6
  • what is the shape of your input tensors? you can check this by printing `x.shape`. Btw, consider using `torch.save` and `torch.load` instead of pickle. – Shir Jun 13 '21 at 16:33
  • please see update post . So what i did is right for saving the output feature map ? excuse me can you write it i mean "torch.save and torch.load instead of pickle." – User Jun 13 '21 at 16:38
  • you printed the tensor itself, I ask that you print its shape `print(x.shape)`. It is important to understand which dimensions represent the batch, channels, height, and width in your example. The PyTorch convention is to hold inputs & outputs in 4-dimensional tensors of shape BxCxHXW. – Shir Jun 13 '21 at 16:42
  • Regarding the other part, you should use the method `torch.save(x, 'file_name.pth')` to save any pytorch object, and `x = torch.load('file_name.pth')` to load it later. It wraps the pickle model but is more adequate for Pytorch objects. See [Pytorch Documentations](https://pytorch.org/docs/stable/generated/torch.save.html#torch.save) – Shir Jun 13 '21 at 16:44
  • i will try it but it will take much time to finish .. but why dimensions are important or what can i got from it ? – User Jun 13 '21 at 16:44
  • i got AttributeError: 'list' object has no attribute 'shape' – User Jun 13 '21 at 16:51
  • Are you iterating through batches of inputs or are you just doing a single forward pass? Can you update the question to provide the full code you are using to save the tensors ? it's difficult to see what's wrong without a bigger context – Alka Jun 13 '21 at 16:55
  • all what i made i wrote here in the post and the question title is in specific for what i want so this my little lines to save the tensor .. exactly which part do you see it is important to share it ? – User Jun 13 '21 at 17:01
  • Well, we can't see where you are doing your forward pass, also why are you taking the last element of x `x[-1].cpu().numpy()`, what is even x, and where did it come from, ... ? – Alka Jun 13 '21 at 17:04
  • ok the concept of what i'm doing is get the features of the images for image captioning . when i searched how can i make i got that i need to transform the x[-1] to numpy matrix, and then save it as a pkl file. if it is wrong please correct .. thanks – User Jun 13 '21 at 17:13
  • As I said we can't guess what you are trying to do concretely without seeing the full code where you are doing you the forward pass, where exactly you are computing the features, whether you are doing multiple forward passes of just single one ... Without further information what I can suggest is maybe you can create an empty list `list_features = []` and each time you compute new features you append the results in the list like `list_features.append(x[-1].cpu().numpy())`. Then when you finish features extraction for all images you can save pickle the final list. – Alka Jun 13 '21 at 17:23
  • i will try Thanks – User Jun 13 '21 at 17:44
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/233723/discussion-between-user-and-alka). – User Jun 13 '21 at 17:51

1 Answers1

0

If you wish to save the "output feature maps" (i.e. the outputs of the penultimate layer of your network, before these are combined to give the final predictions in the 'head'), you will need to export these in the forward like so:

 def forward(self, x):
        feats = self.forward_features(x)
        output = self.head(feats)

        return output, feats

### ...

my_model = My_Model()
my_model.eval()
_, features = my_model(inputs)
iacob
  • 20,084
  • 6
  • 92
  • 119
  • Thanks a lot for replying . Does these code will give me the intermediate features ? – User Jun 14 '21 at 14:43
  • 1
    @User it gives you the final output (`head`) and the 'intermediate features (`forward_features)`. – iacob Jun 14 '21 at 14:45
  • I tried it but got a different tensor for the same image every time I run the code .. how can I solve it ? – User Sep 27 '21 at 10:27
  • @User have you set your model to [`eval` mode](https://stackoverflow.com/questions/60018578/what-does-model-eval-do-in-pytorch/66843176#66843176)? – iacob Sep 27 '21 at 10:41
  • yes I did :-(.. – User Sep 27 '21 at 10:52
  • @User post a new question with the code for your model and example input – iacob Sep 27 '21 at 11:01