0

I am building a binary classification where the class I want to predict is present only <2% of times. I am using pytorch

The last layer could be logosftmax or softmax.

self.softmax = nn.Softmax(dim=1) or self.softmax = nn.LogSoftmax(dim=1)

my questions

  1. I should use softmax as it will provide outputs that sum up to 1 and I can check performance for various prob thresholds. is that understanding correct?

  2. if I use softmax then can I use cross_entropy loss? This seems to suggest that it is okay to use

  3. if i use logsoftmax then can I use cross_entropy loss? This seems to suggest that I shouldnt.

  4. if I use softmax then is there any better option than cross_entropy loss?

        ` cross_entropy = nn.CrossEntropyLoss(weight=class_wts)`
user2543622
  • 5,760
  • 25
  • 91
  • 159
  • 1
    You have a severe class imbalance. Cross entropy loss alone might not work for you. See [this](https://stackoverflow.com/a/65766155/1714410) for more details and options. – Shai Mar 03 '22 at 16:22
  • thanks, could you reply questions 1,2,3,4? thanks – user2543622 Mar 03 '22 at 16:25
  • After [ECCV](https://eccv2022.ecva.net/) deadline. However, for _binary_ classification one usually use a single output with `sigmoid` activation and [`nn.BCEWithLogitsLoss`](https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss) and not `softmax` and `CrossEntropy` loss. – Shai Mar 03 '22 at 16:28
  • I understand. just yes or no would suffice for time being – user2543622 Mar 03 '22 at 17:03

0 Answers0