I have a dataset with classes [a, b] where during training I have made sure that the dataset is equally balanced. I have trained the network using cross-entropy loss with equal importance.
I am able to achieve an accuracy of around 90% for both classes a and b. However in reality I would prefer to network to have much higher accuracy for class a with an allowed decrease of accuracy for class b.
I have managed to somewhat solve this in the following way:
- Split data into train/validtion/test data.
- Train network on train data and end at maximum validation accuracy.
- As a post proccess step. Change so that the binary output of the network is not taken as argmax of the output but instead a modified argmax requiring an output value larger than 0.5 for b to classify as b instead of a.
- Use test dataset to verify that the result is as expected.
With this I can reach an example result of the following: Before modifying argmax function:
- Train accuracy {a : 0.95, b : 0.95}
- Validation accuracy {a : 0.90, b : 0.90}
- Test accuracy {a : 0.90, b : 0.90}
After modifying argmax function until 0.99 accuracy for a:
- Validation accuracy {a : 0.99, b : 0.80}
- Test accuracy {a : 0.98, b : 0.78}
This is fine for my application but is there a more scientific and "correct" way to achieve this result? I am using pytorch to train the model.
