1

As neural networks are a set of linear operators, using categorical variables with an arbitrary assignment of order where there is no justification (e.g., Doctor:1, Teacher:2) will teach the model to treat higher values as more important.

However is this true if you add a non-linear activation function between layers--this will transform the network to be non-linear and as such there would be no need to remove the ordered nature of nominal variables using, for example, non-hot encoder.

tr53
  • 11
  • Do you know about the standard way to encode categories? // I do not see doctor/teacher as having any order to it. – Dave Oct 26 '22 at 20:50
  • @Dave but the ordering of categorical features that do not have any inherent order (such as above) should not matter with non-linear models such as random forests as the predictions are evaluated based on if-statements rather than coefficients. My question, therefore, is if you transform a linear neural network to be non-linear by adding non-linear activation functions such as ReLU, then should I still worry about the handling of categorical features? – tr53 Oct 27 '22 at 08:31
  • What order to the categorical features is there? – Dave Oct 27 '22 at 10:38

0 Answers0