2

I want to predict condominium prices with a neural network. Some of my variables are numeric but are not assumed to relate to the price in form of a mathematical funktion (linear, square, ...).

For example the floor or the number of flats in one building. You can not say, which floor is the best or that the more flats in one building, the lower the price in general. Should these variables be cut into categorical variables (Dummies, e.g.: 1-5 flats; 6-10 flats; 10-...)? Or will the network find a way to detect the differences in a numeric variable, even though the higher the value the higher is the product of the variable and its weights.

I hope you can get my point, since in linear regression it is very important to cut those variables if they are not related in a linear way.

Scrabyard

sebp
  • 2,097
Scrabyard
  • 31

3 Answers3

1

If you convert the numerical features into categorial and then mistakenly use one-hot encoding, you lose the ordinality embedded in the feature. If you just call the transformed features as $1,2,3,...$ to preserve the ordinal structure, it just becomes a nonlinear transformation of the initial feature. This is naturally neural net's job to find out, however you might need a larger (more complex) network to do it and operate with care. If you're confident with split points, then of course it's better to engineer those features and feed into your neural net to save it from trying to find what you already know. But, when you're unsure where to split, you can easily lose information. You may also want to try tree-based algorithms which can capture these kinds of relationships easier.

gunes
  • 57,205
0

I don‘t think that it is possible to tell ex ante which representation of your data will work best. I think you need to try/test different ways of representing the data. In general, NN are very good in capturing non-linear or convolutional patterns in data. However, there are a number of limitations, e.g. that you need „a lot of data“ to fit NN successfully and you need to find a proper model including hyperparameter tuning.

I suggest you try two cases, a representation with continuous values and one with one-hot (aka dummy) representation. Don‘t forget to scale data. I would also come up with a linear „baseline model“ to which one can compare results from NN. The reason is that NN sometimes does not perform very well (or better) but NN can be very expensive in terms of time needed for tuning and fitting. You could look into GAM models with regression splines, theses model types are usually able to capture non-linear patterns very well.

Here are two minimal examples for GAM in R:

https://github.com/Bixi81/R-ml/blob/master/GAM_regression_splines.R

https://github.com/Bixi81/R-ml/blob/master/GAM_regression_splines_simulated_data.R

Peter
  • 256
0

in linear regression it is very important to cut those variables if they are not related in a linear way

This is not true in general. As you can learn from the What is the benefit of breaking up a continuous predictor variable? thread, binning loses information and is not recommended strategy for creating features. If you need to model non-linear relationships, you can use transformations that are continuous like polynomials. Binning is a rather primitive way of creating features, definitely not the default approach.

As for neural networks, the point of using them is that they can learn features and model non-linear relationships by themselves, so creating such features is not useful. That said, there are cases where feature engineering may be useful as you can learn from the Utility of feature-engineering : Why create new features based on existing features? thread.

Tim
  • 138,066