1

  1. The dataset which was extracted from the database consists of more than 50 columns, I call these columns dimensions, can I call them dimensions?

  2. Obviously, I have to do dimension reduction on them. But since PCA like algorithms often do axis rotating to generate some new axises. I don't think I will PCA algorithm in dimensions reduction. So I calculated the correlations between these columns(parameters), and filtered these who has a high value and some other rules. So can I still call it dimensions reduction? Since I only did some parameters filtering

  3. The reason I don't use PCA like algorithms is because I want to implement Neural Network classification, and I need the real parameters.

Please comment on these, anything even criticizing is welcomed.

cinqS
  • 367
  • 1
  • 2
  • 13

3 Answers3

0

The approach you use to do dimensions reduction is agnostic to the method you use for classification. You can use PCA to preprocess your data before to train any type of classifier, including artificial neural networks if that's what you want to use.

tony
  • 581
  • 2
  • 5
0

If your data consists of observations in rows and each column is a variable for that observation, then you can call these columns dimensions. These can also be called features of the observation.

Dropping columns/features from your dataset is essentially dimensionality reduction. You go from a higher dimensional space to a lower dimensional one (less columns). The fact that you're not combining features, rotating spaces doesn't mean it's not a valid form of dimensionality reduction.

Why are you against using PCA for Neural Networks in particular? PCA doesn't care about what kind of classification you're running after it. In fact there is a convenience function in R's caret package implement PCA before Neural nets to ease computation of the NN.

Zephyr
  • 997
  • 4
  • 10
  • 20
niczky12
  • 111
  • 3
  • Hi, If I do PCA before training the machine, so the input of the NN model would be some combined features(maybe). So if I have a single input, observation, how should I map its features to the combined features so that I can feed the observation to the model? Is there a mapping mechanism for PCA for single observation, and this mapping mechanism should be the same with training. thanks, any knowledge would be appreciated. – cinqS Mar 22 '17 at 03:38
  • PCA does a linear combination of your feature set. I.e. if you have 2 features per observation x1 and x2, there will be linear combination, let's say z1 = x1 * a1 + x2 * b1 and z2 = x1 * a2 + x2 * b2 to generate 2 new z features. Then you will pick the top few features, let's say only z1to reduce dimensionality. This means that you will be able to translate z1, z2 back into x1 and x2 if required. Some further reading on PCA might be useful: http://www4.ncsu.edu/~slrace/LinearAlgebra2016/Slides/EigenspacesPrint.pdf – niczky12 Mar 22 '17 at 08:45
0

The dataset which was extracted from the database consists of more than 50 columns

You can call them Features. Feature Set is something that you operate on.

Obviously, I have to do dimension reduction on them. But since PCA like algorithms often does axis rotating to generate some new axises. I don't think I will PCA algorithm in dimensions reduction.

I am not sure if you understand PCA completely. PCA does not generate new random axis, but an axis, that represents the MOST variance of your data. So, PCA will surely perform dimension reduction given that data has some values which don't add info to your model. You can call this Feature Extraction or Feature Filtering

The reason I don't use PCA like algorithms is because I want to implement Neural Network classification, and I need the real parameters.

What makes you think, post PCA you don't have 'real' parameters. It's just a different view of the same data.(Imagine these axes as eyes, and your data as the object, you just have a different view of the same object.)

The only problem I see is the axes on which the components are projected are orthogonal, which if you want to avoid, you can go ahead with ICA (Independent Component Analysis)

desertnaut
  • 1,988
  • 2
  • 14
  • 23
Ronak Agrawal
  • 206
  • 3
  • 11
  • I understood that it's just a different view of the same data. So if it's another view, and I used these views to train the machine. But when I use the trained machine to predict the input from the users, how do I map this input to this another view? can I map this input to another view, if yes, could you plz give me some example to read on? thanks – cinqS Mar 22 '17 at 03:32
  • @cinqS: When you are processing the test data, you should be using transform(X_test), so that the test data undergoes the same transformation as the training data. Then just call the 'predict' method. Refer :http://stackoverflow.com/a/21331792/2128723 – Ronak Agrawal Apr 12 '17 at 10:51