Most Popular
1500 questions
1343
votes
27 answers
Making sense of principal component analysis, eigenvectors & eigenvalues
In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues.
I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like a machine. But I didn't understand it. I didn't…
claws
- 13,715
816
votes
11 answers
How to choose the number of hidden layers and nodes in a feedforward neural network?
Is there a standard and accepted method for selecting the number of layers, and the number of nodes in each layer, in a feed-forward neural network? I'm interested in automated ways of building neural networks.
Rob Hyndman
- 56,782
669
votes
12 answers
What is the difference between "likelihood" and "probability"?
The wikipedia page claims that likelihood and probability are distinct concepts.
In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a clear distinction in perspective: the number that…
Douglas S. Stones
- 7,561
606
votes
5 answers
Relationship between SVD and PCA. How to use SVD to perform PCA?
Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix $\mathbf X$. How does it work? What is the…
amoeba
- 104,745
570
votes
15 answers
What is the intuition behind beta distribution?
Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial for other people here. So I would be very thankful…
ffriend
- 9,990
563
votes
11 answers
What is the difference between test set and validation set?
I found this confusing when I use the neural network toolbox in Matlab.
It divided the raw data set into three parts:
training set
validation set
test set
I notice in many training or learning algorithm, the data is often divided into 2 parts, the…
xiaohan2012
- 7,179
560
votes
23 answers
Why square the difference instead of taking the absolute value in standard deviation?
In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute value of the difference instead and get the expected…
c4il
- 5,855
495
votes
20 answers
The Two Cultures: statistics vs. machine learning?
Last year, I read a blog post from Brendan O'Connor entitled "Statistics vs. Machine Learning, fight!" that discussed some of the differences between the two fields. Andrew Gelman responded favorably to this:
Simon Blomberg:
From R's fortunes
…
Shane
- 12,461
444
votes
5 answers
How to understand the drawbacks of K-means
K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just apply this algorithm which minimizes the sum of…
KevinKim
- 6,899
434
votes
14 answers
Bayesian and frequentist reasoning in plain English
How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
Daniel Vassallo
- 4,449
431
votes
9 answers
What is the difference between fixed effect, random effect in mixed effect models?
In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect in mixed effect models?
Andrew
- 6,198
416
votes
11 answers
Explaining to laypeople why bootstrapping works
I recently used bootstrapping to estimate confidence intervals for a project. Someone who doesn't know much about statistics recently asked me to explain why bootstrapping works, i.e., why is it that resampling the same sample over and over gives…
Alan H.
- 5,169
408
votes
7 answers
When conducting multiple regression, when should you center your predictor variables & when should you standardize them?
In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing by the standard deviation.) In which other cases…
mathieu_r
- 4,491
407
votes
17 answers
What happens if the explanatory and response variables are sorted independently before regression?
Suppose we have data set $(X_i,Y_i)$ with $n$ points. We want to perform a linear regression, but first we sort the $X_i$ values and the $Y_i$ values independently of each other, forming data set $(X_i,Y_j)$. Is there any meaningful interpretation…
arbitrary user
- 3,751
403
votes
7 answers
How to normalize data to 0-1 range?
I am lost in normalizing, could anyone guide me please.
I have a minimum and maximum values, say -23.89 and 7.54990767, respectively.
If I get a value of 5.6878 how can I scale this value on a scale of 0 to 1.
Angelo
- 4,485