Most Popular
1500 questions
36
votes
2 answers
How are Bayesian Priors Decided in Real Life?
I always had the following question: How are Bayesian Priors decided in real life?
I created the following scenario to pose my question: Suppose you are researcher and you are interested in studying if the age of a giraffe can be predicted by the…
stats_noob
- 1
- 3
- 32
- 105
36
votes
1 answer
Multiple comparisons on a mixed effects model
I am trying to analyse some data using a mixed effect model. The data I collected represent the weight of some young animals of different genotype over time.
I am using the approach proposed…
nico
- 4,581
- 3
- 32
- 45
36
votes
4 answers
How is finding the centroid different from finding the mean?
When performing hierarchical clustering, one can use many metrics to measure the distance between clusters. Two such metrics imply calculation of the centroids and means of data points in the clusters.
What is the difference between the mean and the…
John Hoffman
- 513
36
votes
1 answer
What concepts/objects are "wrongly" formed in probability and statistics?
Some background: There is a wonderful mathematical article which argues that mathematicians have been wrong to frame mathematical formulae in terms of the constant $\pi$, and that they should have framed these things in terms of $2 \pi$ (the…
Ben
- 124,856
36
votes
1 answer
How exactly is the "effectiveness" in the Moderna and Pfizer vaccine trials estimated?
As in the title. Is this "a risk ratio"? How is it calculated, if you could provide an example with numbers for both trials, please? I am not a statistician, but I am familiar with the binomial distribution - I suppose it is used here to calculate…
36
votes
3 answers
I know the 95% confidence interval for ln(x), do I also know the 95% confidence interval of x?
Suppose the 95% confidence interval for $\ln(x)$ is $[l,u]$. Is it true that the 95% CI for $x$ is simply $[e^l, e^u]$?
I have the intuition the answer is yes, because $\ln$ is a continuous function. Is there some theorem that supports/refutes my…
Tamay
- 495
36
votes
2 answers
How to statistically compare the performance of machine learning classifiers?
Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing sample randomly from the base set, train the…
entropy
- 1,242
36
votes
5 answers
Extrapolation v. Interpolation
What is the difference between extrapolation and interpolation, and what is the most precise way of using these terms?
For example, I have seen a statement in a paper using interpolation as:
"The procedure interpolates the shape of the estimated…
Frank Swanton
- 641
36
votes
5 answers
What if my linear regression data contains several co-mingled linear relationships?
Let's say I am studying how daffodils respond to various soil conditions. I have collected data on the pH of the soil versus the mature height of the daffodil. I'm expecting a linear relationship, so I go about running a linear…
SlowMagic
- 623
36
votes
7 answers
What's the point of time series analysis?
What is the point of time series analysis?
There are plenty of other statistical methods, such as regression and machine learning, that have obvious use cases: regression can provide information on the relationship between two variables, while…
Dhalsim
- 401
36
votes
2 answers
Performing a statistical test after visualizing data - data dredging?
I'll propose this question by means of an example.
Suppose I have a data set, such as the boston housing price data set, in which I have continuous and categorical variables. Here, we have a "quality" variable, from 1 to 10, and the sale price. I…
Marcel
- 1,370
36
votes
1 answer
explain meaning and purpose of L2 normalization
Let me say at the outset that I am very new to machine learning, and not great at math. I understand what TF-IDF does, but in the book I am reading it also notes the following (it's discussing how scikit-learn does things):
Both classes…
Stephen
- 1,076
36
votes
2 answers
What are variational autoencoders and to what learning tasks are they used?
As per this and this answer, autoencoders seem to be a technique that uses neural networks for dimension reduction. I would like to additionally know what is a variational autoencoder (its main differences/benefits over a "traditional" autoencoders)…
user188529
36
votes
4 answers
How does batch size affect convergence of SGD and why?
I've seen similar conclusion from many discussions, that as the minibatch size gets larger the convergence of SGD actually gets harder/worse, for example this paper and this answer. Also I've heard of people using tricks like small learning rates or…
dontloo
- 16,356
36
votes
3 answers
Feature importance with dummy variables
I am trying to understand how I can get the feature importance of a categorical variable that has been broken down into dummy variables. I am using scikit-learn which doesn't handle categorical variables for you the way R or h2o do.
If I break a…
Dan
- 1,437