I am experimenting with stochastic gradient descent and observing very strange output.
In a toy problem, the $X\beta$ for stochastic gradient descent is always larger than $0$, which will be predicting $1$ all the time. But the shape is similar to glm.
What could be wrong?
My code:
library(sgd)
set.seed(0)
d=ggplot2::diamonds
d$price=ifelse(d$price>2500,1,0)
noise_idx=sample(nrow(d),nrow(d)/5)
d$price[noise_idx]= ifelse(d$price[noise_idx]==1,0,1)
glm.fit=glm(price~.-carat,d,family=binomial)
sgd.fit <- sgd(price ~ .-carat, data=d,
model="glm", model.control=binomial(link="logit"))
glm.p=model.matrix(price~.-carat,d) %*% glm.fit$coefficients
sgd.p=model.matrix(price~.-carat,d) %*% sgd.fit$coefficients
par(mfrow=c(1,2))
hist(glm.p,30)
hist(sgd.p,30)
