3

I am trying to use Spark ALS to do recommendation with implicit feedback. However, I found there are two totally different kinds of settings available:

The first one is the setting used by the original paper: Collaborative Filtering for Implicit Feedback Datasets. They use $\lambda={\sim}150$ and $\alpha={\sim}40$

The other is the setting suggested by Sean Owen in this discussion, which uses $\lambda=1$ and $\alpha=40$.

Through I don't very clearly understand the underlying mechanism of implicit feedback recommendation. I have to admit that the recommendations made from Owen's setting is much better than those from the original setting.

Can someone explain what makes this difference?

GeoMatt22
  • 12,950
Fred Pym
  • 245
  • 3
  • 7

1 Answers1

4

The term $\lambda$ is used for regularization here and $\alpha$ is a constant controlling the rate of increase of the confidence matrix $C_{ui}$. The confidence matrix is your confidence in observing a particular preference a user, $u$, has on an item, $i$.

Remember that regularization is to help reduce overfitting by penalizing $X_u$ and $Y_i$ when minimizing the overall loss function.

guy
  • 573
  • Is there a guideline for dummy that if the value of lambda and alpha is smaller or larger, the more the accuracy increase? – tom10271 Aug 02 '18 at 03:02
  • @tom10271 There is no clear cut values to start with as you always have to tune these problems to your particular dataset. Having said that, starting with values similar to the paper is a good place to start. – guy Aug 02 '18 at 12:50