I just found slides from Matt Gormley (CMU) about recommendation systems. Under the heading "Unconstrained Matrix Factorization" he mentions:
What does "User/item bias terms (matrix trick)" mean?
The "user/item bias terms (matrix trick)" refers to a technique for incorporating bias terms into the matrix factorization optimization problem in a computationally efficient way. The matrix trick involves creating two additional matrices that represent the user and item bias terms, and adding them to the original matrix in a way that allows the optimization problem to be solved using standard algorithms.
In simple, the individual biases or preferences of each user and each item in the dataset can be incorporated into the model by using a matrix trick, which is a mathematical technique for representing the biases as a matrix and incorporating them into the factorization process. This allows the model to capture the individual preferences of each user and item, leading to more personalized and accurate recommendations.
For example, let's say we have a matrix $R$ that represents user ratings of items, where $R[i,j]$ is the rating given by user $i$ to item $j$. We can add bias terms for each user and each item by creating two additional matrices, $B_u$ and $B_i$, where $B_u[i,0]$ is the bias for user $i$ and $B_i[j,0]$ is the bias for item $j$. Then, we can add the bias terms to the original matrix $R$ by computing a new matrix $R_{bias}$ as follows:
$R_{bias} = R + B_u + B_i.T$
This new matrix $R_{bias}$ contains the original ratings plus the bias terms for each user and item. By using this matrix in the optimization problem, we can effectively incorporate the user and item bias terms into the matrix factorization model. This allows us to optimize the model using standard algorithms, without the need for specialized solvers.
