When talking about artificial neurons, inputs, weights and biases, I understand the role of each but the latter.
In short if we have a neuron such as sigmoid(sum(w*x) + b) I get that the weights basically say which of the inputs is more "important", but what about the bias? I've read it in this other question "is as means how far off our predictions are from real values."
But how can this be true, if we start them "at random"? Also this isn`t supposed be be the job of the loss/cost function?
Bias simply takes care of variables which are unaccounted for i.e you did not include it in X. All Machine Learning and Neural Nets do is approximate a function. All functions can be represented as a series of exponentials using Fourier Transform which basically means a function can be expressed as a sum of many other special functions.
In this series there is a constant term to adjust the elevation of the curve above the x-axis. This constant can be thought of as the bias. Thus it basically takes care of the fact that when all variables are 0, a function need not be 0.
Bias simply means how much the output does not depend on the inputs. It is exactly equivalent to intercept term which means by dropping all the inputs what the outputs will be. We usually set bias terms to zeros. You have cost function and it is the error rate based on the weights and biases. Consequently, you try to change these parameters to reduce the error.
