6

Let N be the number of examples, M be the number of features. We are trying to create a linear regression to fit the training samples.

  1. When N == M, do we have perfect 0 training loss?
  2. When N > M, do we have unique or many solutions?
  3. When N < M, do we have unique or many solutions?

Attempt:

  1. Depends, but if the instance is not colinear, then yes
  2. Yes, unique solution.
  3. Many solution.
wrek
  • 185

1 Answers1

1

To address point 1, some sources might tell you that you an achieve perfect training error by having as many features as variables. A typical example given is to use a high-order polynomial.

However, this need not result in perfect training error! An example would be if two points have the same feature values but different outcomes. Unless you find new variables (not just functions of your existing variables), such data cannot ever be fit perfectly to any function.

Dave
  • 62,186