3

The Wikipedia page for Partial Least Squares (PLS) gives an algorithm for the method which is uncited and for which I cannot find the source material. It appears to be very much simpler than most if not all other implementations which I have come across.

My questions boil down to:

  1. Is the algorithm as stated valid?
  2. Where does it originate from?
  3. Why is it so much simpler than other implementations?
Matti Wens
  • 452
  • Rather than simpler, it looks more complicated than necessary :) – amoeba May 02 '19 at 19:12
  • If you know of a simpler PLS regression algorithm, I'd love to see it... – Matti Wens May 02 '19 at 21:26
  • Well, in PLS1, given $X$ and $y$, the direction of the first weight vector is simply $X^\top y$, it just needs to be normalized to unit length. Then deflate $X$ to make all columns uncorrelated to the extracted component, and compute the next component in exactly the same way. Repeat. – amoeba May 02 '19 at 21:34
  • But to make your question more answerable, can you cite some source that explains PLS1 algorithm in a way that is familiar to you? I.e. give some example reference in your Q3. – amoeba May 02 '19 at 21:34
  • Aren't you describing PCA there, as opposed to PLS. – Matti Wens May 02 '19 at 21:36
  • @MattWenham The idea is very very similar. – gunakkoc May 03 '19 at 13:49
  • To return to this question, all other implementations I have seen involve either a NIPALS or SVD Eigen decomposition at every iteration. How does the above algorithm do without this decomposition, or else where /how does it perform it? – Matti Wens Jun 12 '19 at 15:05

1 Answers1

1

Further research has thrown up this webpage at purdue.edu which links to source code for various variants of PLS. On the latter page, the PLS1 method appears to be very similar to the algorithm shown on the PLS regression Wikipedia page.

The purdue.edu implementation cites "Overview and Recent Advances in Partial Least Squares" by Roman Rosipal and Nicole Kramer, LNCS, 2006 (PDF here or here) as a source.

Comparing the purdue.edu PLS1 function with the Wikipedia algorithm, it appears that the latter is an optimised version of the former, and probably cannot be optimised / simplified any further.

My original question came about because the Wikipedia algorithm has no 'inner loop', which I now understand comes about as a result of it being an algorithm specifically for a single response variable, which allows further significant simplification.

Rosipal and Kramer repeatedly cite "PLS Regression Methods" by A. Höskuldsson, Journal of Chemometrics vol 2: 211-228 (1988) which is another excellent reference. It states that "if ... Y is a one-dimensional vector ... the NIPALS procedure converges in a single iteration".

I also found this paper very useful.

Matti Wens
  • 452