8

I've got two inverse problems,

$$A_1 ~ x = b_1 \qquad A_2 ~ x = b_2$$

So far I've been solving them independently using Tikhonov Regularization and getting two estimates for $x$. However in my case $x$ represents the same solution in both equations. Is it possible to do a 'simultaneous' solve? Ideally I would be finding the answer for

$$\min \left( \lVert A_1 x - b_1 \rVert^2 + \lVert A_2 x - b_2 \rVert^2 + \lVert\Gamma x\lVert^2 \right)$$

Where $\Gamma = \alpha ~ I$ and $I$ is the identity matrix as in Tikhonov Regularization (aka ridge regression). I suppose I could just take the average of both solutions, wondering if there is a more statistically powerful way of approaching this however.

Gilles
  • 253
  • 1
  • 2
  • 10
abnowack
  • 255
  • 1
  • 4
  • 1
    What's the relative accuracy of the measurements in $b_{1}$ and $b_{2}$? You may need to scale to adjust for this. Are all the measurements independent? Correlation can complicate things. – Brian Borchers Aug 06 '16 at 21:44
  • Right now it's all modeled so I know $b_1$ and $b_2$ perfectly, but in practice I will know $b_1$ with perhaps 10x more accuracy. However at this step I want to assume I know them both equally and that they are independent. – abnowack Aug 06 '16 at 21:53
  • So what is your question here? It's easy to solve the three term least squares problem you've given in your question. – Brian Borchers Aug 06 '16 at 22:06
  • Is it? If you explain in an answer I'll mark it correct. I just use basic routines like numpy's least square solver . I'm not from a CS background, so I could be missing something obvious. – abnowack Aug 06 '16 at 22:24

1 Answers1

9

You can write your problem as

$\min \| Fm - g \|_{2}^{2}$

where

$F=\left[ \begin{array}{c} A_{1} \\ A_{2} \\ \alpha I \\ \end{array} \right] $

and

$g=\left[ \begin{array}{c} b_{1} \\ b_{2} \\ 0 \\ \end{array} \right]. $

You can use whatever linear least squares solver you want to solve this problem.

Brian Borchers
  • 18,719
  • 1
  • 39
  • 70