How to optimize a fractional function $\frac{\left|u^HXu\right|^2}{\left|u^HYu\right|^2}$ with $\left\|u\right\|_2 = 1$

Question

How to optimize a fractional function $\frac{|u^H X u|^2}{\left|u^HYu\right|^2}$ with $\left\|u\right\|_2 = 1$? Here, the matrix $X \in C^{n\times n}$ is positive semidefinite, and $Y\in C^{n\times n}$ are positive definite, i.e., $X \succeq 0, Y \succ 0$. Is there any closed form expression for $u$?

Answer 1

Write $f_A(u) = u^\prime A u$ for any $n\times n$ matrix $A$ and $n$-vector $u$. It suffices to restrict attention to symmetric (real) matrices, because $f_A$ and $f_B$ with $B=(A + A^\prime)/2$ are identical forms, allowing us to replace $A$ by its symmetric version $B=B^\prime$.

Because $f_X$, $f_Y$, and $u\to ||u||^2$ are all homogeneous quadratic forms, the problem is equivalent to optimizing

$$\frac{f_X(u)/||u||^2}{f_Y(u)/||u||^2} = \frac{f_X(u)}{f_Y(u)}$$

with no constraints. Do this by setting $f_Y(u)=1$ and maximizing $f_X(u)$, which can be accomplished by introducing a Lagrange multiplier $\lambda$. Taking derivatives with respect to $u$ shows we need to solve the system of equations

$$uX - \lambda uY = 0.\tag{1}$$

Because $Y$ is definite, it is invertible, permitting us to right-multiply both sides of $(1)$ by $Y^{-1}$, giving

$$u(XY^{-1}) - \lambda u = 0.$$

This is the (left) eigenvalue equation for $XY^{-1}$. Thus, the extrema are the extreme eigenvalues of $XY^{-1}$ and they are attained when $u$ is a corresponding eigenvector, scaled to unit length.

---

There won't generally be a closed form expression for such eigenvectors, but there is extensive software to compute them and the theory and interpretation of eigenvectors is very well established.