4

Let $S\subseteq \mathbb{R}^n$ be a convex region defined by $$g_i(x)\leq 0, ~~i\in 1,\ldots,m,$$ where $g_i$ are convex functions. The goal is to decide whether $S$ is empty, and if not - find a point $x\in S$.

In the book of Boyd and Vandenberghe (chapter 11), I found the following algorithm. Solve the following optimization problem:

$$\text{ mininize } ~~p~~ \text{ subject to } ~~g_i(x)\leq p, ~~i\in 1,\ldots,m.$$

Denote the optimal solution by $(p^*,x^*)$. If $p^*>0$, then $S$ is empty. If $p^*\leq 0$, then $x^*$ is a point in $S$ (in particular, if $p^* < 0$ then $x^*$ is in the interior of $S$; if $p^*=0$ then interior of $S$ is empty and $x^*$ is a boundary point).

The problem is that, when $p^*$ approaches 0, the runtime complexity of solving this auxiliary problem approaches infinity; here is the exact runtime from the book of Boyd and Vandenberghe:

enter image description here

As you can see, $p^*$ is in the denominator. They also say that:

enter image description here

I would like to know what is known about the complexity of this decision problem. In particular:

  • Suppose the functions $g_i$ are given explicitly (e.g. they are all polynomials with rational coefficients; this also means that they are continuously differentiable infinitely many times), and every basic arithmetic operation on real numbers takes unit time. Is there a proof that the decision problem cannot be solved in a finite number of arithmetic operations?
  • Are there sub-classes of convex functions (except linear functions) for which the decision problem can be solved in finite time? In polynomial time?

EDIT: Based on the comments, it seems the question is non-trivial even in the case of quadratic programming, where all functions $g_i$ are convex quadratic functions. The Wikipedia page mentions that the problem can be solved in weakly-polynomial time using the ellipsoid method, but to use the ellipsoid method we need a lower bound on the volume of the feasible region, which is not always guaranteed.

Erel Segal-Halevi
  • 2,212
  • 12
  • 20
  • What do you know about the $g_i$? This would be easier if the inequalities stemmed from, say, a linear matrix inequality – Rodrigo de Azevedo Dec 15 '23 at 08:40
  • There's semi-definite programming... – Neal Young Dec 15 '23 at 16:40
  • More generally, if you can compute the gradients (derivatives) accurately, then you have a separation oracle, so you can use the ellipsoid method.. with the caveat that you need (roughly) an initial containing ellipsoid and a lower bound on the volume of your convex set if it is not empty. – Neal Young Dec 15 '23 at 16:58
  • See https://en.wikipedia.org/wiki/Quadratic_programming#Complexity and https://en.wikipedia.org/wiki/Convex_optimization and https://or.stackexchange.com/questions/989/cubic-programming-and-beyond . – Neal Young Dec 16 '23 at 16:55
  • @NealYoung "with the caveat that you need... a lower bound on the volume of your convex set if it is not empty" --- this caveat is quite similar to the caveat with the interior-point method I cited from the book of Boyd and Vandenberghe, which works only if the optimal value of the auxiliary problem is sufficiently far from zero. But what happens if we cannot guarantee a lower bound on the volume of the convex set? Or cannot guarantee that the optimal value is sufficiently far from zero? Is there a hardness result for the general case? – Erel Segal-Halevi Dec 16 '23 at 17:49
  • @RodrigodeAzevedo If the $g_i$ are linear, then it is known that the problem can be solved in polynomial time. I am interested in the more general case. For example, suppose the $g_i$ are polynomials of a finite degree, with rational coefficients that are given as an input. This also means that they are continuously differentiable infinitely many times. – Erel Segal-Halevi Dec 16 '23 at 17:54
  • @ErelSegal-Halevi I was thinking along the lines of Helton & Nie – Rodrigo de Azevedo Dec 16 '23 at 18:19
  • 1
    @NealYoung I wrote in the question: every basic arithmetic operation on real numbers takes unit time (i.e., the real RAM model) – Erel Segal-Halevi Dec 30 '23 at 18:00
  • In that case, have a look at https://cstheory.stackexchange.com/questions/4053/sum-of-square-roots-hard-problems and https://cstheory.stackexchange.com/questions/79/problems-between-p-and-npc/4010#4010 . There are decision problems that can be solved in linear time in the real RAM model (assuming you can do square roots), for which no known polytime algorithm is known (in the standard model). So one has to be careful to specify what a hardness result would mean in this model. – Neal Young Jan 04 '24 at 20:59
  • See also the closely-related questions https://cstheory.stackexchange.com/questions/6833/what-classes-of-mathematical-programs-can-be-solved-exactly-or-approximately-in and https://cstheory.stackexchange.com/questions/44324/is-convex-optimisation-in-p/44333#44333 – Neal Young Jan 04 '24 at 21:24
  • Related: this article shows that it is NP-hard to decide whether a feasible region defined by quartic polynomials is convex. [Sorry for so many comments.. I think this question is very interesting!] – Neal Young Jan 15 '24 at 15:56

1 Answers1

4

Warning: As one of the comments points out, the sum of squares is not necessarily convex, so the hardness reduction suggested below does not work. The problem still lies in $\exists\mathbb{R} \subseteq \mathrm{PSPACE}$ though.

Deciding whether there is an $x \in \mathbb{R}^n$ such that $f_i(x) = 0$ for a family of quadratic polynomials is complete for the existential theory of the reals, $\exists\mathbb{R}$. So testing whether $f(x) = \sum_i (f_i(x))^2 \leq 0$ has a solution is also complete for $\exists\mathbb{R}$, and, as a sum of squares, $f$ is a convex function. So solving the non-emptiness problem of a convex set is as hard as deciding truth in the existential theory of the reals.

The hardness result goes back to Blum, Shub, and Smale. On a Theory of Complexity over the Real Numbers, it follows from the main theorem, but it's stated in the BSS-model. A statement in the $\exists\mathbb{R}$-framework can be found as Lemma 3.2 in Schaefer, Stefankovic. Fixed Points, Nash Equilibria, and the Existential Theory of the Reals.

user67422
  • 144
  • 2
  • Does it mean that it is in PSPACE? So it can be solved in exponential time? By what algorithm? – Erel Segal-Halevi Dec 14 '23 at 21:51
  • 1
    A simple Google search gives this useful description on Wikipedia. https://en.wikipedia.org/wiki/Existential_theory_of_the_reals – Chandra Chekuri Dec 15 '23 at 06:02
  • 1
    @ErelSegal-Halevi The original algorithm is due to Canny, Some algebraic and geometric computations in PSPACE, https://doi.org/10.1145/62212.62257 . A follow-up improvement: https://doi.org/10.1093/comjnl/36.5.409 – Emil Jeřábek Dec 15 '23 at 07:46
  • 1
    Wait a minute, the sum of squares is not necessarily convex in $x$. Eg. consider $[(1-x)(1+x)]^2$ (here's a plot – Neal Young Dec 15 '23 at 16:38
  • 1
    When you say "the problem still lies in PSPACE", which problem do you mean? It seems like you mean the problem you discuss in your answer. But the problem in OP's post doesn't (in general) reduce to the problem you discuss in your answer, so I don't think you can conclude that OP's problem is in PSPACE. – Neal Young Dec 16 '23 at 14:34
  • 1
    @NealYoung The case of “polynomials with rational coefficients” in the OP’s question is solvable in $\exists\mathbb R\subseteq \mathrm{PSPACE}$. – Emil Jeřábek Dec 16 '23 at 19:37
  • @EmilJeřábek,perhaps, but I don't think the reduction described in the answer shows that (except in the easy case when the given polynomials are linear). – Neal Young Dec 17 '23 at 13:36
  • 3
    For this direction, the trivial reduction works: $\exists\vec x,\bigwedge_ig_i(\vec x)\le0$ is, literally, an existential statement in the theory of the reals. – Emil Jeřábek Dec 17 '23 at 15:51
  • I am surprised that the problem can be decided in finite time, since the passage I quoted from the book on convex optimization apparently shows that the run-time might be infinite. Or maybe I did not understand it correctly? – Erel Segal-Halevi Dec 17 '23 at 18:01
  • 3
    @ErelSegal-Halevi The quoted passage doesn't focus on polynomials, does it? The existential theory of the reals does, so this might be were the disconnect lies. Also it says "in practice". I don't think that a PSPACE algorithm is very practical, in general. – Tassle Dec 18 '23 at 13:47
  • @EmilJeřábek, for that to work don't you need to assume something about the $g_i$'s (beyond the given assumption that they are convex)? – Neal Young Dec 18 '23 at 14:08
  • 2
    @NealYoung I don’t even need that they are convex. I only need that they are polynomials with rational (or: real algebraic, if represented in a suitable way) coefficients, as I wrote above. – Emil Jeřábek Dec 18 '23 at 14:26
  • @EmilJeřábek, okay, that makes sense. (I was considering OP's question to be the general one, without the assumption that the functions are polynomials.) – Neal Young Dec 19 '23 at 16:27