I am looking at using Interior Point method for optimizing a convex function. The convex function is basically the log-likelihood of a binary logistic regression model. Can I use this technique?
In generally, is there anything that prevents applying a constrained optimization technique to an unconstrained problem? From what I think, an unconstrained problem is just a constrained problem without the constraints and thus should be solvable using these techniques.
As far as I'm concerned, constrained optimization is a less-than-optimal way of avoiding strong fluctuations in your parameters for the independents due to a bad model-specification. Pretty often a constraint is "needed" when the variance-covariance matrix is ill-structured, when there is a lot of (unaccounted) correlation between independents, when you have aliasing or near-aliasing in datasets, when you gave the model too many degrees of freedom, and so on. Basically, every condition that inflates the variance on the parameter estimates will cause an unconstrained method to behave poorly.
You can look at constrained optimization, but I reckon you should first take a look at your model if you believe constrained optimization is necessary. This for two reasons :
So depending on the goal of the analysis, constrained optimization can be a sub-optimal solution (purely estimating the parameters) or inappropriate (when inference is needed).
On a side note, penalized methods (in this case penalized likelihoods) are specifically designed for these cases, and introduce the bias in a controlled manner where it is accounted for (mostly). Using these, there is no need to go into constrained methods, as the classic optimization algorithms will do a pretty good job. And with the correct penalization, inference is still valid in many cases. So I'd rather go for such a method instead of putting arbitrary constraints that are not backed up with an inferential framework.
My 2 cents, YMMV.
As far as I know, there is no reason to stop you from applying constrained optimization to an unconstrainted problem. However, this may not be a great idea in terms of computational complexity and convergence. For example, fitting a logistic regression model can done efficiently with the Newton-Raphson approach (or the Fisher scoring variant). I am not sure if there is much to gain with the interior point approach in this particular case.
The general sense in optimization is that if you have a convex function and no constraints, you want to use the "powerful stuff", gradient descent, Newton, etc. Without constraints interior point methods are not very good (competitive).
In particular for the problem you're studying (binary logistic regression) you should consider trying simple stochastic gradient descent.
Nothing really stops you from applying constrained optimization techniques to unconstrained problems. The same way nothing stops from pushing (instead of riding) your car to work. But you should definitely try interior point methods w/o constraints and convince yourself about it.
Finally you mention that you want to try linear programming-based methods presumably without constraints, what you plan to do in this case I don't quite understand.
