2

I am relatively new to Gaussian Processes and Bayesian Optimization. My question is very simple:

Suppose I am trying to learn a function from a parametric family of curves which best describes the observed data. For eg. I could be trying to learn a polynomial function $f(x) = ax^2 + bx +c$ and assume the observation model is $y_i \sim \mathcal{N}(f(x_i), \epsilon^2)$.

Also assume that I specify prior beliefs on the parameters $a, b, c, \epsilon$. Also assume that $0 < x <= 100$. Also suppose $x \in$ $\mathbb{N}$, so that there are only $100$ choices for $x$

I can observe data at maximum of $40$ points, ie, $x_1, \ldots x_{40} \in (0, 100]$

The question then becomes: how should I sequentially go about spacing $x_i$ in a well defined manner so that I am most confident about my choice of the learnt $f(x)$.

Whatever I found so far deals with Gaussian Process which optimizes sampling assuming a blackbox function. But here, I am looking to learn a specified parametric function.

I am not looking for an answer, but I am seeking guidance on where to look. What are the keywords that describe my problem? Because whenever I look up Bayesian Optimization, it always has Gaussian Process.

chesslad
  • 31
  • 2
    Based on what you've written, I don't think BO is what you want to use to solve your problem. "Bayesian Optimization" refers to using Bayesian methods to solve complicated optimization problems, often the problems themselves are not directly related to any statistical inference. Here, you have an optimization problem induced by a Bayesian inference problem. The relevant search term is not "Bayesian Optimziation", but "Maximum A Posteriori Inference" (MAP Inference). – John Madden Jan 10 '24 at 17:04
  • 1
    Are you sure that you need BO and GPs? This sounds like an OLS regression problem where you want to know how to choose $x_1, \cdots, x_{40}$ so that some property of the estimated function $\hat f$ is optimal. Can you be specific about what it means to be "most confident about my choice fo the learnt $f(x)$"? BO is used to sequentially select points to optimize some criterion, but it's not clear if the criteria you're optimizing fits in this sequential setting. It seems like off the bat you could designate 40 points according to your criteria, gather the $y$s, and then apply OLS. – Sycorax Jan 10 '24 at 17:27
  • @Sycorax I had ELPD LOO score in mind when I said "most confident about my choice for the learnt ()". – chesslad Jan 10 '24 at 18:19
  • @Sycorax sequential sampling is indeed required in my case. The algorithm would be run online with a max budget of 40 inferences. Let's say we always start at 1 and 100. So, we have $f(1)$ and $f(100)$. The question is, where to sample next $x$ based on maximizing some criterion (like ELPD Loo or some other metric that minimizes the overall posterior predictive width of the curve) – chesslad Jan 10 '24 at 18:23
  • Possible duplicate of https://stats.stackexchange.com/questions/636516/bayesian-optimization-for-solving-least-squares – AdamO Jan 10 '24 at 19:17

0 Answers0