8

The terminology "inference" and "prediction" seems to have different usage across numerous sources and at my current work.

In some places, it seems "inference" refers to the training step, i.e., inferring the properties of a model (e.g., beta coefficients in least squares) from the input (training) dataset, and then "prediction" refers to using the inferred properties/parameters of the model to predict the output for unseen inputs.

At my current job where I primarily work with deep learning, these two terms seem to be used synonymously and both refer to the prediction step, so I'm a bit confused.

Are there actually formal definitions for these?

David
  • 705
  • In statistics, inference includes things like parameter estimation (point and interval), hypothesis testing and a few other activities (many people would include clustering and classification in inference for example). See https://en.wikipedia.org/wiki/Statistical_inference . The definition offered there is from a dictionary of statistics; it's about as good as you're likely to do. Check also the references. In short you're using data to infer some property of a population or data generating process, from a sample via a distributional model (parametric/ otherwise) that aims to describe it – Glen_b Nov 27 '23 at 00:23
  • 2
    The standard meanings of "inference" and "prediction" in the statistical literature are explained at https://stats.stackexchange.com/questions/16493. That there exists such a well-known convention (and has for a long time) is of course no guarantee that some other community will use the terms in the same way! – whuber Nov 27 '23 at 15:17

3 Answers3

9

Sure. But that doesn't solve the problem. As with words in general, meanings change and are different in different fields. And there are different formal definitions. For instance, the Oxford Dictionary of Statistics ed. by Upton and Cook, defines "inference" as

The process of deducing properties of the underlying distribution by analysis of data.

It does not have an entry for "prediction" but, for "prediction model" it has

In a medical context, a model used to predict an outcome or the probability of an outcome....

That seems to fit how the term is used in some other fields, as well.

Britannica has a similar definition for "inference":

inference, in statistics, the process of drawing conclusions about a parameter one is seeking to measure or estimate.

And, of course, there are definitions right here on CrossValidated, although they may not meet your idea of "formal".

For myself, I would say that prediction is about some future, possibly hypothetical, state of things, while inference is about a population from which you have a sample. If I take a sample mean and then take confidence intervals and so on, I am not necessarily predicting anything, but I am inferring from the sample to a population.

But you may not agree with my (informal) definition.

EDIT: After reading Richard's comment, I agree with his distinctions.

Peter Flom
  • 119,535
  • 36
  • 175
  • 383
  • 2
    My feeling is that it is forecasting that refers to future state of things, as the term is typically used in a time-series context. Meanwhile, prediction could just as well apply to additional elements from a cross section where there is no dimension of time. (On the other hand, we predict something we have not seen yet even if that thing comes from the past, so from our reference point the revelation of that value is in the future. In that sense I can see where your definition might be originating from.) – Richard Hardy Nov 27 '23 at 07:08
  • 1
    @RichardHardy I like that distinction between prediction and forecasting. – Peter Flom Nov 27 '23 at 12:17
  • +1 and optionally, to merge the comment into the answer (albeit by implying that all future states are inherently hypothetical), one could modify the sentence "I would say that prediction is about some future, possibly hypothetical, state of things" to become "prediction is about some hypothetical, possibly future, state of things" – michael Dec 07 '23 at 07:42
8

I am quoting Petruccelli, Nandram, Chen Applied Statistics for Engineers and Scientists.

Statistical inference is the use of a subset of a population, called a sample to draw conclusions about the entire population from which it was taken. So, for example, a poll of 1,500 voters (the sample) might be used to estimate the preferences of all voters (the population), or 32 sets of five ball bearings each (the sample) are taken from a production line every 15 minutes during an eight hour shift in order to monitor the entire production of bearings turned out during that shift (the population).

Three popular types of statistical inference are:

  • Estimation of model parameters uses the data to estimate model parameters and to tell us how much uncertainty is contained in the estimates we make and
  • Prediction of a future observation uses presently-available data to predict the value of a new observation from a population.
  • A Tolerance interval is a range of values which has a user-specified probability of containing a user-specified proportion of the population.
Richard Hardy
  • 67,272
Tran Khanh
  • 710
  • 3
    +1 This quotation carefully uses "future" in the sense of an observation not included in the data or considered in its modeling. Others mistakenly take "future" to mean literally in times to come, but that leads to all kinds of misconceptions about what statistical prediction is -- some of which can be found elsewhere in this thread. – whuber Nov 28 '23 at 14:53
6

At first glance, this is a question about terminology. But after a moment, I realized that perhaps it reflects deeper assumptions.

In a strictly Bayesian context, there isn't much difference between inference and prediction. For inference, we might mean conditioning what we know about an unknown with some observations. E.g., parameter inference; inference of a parameter, $\theta$, given data, $x$, through the posterior $$ p(\theta | x) \propto p(x | \theta) \, p(\theta) $$ For prediction, we might mean more specifically conditioning what we know about future observations on some observations. E.g., inference of some future data $x^\star$ given data, $x$, through the posterior predictive $$ p(x^* | x) = \int p(x^* | \theta, x) \, p(\theta | x) \, d\theta $$ You'll notice that this is still inference; were still conditioning on $x$. In other approaches, prediction isn't done in the same way as inference. In these cases, it may be helpful to use language more carefully.

innisfree
  • 1,480