7

The Uniform distribution is not differentiable. Is there a differentiable distribution that approximates a Uniform?

foobar
  • 723
  • Uh, wouldn't it be zero? – generic_user Mar 09 '18 at 01:31
  • 2
    You mean that it's not differentiable at the edges? Maybe you could somehow approximate it with a Fourier series... The Fourier transform of a rectangular function is the sinc function. This would give the coefficients of the sin or cos waves that you could use for the approximation. You'd just have to make sure you renormalized the distribution appropriately. – Vivek Subramanian Mar 09 '18 at 01:37
  • 4
    The continuous uniform density has first derivative 0 at all points within the interval that it is non-zero. – Michael R. Chernick Mar 09 '18 at 01:54
  • Thanks for the comments everyone. Yes, I mean it's not differentiable around the edges. I'd like a function that has a closed form first derivative, and is approximately equal to the uniform distribution. That means it doesn't have to be completely flat in the middle--a slight curve is fine-- as long as the general shape of the curve looks like a rectangle. – foobar Mar 09 '18 at 03:40
  • 1
    There are very simple functions with piecewise definitions (typically involving terms of the form $e^{-a(x-b)^2}$ for appropriate values of $a$ and $b$) that are equal to $1$ on $[0,1]$, equal to zero on $]-\infty,-\epsilon]\cup[1+\epsilon,\infty[$, strictly between $0$ and $1$ on $]-\epsilon,0[,\cup,]1,1+\epsilon[$ and infinitely differentiable. These should do what you want, if you rescale them a bit to integrate to 1. If you are interested, you may want to ask at Math.SE. Why do you need this? – Stephan Kolassa Mar 09 '18 at 07:02
  • 1
    This is a familiar problem in theoretical mathematics, where it helps the analysis when you don't have to worry about lack of differentiability. The standard solution, sometimes called "mollification," is to convolve the density with a scaled, zero-centered, infinitely differentiable density (often of compact support). By making the scale close to zero you can make the approximation as close as you like. – whuber Mar 09 '18 at 14:39
  • Wow! @StephanKolassa this is what I'm looking for! I'm trying to do MLE for a mixture of rectangles on a 2D image, and I'm defining the probability of coordinates of a point, ie. $$p( (x,y) | rect)$$, to be a sample from 2 Uniform distributions. In order to find the derivative for it, I needed to differentiate the Uniform distribution. It looks like the Bump function is probably what I need. Thanks! – foobar Mar 09 '18 at 14:44
  • 1
    @MichaelChernick The continuous uniform density also has first derivative 0 at all points outside the edges… the density simply has zero probability for all ranges outside the edges as well. – Alexis Mar 09 '18 at 16:01
  • 1
    Most of the time, using a differentiable approximation to a non-differentiable function in order to find extrema doesn't work out. The problem is that if the approximation is any good, the derivatives blow up anyway and you still encounter numerical problems. You're likely better off solving the ML equations using a technique that is designed to handle uniform distributions. – whuber Mar 09 '18 at 16:14
  • Certainly the first derivative is 0 outside the range. The only point about differentiation is that the first derivative doesn't exist at the boundary points. – Michael R. Chernick Mar 09 '18 at 16:24

2 Answers2

12

This is a familiar problem in theoretical mathematics, where it helps the analysis when you don't have to worry about lack of differentiability. The standard solution, sometimes called "mollification," is to convolve the density with a scaled, zero-centered, infinitely differentiable density (often of compact support). By setting the scale close to zero you can make the approximation as close as you like.

Figure of a sequence of mollified uniform distributions

The figure is a sequence of graphs of mollified Uniform$(0,1)$ density functions using a Gaussian mollifier with standard deviations $1/4$ (green), $1/10$ (gold), $1/25$ (red), and $0$ (blue: the original Uniform PDF).

It is easy to show (use integration by parts in the formula for a convolution) that when the mollifier is infinitely differentiable everywhere (aka "smooth"), so is the mollified function.

The existence of such families of mollifiers means that for most purposes you don't really have to consider non-differentiable densities (or even singular distributions, which by definition do not have a density everywhere) when thinking about properties of distributions. Singular distributions might indeed be "edge cases" but they can be approached as limits of distributions with smooth densities.

This method is particularly congenial in statistical applications because many properties of the mollified distribution are easily computed. As an example, since the variance of the mollifier is proportional to the square of its scale, if we pick a standard mollifier with unit variance (as in this example), the variance of the mollified Uniform distribution equals the variance of the Uniform distribution (here, $1/12$) plus the square of the scale. Thus, you know immediately that mollification with a Gaussian of standard deviation $1/25$ (as in the red curve) will add only $1/(25)^2$ to the variance of the Uniform distribution. You can select the standard deviation to be so small that the change in variance it induces is negligible for your purposes.

whuber
  • 322,774
8

The standard continuous uniform distribution $\text{U}(a,b)$ distribution has a continuous CDF that is differentiable (in the regular sense) at all points except the edges of its support, $x = a, b$. Since probability theory defines density functions using Radon-Nikodym derivatives, we can still ascribe values to the density function even at these end-points.

In view of the use of Radon-Nikosym derivatives in probability, I cannot think of any context where this lack of (regular) differentiability of the CDF would really matter. Nevertheless, if you really want to approximate the uniform distribution with a distribution having a fully differentiable distribution function (in the regular sense), you could approximate the density with a mixture distribution (e.g., a mixture of evenly spaced normal distributions).

Ben
  • 124,856
  • One can connect your approach with mine by noting that the mixture you propose is a mollification of a discrete approximation to the uniform. Comparing the mollification to the discretization and comparing the discretization to the original gives a two-epsilon demonstration of the correctness of this approach. But why go to that trouble when you can cut out the middleman and just mollify the original distribution directly? – whuber Mar 09 '18 at 16:27
  • Yeah, agreed - your method is better. ;) – Ben Mar 09 '18 at 22:12