4

Suppose $F(z,\Theta)$ is a continuosly differentiable cumulative density function where $z$ is a random variable defined over $[0,\infty)$ and $\Theta$ is a parameter of the distribution. I have a feeling that the follwoing limit is true but how can something like this be proven in a mathematically acceptable way? (this may not be an easy question and even a reference is appreciated)

$$\lim_{n \rightarrow \infty}\frac{1}{n} \sum_{i=1}^n \frac{\frac{\partial F(z_i, \Theta)}{\partial \Theta }-\frac{\partial F(z_{i-1}, \Theta)}{\partial \Theta } }{{(z_i-z_{i-1})}}\rightarrow \ E\big(\frac{\partial f(z, \Theta)}{\partial \Theta }\big) $$

where $z_i$ is evaluated at $F^{-1}(i/n,\Theta)$ , $i=1,...,n, E, f$ and $F^{-1}$ denote expectation, probability density function (derivative of $F$) and quantile functions.

user41838
  • 531
  • 2
  • 7
  • I believe you need to normalize the LHS by dividing by $n$. Otherwise, the summands each ought to approximate $\partial^2F(z_i,\theta)/{\partial z\partial \theta}$ and the sum will diverge. With the factor of $1/n$ included, this looks like a direct application of the Lebesgue-Stieltjes Mean Value Theorem. – whuber Apr 16 '15 at 16:07
  • Yes, it needed a 1/n which is now corrected. My guess is that it requires more than a simple application of Lebesgue-Stieltjes Mean Value Theorem but I need to think more. – user41838 Apr 17 '15 at 03:05

1 Answers1

1

The equality is true provided an additional assumption is made.

Let $z_0 = 0$ (as implied, but not stated, in the question). Then, applying the Mean Value Theorem for Lebesgue-Stieltjes integrals of the form $\int g(z) dF(z)$, the right hand side can be analyzed as

$$\eqalign{ \mathbb{E}\left(\frac{\partial f(z,\theta)}{\partial \theta}\right) &= \int_0^\infty\frac{\partial f(z,\theta)}{\partial \theta} dF(z,\theta) \\ &= \sum_{i=1}^n \int_{z_{i-1}}^{z_i}\frac{\partial f(z,\theta)}{\partial \theta} dF(z,\theta) \\ &= \sum_{i=1}^n \left(F(z_i, \theta) - F(z_{i-1}, \theta)\right)\frac{\partial}{\partial\theta}f(z_i^{*},\theta) \\ &= \sum_{i=1}^n \left(\frac{i}{n} - \frac{i-1}{n}\right)\frac{\partial}{\partial\theta}f(z_i^{*},\theta) \\ &= \frac{1}{n}\sum_{i=1}^n \frac{\partial}{\partial\theta}f(z_i^{*},\theta) }$$

where each $z_i^{*} \in [z_{i-1}, z_{i}]$.

Now let's examine the left hand side, once again invoking the MVT (and assuming continuity of $F$, so that mixed partial derivatives can be reordered):

$$\eqalign{ &\frac{1}{z_i - z_{i-1}} \left(\frac{\partial}{\partial\theta}F(z_i,\theta) -\frac{\partial}{\partial\theta}F(z_{i-1},\theta)\right) \\ &= \frac{\partial}{\partial z}\frac{\partial}{\partial\theta} F(z_{i}^{**}, \theta) = \frac{\partial}{\partial \theta}\frac{\partial}{\partial z} F(z_{i}^{**}, \theta) \\ &= \frac{\partial}{\partial \theta}f(z_{i}^{**}, \theta). }$$

where each $z_i^{**} \in [z_{i-1}, z_{i}]$.

Thus the difference between the left and right sides equals

$$\frac{1}{n}\sum_{i=1}^n \left(\frac{\partial}{\partial \theta}f(z_{i}^{**}, \theta) - \frac{\partial}{\partial \theta}f(z_{i}^{*}, \theta)\right).$$

What would it take for its limit to be zero as $n\to \infty$? One simple (but not necessary) criterion is that for each $\theta$, $\frac{\partial}{\partial\theta}f(z,\theta)$ should be of bounded variation, because--as is immediate from the definition--the magnitude of the difference does not exceed $1/n$ times the total variation of $\frac{\partial}{\partial\theta}f(z,\theta)$, whence its limit would be $0$ (QED).


Indeed, this analysis suggests counterexamples to the original statement. All we need do is construct a family of distributions where these derivatives of the density (with respect to the parameter $\theta$) grow unbounded on the domain $z\in(0,\infty)$. One such family, for $\theta\in (0,1)$, is

$$f(z, \theta) = z^{-\theta} - 1, 0 \le z \le 1; \ f(z,\theta) = 0 \text{ otherwise}.$$

$F(z,\theta)$ is continuously differentiable in $z$ for $z \in (0,\infty)$. If it is desired to make $F$ differentiable at $z=0$, too, then a more complicated counterexample would need to be constructed--but the idea should be clear.

Figure

Plots of $f(z,\theta)$ for $\theta\in \{2/3, 1/2, 1/3\}$ are overlaid in this graphic.

An immediate problem crops up: $\partial F(z_0, \theta) / \partial \theta$ is not even defined. We cannot hope for the left hand side even to make sense, much less to be equated to the right hand side--which is finite (equal to $\frac{1}{(1-2 \theta )^2}-\frac{1}{(\theta -1)^2}$) when $0 \lt \theta \lt 1/2$.

whuber
  • 322,774
  • 1
    Thanks wuber, this is indeed a good answer to my question. I worked out another way to prove this using Taylor expansion that required continuous differentiablity of "∂f(z,θ)/∂θ". I think bounded variation is weaker and implied by continuous differentiablity. – user41838 Apr 18 '15 at 11:59