1

I have the following code for estimating a generalized extreme value distribution from scipy.

from scipy.stats import genextreme
ys = [22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 22.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.3, 23.2, 23.2, 23.2, 23.2, 23.2, 23.2, 23.2, 23.2, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 29.9, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 27.7, 24.5, 24.5, 24.5, 24.5, 24.5, 24.5, 24.5, 24.5, 24.5, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 26.6, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 23.7, 23.7, 23.7, 23.7, 23.7, 23.7, 23.7, 23.7, 23.7, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.9, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764, 22.058823529411764]
shape, loc, scale = genextreme.fit(ys)
mean, var = genextreme.stats(shape, loc, scale, moments='mv')

I got the following fitted parameters (shape, location and scale respecitvely):

-2.787020488783334
22.058823529411782
5.0707584099150134e-14

Thus, the shape is negative but the documentation on https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.genextreme.html allows the shape to go negative.

However, my mean and variance are both nan.

It looks like I can fit a model, and the fitted parameters look reasonable, but why am I unable to get a mean from the fitted distribution?

SmallChess
  • 7,211
  • 2
    Your numbers are constant. There's no way you can estimate anything other than a constant distribution. – whuber Oct 29 '22 at 20:40
  • 1
    As noted by others, why are there so few unique observations? There also seems to be clusters of identical data points. – JimB Oct 30 '22 at 04:19
  • 1
    @whuber The values are not constant -- there is variation -- but they are constant for substantial intervals. See the index plot: https://i.stack.imgur.com/Hk3PG.png ... this raises many questions. – Glen_b Oct 30 '22 at 05:46

1 Answers1

3

In the Wikipedia parameterization, the density of a generalized extreme value distribution (GEVD) function having shape $\xi$ is

$$f(s; \xi)=\Bigl(1+\xi s\Bigr)^{-(1+1/\xi)} \exp\Bigl(-(1+\xi s)^{-1/\xi}\Bigr), $$

where $s=(x-\mu)/\sigma$, with location $\mu$ and scale $\sigma$. Comparison against the scipy docs formula indicates that your shape $c=-\xi$.

The nan mean (and thus variance) in your fit is correct, insofar as you have a correct fit. The mean of the GEVD is infinite if $\xi \ge 1$, or $c \le -1$ in your parameterization. That's what you have, with c = -2.79.

But you have a lousy fit, given the unusual form of your data for a continuous distribution: only 10 unique values for 459 observations. In R:

length(ys)
# [1] 459
length(unique(ys))
# [1] 10

Note that the scale $\sigma$ in your output is very close to 0, which should by itself lead to questions about the fit. I could pretty much reproduce your result with maximum-likelihood estimation via the fevd() function in the R extRemespackage, which follows the Wikipedia parameterization:

fitGEVD <- fevd(ys,method="MLE")
fitGEVD
# some output omitted 
#  Estimated parameters:
#     location        scale        shape 
# 2.205882e+01 2.938005e-11 3.454806e+00

Using L-moments as the fitting method gave something more reasonable:

fitGEVD_L <- fevd(ys,method="Lmoments")
fitGEVD_L
# fevd(x = ys, method = "Lmoments")
# [1] "GEV  Fitted to  ys  using L-moments estimation."
#   location      scale      shape 
# 22.5507850  1.1224576  0.4184415

Comparison of empirical density versus fitted:

observed versus fitted GEVD for MLE and L-moments

The shape $\xi$ is less than 1/2 in the second model (the one that comes close to matching the data), as required for a finite mean and variance.

EdM
  • 92,183
  • 10
  • 92
  • 267
  • 1
    +1. But having only unique values suggests such a fit might be meaningless. – whuber Oct 29 '22 at 23:42
  • 1
    @whuber I agree completely. But I hadn't previously appreciated the cutoffs in $\xi$ that determine whether there's a finite mean or variance, which answers the question of why there's no mean or variance for the GEVD predicted by the MLE fit and would be interesting regardless of the strange data set. – EdM Oct 30 '22 at 00:42