1

I have a set of data in histogram format with uneven bin sizes, which represents the weight of horses at a certain point in their lifetimes when they are switched from grazing to a racing diet. Here is a data sample:

Weight - Headcount

0-600lb: 340,000

600-699lb: 365,000

700-799lb: 494,000

800-899lb: 430,000

900-999lb: 110000

1000-3000lb: 40,000

I know that the majority of the 0-600lb category will be towards the heavier end, and the opposite would be true for the 1000-3000lb category, so I'm looking for a decreasing distribution with a peak around the middle. Additionally, this may be a combination of two distributions, as it's possible male and female horses have their diets switched at different times. Then again, maybe not so if a solution without considering this factor would still be fantastic!

How can I try a series of distributions to see which best fits my data in python?

Hiraphor
  • 11
  • A general procedure is explained and illustrated at https://stats.stackexchange.com/questions/34882. A specific application (to a lognormal distribution) is given at https://stats.stackexchange.com/questions/56015. – whuber May 11 '23 at 15:14
  • 1
    Do you have access to the unbinned data? If so that's what should be analyzed. I don't understand the purpose of binning here. – Frank Harrell Sep 16 '23 at 11:09

1 Answers1

0

I would assume that this data would follow a normal distribution, so that is where I would start.

When the bin width is even, you can use the bin center as the x value and the bin height as the y. In your case, since the bins are uneven, you should use the bin integral of the objective function to compare to your data. For example the code below:

import scipy.optimize
import scipy.stats
import numpy as np
import matplotlib.pyplot as plt

bins = [0, 600, 700, 800, 900, 1000, 3000]
binc = [ 300, 650, 750, 850, 950, 2000]
weights = [340000, 365000, 494000, 430000, 110000, 40000]


def fGaussianCDF(bins, *params):


N = params[0]
mu = params[1]
sigma = params[2]

binwidth = np.diff(bins)

return N*(scipy.stats.norm.cdf(bins[1:], mu, sigma) - scipy.stats.norm.cdf(bins[:-1], mu, sigma) )





fig, ax = plt.subplots(1, 1)
ax.plot(binc, weights, "ok")
ax.set_xlabel("Weight (lbs.)", fontsize=16)
ax.set_ylabel("Counts", fontsize=16)


popt, _ = scipy.optimize.curve_fit(fGaussianCDF, bins, weights, p0=[1.8e6, 730, 150])
plt.plot(binc, fGaussianCDF(bins, *popt), "rx")
print(popt)


plt.show()

Which gives the best fit result of a mean value of mu=736 lb and sigma=146. The results plotted look like:

enter image description here

Which is not a perfect fit, but hopefully is something that you are looking for.

  • Thanks for the answer! As an add-on, I was wondering if there's any way you know of to fix the errors, ie treat it as normally distributed to approximate a continuous function, but since I know the bins are absolutely true, any error in the approximation should be dealt with in some manner so the numbers exactly line up – Hiraphor Jun 18 '21 at 22:16