1

The CA Secretary of State has historical records on ballot measures starting in 1912, and we're attempting to calculate probabilities for the 2024 ballot measures based on the historical average. Over more than 100 years, ballot measures, once qualified and put on the ballot, succeed 35% of the time. Since a ballot measure either succeeds or fails on a majority vote, a binomial distribution seems appropriate, though we are not completely certain that a variation wouldn't be a better fit, especially since the measures themselves are likely not fully independent sequential events. From other posts, we see that a quasi-binomial, negative binomial, poisson and beta binomial distributions are other possibilities. How can we choose the best distribution for this situation? Do we need to run a few models with different parameters? How would we compare them?

As well, each election is distinct, so there could be some lurking variables that might lend themselves better to a variety of linear model? For example, amount of money raised, midterm election or not and the party affiliation of those backing the ballot measure. Rather than controlling for too many of these, which are difficult to collect, perhaps it would be enough to use an average p of success from a subgroup of more recent, similar elections? For example, only using the last 30-40 years and separating presidential elections from midterm elections.

If we set these concerns aside and use the 35% p of success on the current 7 ballot initiatives set to appear on the 2024 ballot, the binomial probability that 2 succeed is about 30%, and the cumulative probability that 2 or fewer succeed is about 53%.

Any thoughts and guidance would be helpful and appreciated!

Andrew Pederson
  • 11
  • "Since a ballot measure either succeeds or fails on a majority vote, a binomial distribution seems appropriate:" I would question that, because it assumes the success probability is a constant over time, which is extremely implausible. Have you considered analyzing the data? – whuber Aug 02 '23 at 17:56
  • The outcome is binomial - it succeeds or fails. I agree that it is not clear whether the generalized historical success probability is appropriate, which is why I'm asking this question. If the outcome is binomial, the success probability is unique to each question but conditional on some other factors, which distribution is the best fit for a predictive model? – Andrew Pederson Aug 02 '23 at 18:00
  • There is no general answer to that, because it depends on how the probabilities vary. Just being a binary outcome tells you nothing. – whuber Aug 02 '23 at 18:52
  • Hmm. So if I look at the dataset and compare the probabilities of each ballot measure succeeding or failing per election, that will tell me which distribution most accurately describes the data? – Andrew Pederson Aug 02 '23 at 19:02
  • I would begin by surveying, in a purely exploratory manner, the results over time. A simple time series plot (coding a success as 1 and failure as 0, for instance) coupled with a smoothed version would be a basic beginning. You might quickly identify long-term secular trends, observe some seasonality (perhaps related to four-year election cycles), sudden changes in success rates, notice changes in rates of ballot initiatives, and more. This already would be far richer than choosing any single distribution to describe the entire history of results. – whuber Aug 02 '23 at 21:11
  • 1
    This is very helpful, thank you! After an initial foray into the dataset today, it looks like it could be more useful to treat a ballot initiative like a candidate, and in both cases the most valuable information is polling and voting information. Will post an answer today or tomorrow to summarize, though not sure there will be time enough to go full Nate Silver on this question. I will also have to be clearer with clients that it's very difficult, if not impossible, to calculate a meaningful probability of success. – Andrew Pederson Aug 03 '23 at 00:27
  • That conclusion, albeit a negative one, is insightful and could be valuable for your clients. – whuber Aug 03 '23 at 12:14

1 Answers1

0

Thanks to some helpful input from @whuber, it's clear that this can't be a straightforward probability calculation because it is not the ballot measures' historical performance that makes any difference, rather the real probability we're interested in is the probability that over 50% of voters vote for or against it, or if enough voters are willing to sign a title and summary in order to qualify it.

The downside is that this makes ballot measure prediction about the same as election prediction, and polling is difficult and expensive. While measure proponents and opponents do conduct polling, these data are not usually released entirely to the public, and the industry is notably partisan.

The draft exploratory analysis is underway here, and I will attempt to use the polling I have available to model some predictions with multilevel regression with poststratification. The GitHub repo also contains the start of a raw data file for all ballot measures that can be expanded to include voting results, polling, money spent for/against, legal text, arguments for/against and notable endorsements. If nothing else, a sufficiently large dataset might be enough to get some interesting results from unsupervised learning.

Andrew Pederson
  • 11