Stable Distributions

Wikipedia says, about stable distributions:

a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters.

In order to understand Colorado elections risk limiting audits, I decided to run through an ancient (1999) Schaum’s Outline for Statistics. Reading through that led me to investigate normal distributions in more detail than I have in the past. The wikipedia page for normal distributions mentioned other “bell curve” distributions, like the Cauchy distribution. The wikipedia page for Cauchy distributions mentioned that they were stable distributions.

But are they really?

What does a “linear combination of two independent variables” means?

How does a pile of numbers (the population or samples) which are scalars, end up with a 2-dimensional “probability distribution function”?

Naturally, I wrote some programs to try these ideas out.

Source code.

The “samples (scalars) to probability distribution function” question is the easiest.

  1. Pick a “bin size”. Each bin is a range, 0 to 0.99, 1.0 to 1.99, 2.0 to 2.99, etc.
  2. Organize all samples in appropriate bins by value
  3. Count the samples in each bin,
  4. Plot the counts of samples. X coordinates are midpoints of bins, Y coordinates are counts of samples in a bin.

If you have a huge number of samples, at any bin size, even vanishingly small, there will be a non-zero, greater-than-1 sample count. Due to the Miracle of Calculus, the curve described by those sample counts comprises the Probability Distribution Function. At least that’s how I currently understand it.

I had trouble creating histograms from groups of samples that matched graphs of published probability distribution functions. What I didn’t understand until right now is that “density” histograms are scaled vertically so that the sum of the areas of the histogram boxes ends up as 1.0. That way, the theoretical probability density function, which has an area under it of 1.0, should match the top midpoints of the histogram boxes.

I found this difficult to accomplish. I bailed out, and used the R statistical computing and graphics language.

A “linear combination of two independent random variables” does not mean concatenating two sets of samples, then looking at their mean, standard deviation, kurtosis, etc. That just gets you a weird bimodal distribution. You have to draw two samples, one from each set, then combine them linearly. I had to generate two sets of samples, make a histogram of each, make a histogram of the two sets concatenated, and discover this the hard way.

Two Cauchy Distributions

To futz around with the “stable distribution” idea, I created a program that would create randomly-generated values, supposedly in a Cauchy Distribution, from two random number generator with different seed values. The program outputs the two distributions, and the values of the two distributions added together. What’s the simplest linear combination? Why, w = x0 + x1, of course.

2 histograms of cauchy distributions with probability density functions

Each of these histograms represents counts of 100,000 values generated using random number generators (RNG) with different seeds. I used the Go programming language package github.com/leesper/go_rng to create the 200,000 values that are summarized in the histograms.

The probability density function of a Cauchy distribution is:

1/[πγ(1 + ((x - x0)/γ)2)]

x0 is the “location”, the x-axis value where the top of the hump resides. γ (Greek lowercase gamma) is the “scale”. A numerically larger scale causes a flatter, wider, distribution. γ has to be greater than 0.0.

The blue and red curves on the graphic above are graphs of the PDF functions of the two distributions. Red curve has γ = 1.0 and x0 = 0. Blue curve has γ = 2.0 and x0 = 5.

I think those curves are good fits of the histogram box top midpoints.

2 histograms of cauchy distributions and their linear combination

Above, the two Cauchy distribution (orange and green shaded boxes), and the distribution that results from adding values from RNGs as they are generated. It’s shaded in blue. It’s a linear combination of the orange and green shaded values.

histogram of combined cauchy distributions and its PDF

Here’s the histogram of the “linear combination”, along with a curve-fitted Cauchy distribution probability distribution function, which has γ = 2.69618 and x0 = 4.96559. Seems like a great fit. I deem this a Cauchy distribution.

It makes sense that the combined distribution is aligned with the distribution with location at 5.0. The first distribution, with location 0.0, has smaller values. Adding those smaller values randomly to the larger values of the second distribution is just going to lower the peak of the combined distribution and smear it out.

Summary

An effort to understand something (risk limiting audits) led to brushing up on statistics, which led to a wikipedia rabbit hole. An article deep in the rabbit hole made an unsupported assertion that I thought I could verify experimentally. I did verify the “stable distribution” assertion empirically. I had to write a couple of programs, learn a little of the R statistical and graphics language, and re-learn how to do a curve fit in gnuplot. Not too shabby.