Benford's Law and file sizes

Benford's Law

Wikipedia has this entry for Benford's Law, which deals with the distribution of leading digits of a collection of samples: In this distribution, the number 1 occurs as the leading digit about 30% of the time, while larger numbers occur in that position less frequently: 9 as the first digit less than 5% of the time.

But is this "law" true for Linux file sizes? Many references claim that "computer file sizes" follow Benford's Law, but only if you don't account for "file type". I decided to test it. I wrote a program to gather data on Linux file sizes to test the common assertion that file sizes follow Benford's Law.

Digit A B C D 0 7901 1236 1233 265 1 63691 91489 25857 11524 2 50970 80713 15732 6791 3 24497 42356 11096 5472 4 18566 32831 8731 4510 5 14738 27335 6390 2787 6 12079 24884 4989 2188 7 12737 22963 4025 1875 8 12578 22362 5503 1781 9 10716 18541 3247 2168 File count 228473 364710 86803 39361

• Machine A: x86 Arch Linux development laptop
• Machine B: Slackware Linux 12.0, x86, SMTP, HTTP, NTP, DHCP server
• Machine C: Freshly installed Arch Linux x86 SMTP, HTTP, NTP, DHCP server
• Machine D: Corporate Developer's $HOME directory, x86_64 RHEL 5.0 server

Conclusion

Benford's Law appears to be true for Linux file sizes.

The Program

I wrote a C-language program so as to get numbers in a finite amount of time. It uses the C standard library function sprintf() to create a string representation of a file's size, then counts the first character of the string representation.

The program uses the GNU version of the ftw() C standard library function, as the POSIX-compatible version will skip files and directory trees. Don't use the POSIX-compatible ftw().