#StackBounty: #goodness-of-fit #kolmogorov-smirnov-test #power-law #zipf Is KS test really appropriate when validating a power law/esti…

Bounty: 50

I’m attempting to find out whether some highly skewed data are drawn from a power law distribution, following the popular paper by Clauset, Shalizi and Newman, 2009.

Clauset et al. use the Kolmogorov-Smirnov (KS) statistic to measure the goodness-of-fit of the data to the hypothesised power law distribution. However, in an old paper on the Whitworth distribution by Nancy Geller, she mentions that once observations are ranked, they are no longer independently and identically distributed and therefore the KS test becomes invalid.

My question: Does this mean that the KS test is also invalid when considering any power law where a quantity x is scaled according to its rank (i.e. Zipf’s law)? Or, is it still valid since Clauset et al. did a simulation test and, in Fig. 4 (p. 673), it appears as though the KS test performs fine anyway?

Apologies if this is a silly question and please feel free to point me in the right direction if I’ve missed something more basic here.

References

Clauset, A., C. R. Shalizi, and M. E. J. Newman. 2009. “Power-Law Distributions in Empirical Data.” SIAM Review 51 (4): 661–703. doi:10.1137/070710111

Geller, N.L. 1979, “A Test of Significance for the Whitworth Distribution”, Journal of the American Society for Information Science, vol. 30, no. 4, pp. 229.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.