*Bounty: 50*

*Bounty: 50*

I’m working on developing a physics lab about radioactive decay, and in analyzing sample data I’ve taken, I ran into a statistics issue that surprised me.

It is well known that the number of decays per unit time by a radioactive source is Poisson distributed. The way the lab works is that students count the number of decays per time window, and then repeat this many many times. Then they bin their data by the number of counts, and do a $chi^2$ goodness of fit test with 1 parameter estimated (the mean) to check whether or not the null hypothesis (the data is drawn from a Poisson distribution with the estimated mean value) holds. Hopefully they’ll get a large p-value and conclude that physics indeed works (yay).

I noticed that the way I binned my data had a large effect on the p-value. For example, if I chose lots of very small bins (e.g. a separate bin for each integer: 78 counts/min, 79 counts/min, etc.) I got a small p-value, and would have had to reject the null hypothesis. If, however, I binned my data into fewer bins (e.g. using the number of bins given by Sturge’s Rule: $1+log_{2}(N)$), I got a much larger p-value, and did NOT reject the null hypothesis.

Looking at my data, it looks extremely Poisson-distributed (It lines up almost perfectly with my expected counts/minutes). That said, there are a few counts in bins very far away from the mean. That means when computing the $chi^2$ statistic using very small bins, I have a few terms like:

$$frac{(Observed-Expected)^2}{Expected} = frac{(1-0.05)^2}{0.05}=18.05$$

This leads to a high $chi^2$ statistic, and thus a low p-value. As expected, the problem goes away for larger bin widths, since the expected value never gets that low.

**Questions:**

Is there a good rule of thumb for choosing bin sizes when doing a $chi^2$ GOF test?

Is this discrepancy between outcomes for different bin sizes something that I should have known about*, or is indicative of some larger problem in my proposed data analysis?

–

Thank you

*(I took a stats class in undergrad, but it’s not my area of expertise.)