#StackBounty: #prior #uninformative-prior #invariance #objective-bayes Significance of parameterisation invariance of Jeffreys prior

Bounty: 50

I often hear it said that the Jeffreys prior is well-motivated because it is invariant under reparametrization. The proof of this is quite straight-forward (I know the proof on e.g., wiki). I’m a bit confused about what the proof really means, though, because the kind of invariance proven is a bit strange to me. It is indeed proven that if
$$
p(x) propto sqrt{I(x)}
$$

then
$$
p(y) propto sqrt{I(y)}
$$

where $I$ is the Fisher information and $y$ was found through a bijective transformation of $x$. Note well that $I(x)$ is an abuse of notation, as it contains derivatives wrt the variable $x$.

I don’t see this as particularly compelling, since I make a similar argument that any choice of prior is parametrization invariant. E.g., by writing an arbitrary prior as
$$
p(theta) = frac{dF(theta)}{dtheta}
$$

where $F$ is the cumulative distribution function, we then find
$$
p(phi) = frac{dF(phi)}{dphi}
$$

To put it another way, I can specify a prior by specifying a cdf rather than a pdf, and the cdf transforms trivially under reparameterizations. This kind of invariance is of basically no interest to me.

So, why do people make a fuss about the Jeffreys prior being invariant under reparameterization? I think I would rather say that the kind of invariance that the Jeffreys prior has is necessary for any objective formal rule for selecting a prior, but not in itself a motivation for using a Jeffreys prior. And I think it would be better to say that the Jeffreys rule for making a prior was parameterisation invariant, than say the Jeffreys prior was parameterisation invariant. Is that fair?


Get this bounty!!!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.