*Bounty: 50*

*Bounty: 50*

I often hear it said that the Jeffreys prior is well-motivated because it is invariant under reparametrization. The proof of this is quite straight-forward (I know the proof on e.g., wiki). I’m a bit confused about what the proof really means, though, because the kind of invariance proven is a bit strange to me. It is indeed proven that if

$$

p(x) propto sqrt{I(x)}

$$

then

$$

p(y) propto sqrt{I(y)}

$$

where $I$ is the Fisher information and $y$ was found through a bijective transformation of $x$. Note well that $I(x)$ is an abuse of notation, as it contains derivatives wrt the variable $x$.

I don’t see this as particularly compelling, since I make a similar argument that *any* choice of prior is parametrization invariant. E.g., by writing an arbitrary prior as

$$

p(theta) = frac{dF(theta)}{dtheta}

$$

where $F$ is the cumulative distribution function, we then find

$$

p(phi) = frac{dF(phi)}{dphi}

$$

To put it another way, I can specify a prior by specifying a cdf rather than a pdf, and the cdf transforms trivially under reparameterizations. This kind of invariance is of basically no interest to me.

So, why do people make a fuss about the Jeffreys prior being invariant under reparameterization? I think I would rather say that the kind of invariance that the Jeffreys prior has is necessary for any objective formal rule for selecting a prior, but not in itself a motivation for using a Jeffreys prior. And I think it would be better to say that the Jeffreys *rule* for making a prior was parameterisation invariant, than say the Jeffreys prior was parameterisation invariant. Is that fair?