A flat prior on a Bernoulli success probability p is described as uninformative. Give the density it induces on the log-odds, then derive the Jeffreys prior for p and state the property that makes it invariant.

A flat prior on a Bernoulli success probability p is described as uninformative. Give the density it induces on the log-odds, then derive the Jeffreys prior for p and state the property that makes it invariant.

Approach: Transform the density with the Jacobian of the log-odds map, then build the prior from the square root of the Fisher information and check how both sides transform.

A uniform prior on p induces the standard logistic density e^L/(1 + e^L)^2 on the log-odds L, and the Jeffreys prior is proportional to 1/sqrt(p*(1 - p)), which is Beta(1/2, 1/2). Changing variable from p to L = log(p/(1 - p)) carries the density through the Jacobian dp/dL = p*(1 - p) = e^L/(1 + e^L)^2, so a flat prior on p puts a mode at L = 0 with thin tails, which is a definite opinion: it says a log-odds of 4 is far less likely than one of 0. Flatness is a property of the coordinate system, so no prior is uninformative in every parameterisation. Jeffreys' rule sets the prior proportional to sqrt(det I(theta)) with I the Fisher information, and since information transforms with the square of the Jacobian while a density transforms with its first power, the rule commutes with reparameterisation. For a Bernoulli, I(p) = 1/(p*(1 - p)), giving p^{-1/2}*(1 - p)^{-1/2}, which adds half an observation of each kind to the posterior.

Follow-up: What is the Jeffreys prior for the mean of a normal with known variance and for its standard deviation, and why is one of them improper?

Key concepts: Jeffreys prior, reparameterisation invariance, Fisher information, Jacobian.