#StackBounty: #covariance #gaussian-process #conditional-expectation #prediction-interval Are the conditional expectation values of y a…

Bounty: 50

Suppose $y$ is a Gaussian process given by $y sim f + epsilon$, where $epsilon$ is a Gaussian noise model with zero mean, and $f$ is a deterministic yet unknown mean function (or a Gaussian process independent of $epsilon$). Therefore, one would find that $mathbb{E}[y] = mathbb{E}[f]$ since $mathbb{E}[epsilon] = 0$. But my question is: does $mathbb{E}[{{ bf y}_b vert { bf y}_a}] = mathbb{E}[{{ bf f}_b vert { bf y}_a}]$? Namely, are the conditional means of $bf f_b$ and $bf y_b$ equivalent?

The reason I ask is because we know $text{Var}[y] neq text{Var}[f]$ and $text{Var}[{{ bf y}_b vert { bf y}_a}] neq text{Var}[{{ bf f}_b vert { bf y}_a}]$. Additionally, the covariance matrix of $y$ is given by: $$Sigma_y(x_1,x_2) = k(x_1,x_2) + sigma^2 (x_1) delta(x_1 – x_2),$$ while the covariance matrix of $f$ is given by (c.f. the lines below equations 5.8 or below 2.30): $$Sigma_f(x_1,x_2) = k(x_1,x_2),$$ i.e. $y$ has an additional (possibly) heteroscedastic noise model, $sigma$, added along the diagonal of covariance matrix to represent the variance of the noise, $epsilon$. But after observing a set of measurements, $boldsymbol y_a$ at inputs $boldsymbol x_a$, the conditional mean of $boldsymbol y_b$ is given by:

$$mathbb{E}[{ bf y}_b vert { bf y}_a] = boldsymbolmu_b+Sigma_y(boldsymbol x_b,boldsymbol x_a){Sigma_y(boldsymbol x_a,boldsymbol x_a)}^{-1}({boldsymbol y_a}-boldsymbolmu_a) $$

but the conditional mean of $boldsymbol f_b$ is given by:

$$mathbb{E}[{ bf f}_b vert { bf y}_a] = boldsymbolmu_b+Sigma_f(boldsymbol x_b,boldsymbol x_a){Sigma_f(boldsymbol x_a,boldsymbol x_a)}^{-1}({boldsymbol y_a}-boldsymbolmu_a) $$

Therefore since $Sigma_y(x_1,x_2)$ does not necessarily equal $Sigma_f(x_1,x_2)$, is it accurate to state that $mathbb{E}[{{ bf y}_b vert { bf y}_a}]$ does not necessarily equal $mathbb{E}[{{ bf f}_b vert { bf y}_a}]$?

Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.