#StackBounty: #self-study #algorithms #entropy #information-theory #maximum-entropy How do I prove conditional entropy is a good measur…

Bounty: 200

This question is a follow-up of Does “expected entropy” make sense?, which you don’t have to read as I’ll reproduce the relevant parts. Let’s begin with the statement of the problem

A student has to pass an exam, with $$k$$ questions to be answered by yes or no, on a subject he knows nothing about. Assume the questions are independently distributed with a half-half probability of being either yes or no. The student is allowed to pass mock exams who have the same questions as the real exam. After each mock exam the teacher tells the student how many right answers he got, and when the student feels ready, he can pass the real exam. How many mock exams on average (a.k.a. take the expectation) must the student take to ensure he can get every single question correct in the real exam, and what should be his optimal strategy?

I have proposed an entropy-based strategy in that question, but for it to work, it must be first established that conditional entropy is a good measure of information to be recovered.

Here is a more concrete statement of my question. Suppose a student Alice has already taken 3 mock exams and got incomplete information about the answers. In a parallel universe, another student Bob has also taken 3 mock exams, but his strategy and insight about the answers may differ from those of Alice. At this point, both Alice and Bob have a conditional distribution of the answers based on outcomes of previous mock exams. I wonder if it can be proved that “the entropy of the conditional distribution from the perspective of Alice is greater or equal than that of Bob” can lead to “the minimum expected number of mock exams to be taken by Alice is greater or equal than that of Bob”.

Intuitively it makes sense because more entropy means more uncertainty and thus more attempts required, but I have no idea how to attack it. As a side note, this will be my bachelor’s thesis, so please just leave hints/pointers instead of spoiling too much 🙂

Get this bounty!!!

#StackBounty: #self-study #stochastic-processes Are the following function families the families of the probability densities of some s…

Bounty: 100

I am a beginner in stochastic processes, and I am trying to learn this branch of math. I have a few questions about the exercises I solved. I would like to ask if my reasoning was proper, and if the solution is good. The exercise states:

Are the following function families the families of the probability densities of some stochastic process?

(a) $$f_n(mathbf{t}_n,mathbf{x}_n)=left{begin{matrix} frac{1}{t_1t_2cdots t_n} & for; 0 leq x_i leq t_i,; i=1,2,…,n\ 0 & otherwise end{matrix}right.$$

(b)$$f_n(mathbf{t}_n,mathbf{x}_n)=left{begin{matrix} a_1a_2cdots a_ncdot exp(-a_1x_1-a_2x_2…-a_nx_n) & for; x_1>0,x_2>0,…,x_n>0\ 0 & otherwise end{matrix}right.$$

Bounty: 50

Suppose $$Xsim N_3(0,Sigma)$$, where $$Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$$.

On the basis of one observation $$x=(x_1,x_2,x_3)’$$, I have to obtain a confidence interval for $$rho$$ with confidence coefficient $$1-alpha$$.

We know that $$X’Sigma^{-1}Xsim chi^2_3$$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $$1-alpha$$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $$g_1(rho)le 0$$ and $$g_2(rho)ge 0$$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $$rho$$, I am guessing that the resulting C.I would be quite complicated.

Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $$bar x=frac{1}{3}sum x_i$$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $$chi^2_1$$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.

Get this bounty!!!

#StackBounty: #self-study #confidence-interval #estimation #multivariate-normal Confidence interval for $rho$ when \$Xsim N_3(0,Sigma…

Bounty: 50

Suppose $$Xsim N_3(0,Sigma)$$, where $$Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$$.

On the basis of one observation $$x=(x_1,x_2,x_3)’$$, I have to obtain a confidence interval for $$rho$$ with confidence coefficient $$1-alpha$$.

We know that $$X’Sigma^{-1}Xsim chi^2_3$$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $$1-alpha$$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $$g_1(rho)le 0$$ and $$g_2(rho)ge 0$$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $$rho$$, I am guessing that the resulting C.I would be quite complicated.

Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $$bar x=frac{1}{3}sum x_i$$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $$chi^2_1$$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.

Get this bounty!!!