#StackBounty: #time-series #estimation #stationarity #garch #estimators Moving estimators for nonstationary time series, like loglikeli…

Bounty: 50

While in standard (“static”) e.g. ML estimation we assume that all values are from a distribution of the same parameters, in practice we often have nonstationary time series: in which these parameters can evolve in time.

It is usually considered by using sophisticated models like GARCH conditioning sigma with recent errors and sigmas, or Kalman filters – they assume some arbitrary hidden mechanism.

I have recently worked on a simpler and more agnostic way: use moving estimator, like loglikelihood with exponentially weakening weights of recent values:
$$theta_T=argmax_theta l_Tqquad textrm{for}qquad l_T= sum_{t<T}eta^{T-t} ln(rho_theta(x_t)) $$
intended to estimate local parameters, separately on each position. We don’t assume any hidden mechanism, only shift the estimator.

For example it turns out that EPD (exponential power distribution) family $rho(x) propto exp(-|x|^kappa)$, which covers Gaussian ($kappa=2$) and Laplace ($kappa=1$) distributions, can have cheaply made such moving estimator (plots below), getting much better loglikelihood for daily log-returns of Dow Jones companies (100 years DJIA, 10 years individual), even exceeding GARCH: https://arxiv.org/pdf/2003.02149 – just using the $(sigma_{T+1})^kappa=eta (sigma_{T})^kappa+(1-eta)|x-mu|^kappa$ formula: replacing estimator as average with moving estimator as exponential moving average:

enter image description here

I have also MSE moving estimator for adaptive least squares linear regression: page 4 of https://arxiv.org/pdf/1906.03238 – can be used to get adaptive AR without Kalman filter, also analogous approach for adaptive estimation of joint distribution with polynomials: https://arxiv.org/pdf/1807.04119

Are such moving estimators considered in literature?

What applications they might be useful for?


Get this bounty!!!

#StackBounty: #estimation #markov-process #transition-matrix Estimating model for transition probabilities of a Markov Chain

Bounty: 100

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $Ttimes S$ values of (independent) variables, where $K$ is the number of variables that I can use to explain the transition probabilities ($p_{ij}$ are my dependent variables and the matrices $X_k$ are the independent variables).

Remember that $sum_j p_{ij}=1$ for each $t$.

In the end, I am looking for panel models to explain the transition probabilities, where the parameters are constant over time and (with maybe exception the constant) the parameters are also constant over different transition probabilities.

Just to be clear … Consider the following example … Imagine that an animal prefers to stay in places that there are food and water. Let the $Ttimes S$ matrix $X_F$ the matrix that tells the amount of food in each place $sin S$ and in each time $tin T$ and $X_W$ the matrix that tells the amount of water in each place $sin S$ and in each time $tin T$.

I want to use $X_F$ and $X_S$ to explain the transition probabilities. I do have the values of the transition probabilities over time and I want to use these matrices to explain their values.

I think I can design a kind of fixed effect logit model for each state in $S$. However, I would have to estimate $S$ logit models. I believe that the probabilities $p_{ij}$ and $p_{ji}$ should not be estimated in different models, since they seem to be related.

Any hints? Are there solutions in the literature of such kind of problem?


Get this bounty!!!

#StackBounty: #estimation #bias #central-limit-theorem Estimation and hypothesis testing for the difference in squared bias for two ran…

Bounty: 50

My Question:

Let $X_t$ and $Y_t$ denote two time-series random variables, both of which are estimates of the random variable $theta_t$. Let $U_t = X_t – theta_t$, and $V_t = Y_t – theta_t$. The bias of each variable is thus $mathbb{E} U_t$ and $mathbb{E} V_t$. I’m interested in estimation and hypothesis testing on the difference in squared bias, that is:
begin{equation}
gamma = (mathbb{E} U_t)^2 – (mathbb{E} V_t)^2
end{equation}

I thought this would be a fairly common problem, but some googling has not revealed much at all. I’m interested in any suggestions users have for estimating $gamma$, and conducting hypothesis tests.

My Current Approach:

Using the difference in squares, we get:
begin{equation}
gamma = (mathbb{E} U_t – mathbb{E} V_t)(mathbb{E} U_t + mathbb{E} V_t)
end{equation}

Let $bar{X} = frac{1}{T} sum_{t=1}^T X_t$, then we have the estimator:
begin{equation}
hat{gamma} = (bar{U} – bar{V})(bar{U} + bar{V})
end{equation}

Assuming that $U_t$ and $V_t$ obey the minimum conditions for Central Limit Theorem (CLT), e.g. bounded moments and weak dependence, the sample means converge in probability to their corresponding moments, and so using Slutsky’s theorem $hat{gamma} rightarrow gamma$ (in probability).

I think a CLT also applies. Multiplying by $sqrt{T}$ gives $ sqrt{T} (bar{U} – bar{V}) (bar{U} + bar{V})$. The first term $sqrt{T} (bar{U} – bar{V})$ will obey a CLT (when centred and scaled by an appropriate standard deviation estimator), and if the second term $bar{U} + bar{V}$ conveges in probability to a non-zero value, then by an application of Cramer’s theorem $hat{gamma} rightarrow mathcal{N}$.

This solution is not totally satisfactory. In practice, the $bar{U} + bar{V}$ converging to a non-zero value can be an issue. Under simulation, if this expression is converging to something close to zero, the simulated distribution definitely looks a bit pointy for a Normal. And this is a situation that could easily arise, if $X_t$ and $Y_t $ are both close to unbiased.

So, any ideas on how I can improve this estimator?


Get this bounty!!!

#StackBounty: #estimation #bias #central-limit-theorem #squared Estimation and hypothesis testing for the difference in squared bias fo…

Bounty: 50

My Question:

Let $X_t$ and $Y_t$ denote two time-series random variables, both of which are estimates of the random variable $theta_t$. Let $U_t = X_t – theta_t$, and $V_t = Y_t – theta_t$. The bias of each variable is thus $mathbb{E} U_t$ and $mathbb{E} V_t$. I’m interested in estimation and hypothesis testing on the difference in squared bias, that is:
begin{equation}
gamma = (mathbb{E} U_t)^2 – (mathbb{E} V_t)^2
end{equation}

I thought this would be a fairly common problem, but some googling has not revealed much at all. I’m interested in any suggestions users have for estimating $gamma$, and conducting hypothesis tests.

My Current Approach:

Using the difference in squares, we get:
begin{equation}
gamma = (mathbb{E} U_t – mathbb{E} V_t)(mathbb{E} U_t + mathbb{E} V_t)
end{equation}

Let $bar{X} = frac{1}{T} sum_{t=1}^T X_t$, then we have the estimator:
begin{equation}
hat{gamma} = (bar{U} – bar{V})(bar{U} + bar{V})
end{equation}

Assuming that $U_t$ and $V_t$ obey the minimum conditions for Central Limit Theorem (CLT), e.g. bounded moments and weak dependence, the sample means converge in probability to their corresponding moments, and so using Slutsky’s theorem $hat{gamma} rightarrow gamma$ (in probability).

I think a CLT also applies. Multiplying by $sqrt{T}$ gives $ sqrt{T} (bar{U} – bar{V}) (bar{U} + bar{V})$. The first term $sqrt{T} (bar{U} – bar{V})$ will obey a CLT (when centred and scaled by an appropriate standard deviation estimator), and if the second term $bar{U} + bar{V}$ conveges in probability to a non-zero value, then by an application of Cramer’s theorem $hat{gamma} rightarrow mathcal{N}$.

This solution is not totally satisfactory. In practice, the $bar{U} + bar{V}$ converging to a non-zero value can be an issue. Under simulation, if this expression is converging to something close to zero, the simulated distribution definitely looks a bit pointy for a Normal. And this is a situation that could easily arise, if $X_t$ and $Y_t $ are both close to unbiased.

So, any ideas on how I can improve this estimator?


Get this bounty!!!

#StackBounty: #mathematical-statistics #estimation The difference of normal means is also minimax?

Bounty: 50

Let $X_i sim N(xi, sigma^2)$ and $Y_i sim N(eta, tau^2)$ for known $sigma^2$ and $tau^2$.

I know that $bar{X}$ and $bar{Y}$ are minimax under squared error loss since their variance is fixed, and a sequence of Bayes estimators can be constructed such that their Bayes risk converge to the maximum risk of $bar{X}$ and $bar{Y}$.

I am wondering how I can show $delta(X,Y) =bar{Y}-bar{X}$ is also minimax for $eta-xi$?

Essentially I need to show that for any $T(X,Y)$, we have

$$sup_{xi,eta} E(T(X,Y)-(eta-xi))^2 leq sup_{eta,xi} E((bar{Y}-bar{X})-(eta-xi))^2 = frac{sigma^2+tau^2}{n}$$

The only thing I can think of doing is constructing another sequence of priors whose Bayes risks converge to the RHS. However, this seems kind of tedious now that we’re in 2 dimensions. I feel like there’s a “trick” here that I should be using?


Get this bounty!!!

#StackBounty: #estimation #causality #identifiability What does it mean to "non-parametrically" identify a causal effect with…

Bounty: 100

I am wondering, within the context of causal inference, what it means to “non-parametrically” identify a causal effect within the super-population perspective. For example, in Hernan/Robins Causal Inference Book Draft:

https://cdn1.sph.harvard.edu/wp-content/uploads/sites/1268/2019/02/hernanrobins_v1.10.38.pdf

It defines non-parametric identification on pg. 43 and 123 as:

…identification that does not require any modeling assumptions when
the size of the study population is quasi-infinite. By acting as if we
could obtain an unlimited number of individuals for our studies, we
could ignore random fluctuations and could focus our attention on
systematic biases due to confounding, selection, and measurement.
Statisticians have a name for problems in which we can assume the size
of the study population is effectively infinite: identification
problems.

I understand the identification part to mean that under the strong ignorability assumption, there is only ONE way for the observed data to correspond to a causal effect estimand. What confuses me is why we need to assume the size of the study is quasi-infinite.

For example, in the book it gives an example of a 20 person study where each subject was representative of 1 billion identical subjects, and to view the hypothetical super-population as that of 20 billion people. Specifically, on pg. 13 it states that:

… we will assume that counterfactual outcomes are
deterministic and that we have recorded data on every subject in a
very large (perhaps hypothetical) super-population. This is equivalent
to viewing our population of 20 subjects as a population of 20 billion
subjects in which 1 billion subjects are identical to the 1st subject, 1 billion
subjects are identical to the 2nd subject, and so on.

My confusion here is what it means to assume a single person is representative of 1 billion identical individuals. Is it assuming that each of the 1 billion are identical with respect to their outcomes and treatment only, but differ with respect to the covariates? Or is it assuming the individual is a summary measure of the 1 billion? My instinct is that the notion of the 1 billion is entertaining the fact we may draw many times without having a case where we have a lack of samples. I.e., small sample sizes result in more unstable estimates.

Essentially, what is so crucial about assuming there are many identical individuals in the “background”, if they are just going to be the same as a patient you observe? What happens or breaks down if instead of the 1 billion, we only had 2 identical individuals?

Thank you for any insight.


Get this bounty!!!

#StackBounty: #self-study #confidence-interval #estimation #multivariate-normal Confidence interval for $rho$ when $Xsim N_3(0,Sigma…

Bounty: 50

Suppose $Xsim N_3(0,Sigma)$, where $Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$.

On the basis of one observation $x=(x_1,x_2,x_3)’$, I have to obtain a confidence interval for $rho$ with confidence coefficient $1-alpha$.

We know that $X’Sigma^{-1}Xsim chi^2_3$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $1-alpha$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $g_1(rho)le 0$ and $g_2(rho)ge 0$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $rho$, I am guessing that the resulting C.I would be quite complicated.


Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $bar x=frac{1}{3}sum x_i$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $chi^2_1$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.


Get this bounty!!!

#StackBounty: #self-study #confidence-interval #estimation #multivariate-normal Confidence interval for $rho$ when $Xsim N_3(0,Sigma…

Bounty: 50

Suppose $Xsim N_3(0,Sigma)$, where $Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$.

On the basis of one observation $x=(x_1,x_2,x_3)’$, I have to obtain a confidence interval for $rho$ with confidence coefficient $1-alpha$.

We know that $X’Sigma^{-1}Xsim chi^2_3$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $1-alpha$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $g_1(rho)le 0$ and $g_2(rho)ge 0$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $rho$, I am guessing that the resulting C.I would be quite complicated.


Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $bar x=frac{1}{3}sum x_i$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $chi^2_1$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.


Get this bounty!!!

#StackBounty: #self-study #confidence-interval #estimation #multivariate-normal Confidence interval for $rho$ when $Xsim N_3(0,Sigma…

Bounty: 50

Suppose $Xsim N_3(0,Sigma)$, where $Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$.

On the basis of one observation $x=(x_1,x_2,x_3)’$, I have to obtain a confidence interval for $rho$ with confidence coefficient $1-alpha$.

We know that $X’Sigma^{-1}Xsim chi^2_3$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $1-alpha$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $g_1(rho)le 0$ and $g_2(rho)ge 0$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $rho$, I am guessing that the resulting C.I would be quite complicated.


Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $bar x=frac{1}{3}sum x_i$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $chi^2_1$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.


Get this bounty!!!

#StackBounty: #self-study #confidence-interval #estimation #multivariate-normal Confidence interval for $rho$ when $Xsim N_3(0,Sigma…

Bounty: 50

Suppose $Xsim N_3(0,Sigma)$, where $Sigma=begin{pmatrix}1&rho&rho^2\rho&1&rho\rho^2&rho&1end{pmatrix}$.

On the basis of one observation $x=(x_1,x_2,x_3)’$, I have to obtain a confidence interval for $rho$ with confidence coefficient $1-alpha$.

We know that $X’Sigma^{-1}Xsim chi^2_3$.

So expanding the quadratic form, I get

$$x’Sigma^{-1}x=frac{1}{1-rho^2}left[x_1^2+(1+rho^2)x_2^2+x_3^2-2rho(x_1x_2+x_2x_3)right]$$

To use this as a pivot for a two-sided C.I with confidence level $1-alpha$, I setup $$chi^2_{1-alpha/2,3}le x’Sigma^{-1}xle chi^2_{alpha/2,3}$$

I get two inequalities of the form $g_1(rho)le 0$ and $g_2(rho)ge 0$, where

$$g_1(rho)=(x_2^2+chi^2_{alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{alpha/2,3}$$

and $$g_2(rho)=(x_2^2+chi^2_{1-alpha/2,3})rho^2-2(x_1x_2+x_2x_3)rho+x_1^2+x_2^2+x_3^2-chi^2_{1-alpha/2,3}$$

Am I right in considering a both-sided C.I.? After solving the quadratics in $rho$, I am guessing that the resulting C.I would be quite complicated.


Another suitable pivot is $$frac{mathbf1′ x}{sqrt{mathbf1’Sigma mathbf 1}}sim N(0,1)quad,,,mathbf1=(1,1,1)’$$

With $bar x=frac{1}{3}sum x_i$, this is same as saying $$frac{3bar x}{sqrt{3+4rho+2rho^2}}sim N(0,1)$$

Using this, I start with the inequality $$left|frac{3bar x}{sqrt{3+4rho+2rho^2}}right|le z_{alpha/2}$$

Therefore, $$frac{9bar x^2}{3+4rho+2rho^2}le z^2_{alpha/2}implies 2(rho+1)^2+1ge frac{9bar x^2}{z^2_{alpha/2}}$$

That is, $$rhoge sqrt{frac{9bar x^2}{2z^2_{alpha/2}}-frac{1}{2}}-1$$

Since the question asks for any confidence interval, there are a number of options available here. I could have also squared the standard normal pivot to get a similar answer in terms of $chi^2_1$ fractiles. I am quite sure that both methods I used are valid but I am not certain whether the resulting C.I. is a valid one. I am also interested in other ways to find a confidence interval here.


Get this bounty!!!