*Bounty: 50*

I have a Markov chain with transition matrix $P$, with transition probabilities:

$$p_{i,j}=

begin{cases}

1-d, & text{if $i=j gt 0$} [2ex]

d , & text{ if $j=i-1 gt 0$} [2ex]

(1- gamma)^{j-1}gamma , &text{ for $0=i lt j$} [2ex]

end{cases}$$

Let $T_{i,j}$ be the first hitting time of state $j$ after the first hitting time of state $i$ (i.e $T_{i,j}= inf{ geq1 : X_n=j , n gt H_i}$).

$(mathbf 1)$I would like to explain why with $S_{i,j}=T_{i,j}-H^i$ for $i gt 0$ we can write $$S_{i,0}=sum^i_{k=1}S_{k,k-1}$$

$(mathbf 2)$Also, I would like to determine the distribution of $S_{k,k-1}$ and confirm it using the fact ${H^k lt infty}$ for all $k$, then $(mathbf 3)$ I would like to justify why the sum is made from a sequence of iid random variables.i.e why the $S_{k,k-1}$ terms are independent of each other

So, for my first task, here’s how I got with it:

By my logic, $T_{i,j}$ is just another way of writing the hitting time of $j$ . So, for example $X_0$=4 – (our first state at which we start is $4$). The say hitting time of state $3$ is $2$ and then hitting time of state $2$ is 4. So, $T_{4,2}=4$ is just the more fancy way of writing $H^2=4$ . But I also observe that $T_{3,2}=4$. I mean, by definition, the $n$ should be greater that $H^3=2$.

Ok, so by my logic, $S_{4,0}$ is just the number of steps we make from $4$ until we hit $0$(i.e $S_{i,j}=H^j-H^i$ So, to understand why we can write $S_{i,0}$ in such a form, I made a small example. Let $X_0=4, H^3=2, H^2=4,H^1=6,H^0=8$

So, $S_{3,0}=S_{1,0}+S_{2,1}+S_{3,2}=H^0-H^1+H^1-H^2+H^2-H^3=8-6+6-4+4-2=6$

And indeed, the steps I make from $3$ until I hit $0$ are $6$.

So, I’m not quite sure how to prove this result mathematically, but an intuitive explanation is such:

“Since the sum $sum^i_{k=1}S_{k,k-1}$ represents the sum of all “steps” taken inbetween the consecutive states, it equals to the total sum of “steps” taken from the first state of the sum to the end state of the sum”

Is there any better way to formulate this? Am I correct in my thinking?

Task$(mathbf 2)$: I observe that the distribution of $S_{k,k-1}sim Geom(d)$ . We can confirm that it is indeed geometrically distributed, using the facts that ${H^k lt infty}$ for all $k$ and that the Geometric distribution is almost surely finite. i.e $P(S_{k,k-1} lt infty)=1$

Task $(mathbf 3)$ . I’m actually not quite sure how to justify why the sum is made of iid random variables. Any suggestions?

Any help is appreciated

Get this bounty!!!