# #StackBounty: #markov-process Different "cases" in Markov chain example

### Bounty: 50

I am presented with the following Markov chain example:

A trader sells large and expensive machines.

$$X_n$$ is the number of machines in stock at the start of week $$n$$.

$$D_n$$ is the number of machines demanded by customers during week $$n$$.

Assume that $$D_n sim text{Poi}(3)$$, the $$D_n$$ are independent, and that $$D_n$$ and $$X_n$$ are independent for each $$n$$.

There are two stipulations:

1. Inventory control: If $$0$$ or $$1$$ machines are left in stock by the end of a week, then machines are ordered and delivered to raise the stock to $$5$$ by the start of the next week. If $$2$$ or more machines are in stock at the end of the week, then no orders are placed.

2. Lost business: If $$D_n > X_n$$, then the unsatisfied demands are lost.

We seek to show that $$X_n$$ is a Markov chain.

$$(X_n – D_n)^+$$ is the number of machines in stock at the end of week $$n$$.

$$X_{n + 1} begin{cases} X_n – D_n & text{if} X_n – D_n ge 2 \ 5 & text{if} X_n – D_n le 1 end{cases}$$

$$S = { 2, 3, 4, 5 }$$

Denote the history of the process up to time $$n$$ by $$H_n = { X_0, X_1, dots, X_n }$$.

Given that $$X_n = i$$, the independence assumptions ensure that $$X_{n + 1}$$ is conditionally independent of $$H_{n – 1}$$.

Case 1: $$j = 2, 3,$$ or $$4$$, and $$i = j, dots, 5$$:

begin{align} P(X_{n + 1} = j vert X_n = i, H_{n – 1}) &= P(X_n – D_n = j vert X_n = i, H_{n – 1}) \ &= P(i – D_n = j vert X_n = i, H_{n – 1}) \ &= P(i – D_n = j) \ &= P(D_n = i – j) \ &= e^{-3} dfrac{3^{i – j}}{(i – j)!} end{align}

Case 2: $$i = 2, 3,$$ or $$4$$, and $$j = 5$$:

begin{align} P(X_{n + 1} = 5 vert X_n = i, H_{n – 1}) &= P(X_n – D_n le 1 vert X_n = i, H_{n – 1}) \ &= P(i – D_n le 1 vert X_n = i, H_{n – 1}) \ &= P(i – D_n le 1) \ &= P(D_n ge i – 1) \ &= 1 – P(D_n le i – 2) \ &=: p_{i5} end{align}

Case 3: $$i = 5, j = 5$$:

$$p_{55} = P(D_n = 0) + P(D_n ge 4) = P(D_n = 0) + 1 – P(D_n le 3).$$

Can someone explain the different “cases” here? Specifically, I’m unsure about of the reasoning the led to the three cases.

I would greatly appreciate it if people would please take the time to clarify this.

Get this bounty!!!

This site uses Akismet to reduce spam. Learn how your comment data is processed.