#StackBounty: #markov-process Different "cases" in Markov chain example

Bounty: 50

I am presented with the following Markov chain example:

A trader sells large and expensive machines.

$X_n$ is the number of machines in stock at the start of week $n$.

$D_n$ is the number of machines demanded by customers during week $n$.

Assume that $D_n sim text{Poi}(3)$, the $D_n$ are independent, and that $D_n$ and $X_n$ are independent for each $n$.

There are two stipulations:

  1. Inventory control: If $0$ or $1$ machines are left in stock by the end of a week, then machines are ordered and delivered to raise the stock to $5$ by the start of the next week. If $2$ or more machines are in stock at the end of the week, then no orders are placed.

  2. Lost business: If $D_n > X_n$, then the unsatisfied demands are lost.

We seek to show that $X_n$ is a Markov chain.

$(X_n – D_n)^+$ is the number of machines in stock at the end of week $n$.

$$X_{n + 1} begin{cases} X_n – D_n & text{if} X_n – D_n ge 2 \ 5 & text{if} X_n – D_n le 1 end{cases}$$

$S = { 2, 3, 4, 5 }$

Denote the history of the process up to time $n$ by $H_n = { X_0, X_1, dots, X_n }$.

Given that $X_n = i$, the independence assumptions ensure that $X_{n + 1}$ is conditionally independent of $H_{n – 1}$.

Case 1: $j = 2, 3,$ or $4$, and $i = j, dots, 5$:

$$begin{align} P(X_{n + 1} = j vert X_n = i, H_{n – 1}) &= P(X_n – D_n = j vert X_n = i, H_{n – 1}) \ &= P(i – D_n = j vert X_n = i, H_{n – 1}) \ &= P(i – D_n = j) \ &= P(D_n = i – j) \ &= e^{-3} dfrac{3^{i – j}}{(i – j)!} end{align}$$

Case 2: $i = 2, 3,$ or $4$, and $j = 5$:

$$begin{align} P(X_{n + 1} = 5 vert X_n = i, H_{n – 1}) &= P(X_n – D_n le 1 vert X_n = i, H_{n – 1}) \ &= P(i – D_n le 1 vert X_n = i, H_{n – 1}) \ &= P(i – D_n le 1) \ &= P(D_n ge i – 1) \ &= 1 – P(D_n le i – 2) \ &=: p_{i5} end{align}$$

Case 3: $i = 5, j = 5$:

$$p_{55} = P(D_n = 0) + P(D_n ge 4) = P(D_n = 0) + 1 – P(D_n le 3).$$

Can someone explain the different “cases” here? Specifically, I’m unsure about of the reasoning the led to the three cases.

I would greatly appreciate it if people would please take the time to clarify this.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.