Markov chains

 

Key features of Markov chains

The key features of Markov chains are as follows:
1. The outcome of each experiment is one of the discrete state of the chain.
2. The outcome of an experiment depends on only the present state and not on any past states.
3. The transition probabilities remain constant from one transition to the next.

---

Markov process with example

- If the future outcome depends only on the present outcome but not on the past outcomes, then the process is known as Markov process.
- If the states of the Markov process is discrete, then it is known as Markov chain.

---

Transition Probability Matrix:

- The transition probability matrix is the matrix that shows probabilities of moving from one state to another state from current generation to next generation.
- It is represented by P.
- P^k represents the probabilities of transition from one state to another in k repetitions of the experiment if and only if P is constant for each transition.
- It is a square matrix.
- All entries are between 0 and 1.
- The sum of the entries of any row is equal to 1.

---

Current Status Distribution Matrix:

- Current status distribution matrix is a row matrix that provides the status of current state of all the discrete states of the Markov chain.
- Each entry must be between 0 and 1 inclusive.
- The sum of entries of each row must be 1.
- It is denoted by Qo.

---

Steps to make predictions:

1. Create current status distribution matrix Qo.
2. Create probability distribution matrix P.
3. Calculate Qo * P ^ n, which represents probability vector after n repetitions of the experiment.

---

Application of Markov chain

The general applications of Markov chain are as follows:
1. Turing System
2. Traffic System
3. Telephone System
4. Weather Forecasting

---

Q. Given that chance of a Honda bike user to buy Honda bike at next purchase is 70% and that his next purchase will be Yamaha is 30%. The chance of Yamaha bike user to buy Yamaha bike at next purchase is 80% and that his next purchase will be Honda is 20%. What is the probability to buy Yamaha bike after three purchase of a current Honda Bike user?

Answer
-------->>>>

---

Question
---------