Foster’s Lemma provides a natural condition to prove the positive recurrence of a Markov chain.
- Weighted majority algorithm its variant for Bandit Problems.
- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality
Heuristic derivation of
- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula
- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.
- A short introduction to Markov chains for dynamic programming
- Definition, Markov Property, some Potential Theory.