- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality
Heuristic derivation of
- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula
- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.
- A short introduction to Markov chains for dynamic programming
- Definition, Markov Property, some Potential Theory.
We prove a powerful inequality which provides very tight gaussian tail bounds “” for probabilities on product state spaces . Talagrand’s Inequality has found lots of applications in probability and combinatorial optimization and, if one can apply it, it generally outperforms inequalities like Azzuma-Hoeffding.
- Laplacian; Adjoints; Harmonic fn; Green’s fn; Forward Eqn; Backward Eqn.
- Markov Chains and Martingales; Green’s Functions and occupancy; Potential functions; time-reversal and adjoints.
We show that relative entropy decreases for continuous time Markov chains.