We consider a specific diffusion control problem. We focus on setting where there is one risky asset and one riskless asset, though we will see that much of the analysis passes over to multiple assets.
- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality
Heuristic derivation of
- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula
- Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.
- Optimal Stopping Problems; One-Step-Look-Ahead Rule
- The Secretary Problem.
- Infinite Time Stopping
- High level idea: Policy Improvement and Policy Evaluation.
- Value Iteration; Policy Iteration.
- Temporal Differences; Q-factors.
- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.