Advanced Dynamic Programming

Fall 2017


Instructor: Dr. Rajesh Ganesan




Eng Bldg. Room 2217

Phone: (703) 993-1693                                                    

Fax: (703) 993-1521                                                                                 

Email: rganesan at gmu dot edu




DP refresher  Notes

ADP need and eqautions


Machine replacement problem:

Limiting Probabilities and MDP, Exhaustive enumeration, LP solution to MDP,  MDP- Average Reward/cost- Policy and Value Iteration, Discounted Cost-Policy and Value Iteration


Excel Example value iteration for MDP


Matlab files:

Value iteration Discounted cost criteria

Figure 4.2 page 120 value iteration_ADP

Figure 4.4 page 128 value iteration_ADP2

Figure 4.7 page 141 value iteration_ADP 3