Advanced Dynamic Programming

Fall 2015

Instructor: Dr. Rajesh Ganesan

Eng Bldg. Room 2217

Phone: (703) 993-1693                                                    

Fax: (703) 993-1521                                                                                 

Email: rganesan at gmu dot edu

Syllabus

DP refresher  Notes

ADP need and eqautions

Machine replacement problem:

Limiting Probabilities and MDP, Exhaustive enumeration, LP solution to MDP,  MDP- Average Reward/cost- Policy and Value Iteration, Discounted Cost-Policy and Value Iteration

notes

Excel Example value iteration for MDP

Matlab files:

Value iteration Discounted cost criteria

Figure 4.2 page 120 value iteration_ADP

Figure 4.4 page 128 value iteration_ADP2

Figure 4.7 page 141 value iteration_ADP 3

Fig 4.7 page 141 with MSE value iteration_ADP3x

1) Solve the following using Matab or any other language using the algorithms a) value iteration b) 4.2 c) 4.4, and d) 4.7 given above.

Inventory control Example,  see hand out,     excel for inventory control

2) Water Resource example: see handout      value iteration using excel

Solve the water resource problem using a) value iteration b) 4.2 c) 4.4, and d) 4.7 given above with dam capcity 20 units. The possible decisions are to release upto the dam's availability. Use same cost functions as in the handout. The probability of inflow is

Prob of inflow
P(0)	0.02
P(1)	0.05
P(2)	0.01
P(3)	0.06
P(4)	0.02
P(5)	0.02
P(6)	0.08
P(7)	0.09
P(8)	0.01
P(9)	0.04
P(10)	0.1
P(11)	0.04
P(12)	0.02
P(13)	0.16
P(14)	0.04
P(15)	0.05
P(16)	0.02
P(17)	0.08
P(18)	0.06
P(19)	0.02
P(20)	0.01

 

How does the solution change if the probability of inflow is uniform (equal prob of 1/21) or poisson with mean 10.

alpha decay

Fig 4.7 page 141 with MSE value iteration_ADP4 to study different alpha decay rates

VFA matlab code with 4 schemes

VFA with diffusion wavelet

VFA DW Theory

DW - Haar wavelet demo to go with the ppt above

Diffusion wavelets DW code for best basis - multiple levels

excel to show value determination

ADP with only scaling function code

ADP with scaling and wavelet functions code