Structural properties of stochastic dynamic programs

In Markov models of sequential decision processes one is often interested in showing that the value function is monotonic convex and/or supermodular in the state variables These kinds of results can be used to develop a qualitative understanding of the model and characterize how the results will change with changes in model parameters In this paper we present several fundamental results for establishing these kinds of properties The results are in essence metatheorems showing that the value functions satisfy property P if the reward functions satisfy property P and the transition probabilities satisfy a stochastic version of this property We focus our attention on closed convex cone properties a large class of properties that includes monotonicity convexity and supermodularity as well as combinations of these and many other properties of interest.