Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes

This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, finite-state Markov decision processes (MDPs). We synthesize optimal policies with respect to both criteria, both for the finite horizon and the discounted infinite horizon problem. A generalized decision-making framework is introduced, which includes as special cases a number of approaches that have been considered in the literature. The framework allows for discounted risk-sensitive and minimax formulations leading to stationary optimal policies on the infinite horizon. We illustrate our results with a simple machine replacement problem. (C) 1999 Elsevier Science Ltd. All rights reserved.