Extending the Scope of Hindsight Optimization for Emergency Response Planning
Hybrid State and Action Markov Decision Processes (HSA-MDPs) provide an expressive formal model for large sequential decision-making problems with both continuous and discrete state and action variables. The previous work solves HSA-MDPs by a domain-independent approach based on Hindsight optimization (HOP) [Raghavan, Ph.D. Dissertation, Oregon State University, 2017]. The main idea behind this method is a linear time reduction of the objective function and PWL dynamics to a Mixed Integer Linear Program (MILP). The current work extends the capability of HOP-MILP solver and has two primary contributions. In our first contribution, we extend the scope of HOP-MILP solver in three dimensions. a) We propose an approach for incorporating non-piece wise linear functions(NPWL) in objective and dynamics of state and action variables. This is achieved by symbolic approximation of NPWL as a PWL function. b) We extend the solver ability to handle ENUM type definition. This allows the HOP-MILP planner to solve extensive domain definitions. c) We propose an approach for selecting hyper parameters in HOP algorithm. Our second primary contribution is the evaluation of three variants of HOP algorithm in the Emergency Response domain for the City of Corvallis with some interesting insights.
Co Advisor: ALAN FERN
Co Advisor: PRASAD TADEPALLI
Committee: XAIO FU
Thursday, March 21 at 9:00am to 10:00am
Kelley Engineering Center, 1114
110 SW Park Terrace, Corvallis, OR 97331