Advanced search
1 file | 169.63 KB

Optimal control and optimal sensor activation for Markov decision problems with costly observations

Author
Organization
Abstract
This paper considers partial observation Markov decision processes. Besides the classical control decisions influencing the transition probabilities of the Markov process, we also consider control actions that can activate the sensors to provide more or less accurate information about the system state, explicitly including the cost of activating sensors. We synthesize control laws that minimize a discounted operating cost of the system over an infinite interval of time, where the instantaneous cost function depends on the current state, the control influencing the transition probabilities, and the control actions activating the sensors. A general computationally efficient optimal solution for this problem is not known. Hence we design supoptimal controllers that only use knowledge of the value function for the full state information Markov decision problem. Our solution guarantees that the discounted cost of operating the plant increases only by a bounded amount with respect to the minimal cost for the full state information problem. A new concept of pinned conditional distributions of the state given the observed history of the plant is required in order to implement these control laws online.
Keywords
POMDP, active sensor control, stochastic control, partial information control, suboptimal feedback control

Downloads

  • MSCpaper0216.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 169.63 KB

Citation

Please use this url to cite or link to this publication:

Chicago
Boel, René, and Jan H. van Schuppen. 2015. “Optimal Control and Optimal Sensor Activation for Markov Decision Problems with Costly Observations.” In IEEE International Conference on Control Applications, 1140–1147. Sydney: IEEE.
APA
Boel, R., & van Schuppen, J. H. (2015). Optimal control and optimal sensor activation for Markov decision problems with costly observations. IEEE International Conference on Control Applications (pp. 1140–1147). Presented at the IEEE Conference on Control and Applications (CCA), Sydney: IEEE.
Vancouver
1.
Boel R, van Schuppen JH. Optimal control and optimal sensor activation for Markov decision problems with costly observations. IEEE International Conference on Control Applications. Sydney: IEEE; 2015. p. 1140–7.
MLA
Boel, René, and Jan H. van Schuppen. “Optimal Control and Optimal Sensor Activation for Markov Decision Problems with Costly Observations.” IEEE International Conference on Control Applications. Sydney: IEEE, 2015. 1140–1147. Print.
@inproceedings{8082628,
  abstract     = {This paper considers partial observation Markov decision processes. Besides the classical control decisions influencing the transition probabilities of the Markov process, we also consider control actions that can activate the sensors to provide more or less accurate information about the system state, explicitly including the cost of activating sensors. We synthesize control laws that minimize a discounted operating cost of the system over an infinite interval of time, where the instantaneous cost function depends on the current state, the control influencing the transition probabilities, and the control actions activating the sensors. A general computationally efficient optimal solution for this problem is not known. Hence we design supoptimal controllers that only use knowledge of the value function for the full state information Markov decision problem. Our solution guarantees that the discounted cost of operating the plant increases only by a bounded amount with respect to the minimal cost for the full state information problem. A new concept of pinned conditional distributions of the state given the observed history of the plant is required in order to implement these control laws online.},
  author       = {Boel, Ren{\'e} and van Schuppen, Jan H.},
  booktitle    = {IEEE International Conference on Control Applications},
  isbn         = {978-1-4799-7786-4},
  language     = {eng},
  location     = {Sydney, AUSTRALIA},
  pages        = {1140--1147},
  publisher    = {IEEE},
  title        = {Optimal control and optimal sensor activation for Markov decision problems with costly observations},
  year         = {2015},
}

Web of Science
Times cited: