Bayesian methods for machine learning have been widely investigated, yielding
principled methods for incorporating prior information into inference
algorithms. In this survey, we provide an in-depth review of the role of
Bayesian methods for the reinforcement learning (RL) paradigm. The major
incentives for incorporating Bayesian reasoning in RL are: 1) it provides an
elegant approach to action-selection (exploration/exploitation) as a function
of the uncertainty in learning; and 2) it provides a machinery to incorporate
prior knowledge into the algorithms. We first discuss models and methods for
Bayesian inference in the simple single-step Bandit model. We then review the
extensive recent literature on Bayesian methods for model-based RL, where prior
information can be expressed on the parameters of the Markov model. We also
present Bayesian methods for model-free RL, where priors are expressed over the
value function or policy class. The objective of the paper is to provide a
comprehensive survey on Bayesian RL algorithms and their theoretical and
empirical properties.
Описание
[1609.04436] Bayesian Reinforcement Learning: A Survey
%0 Generic
%1 ghavamzadeh2016bayesian
%A Ghavamzadeh, Mohammad
%A Mannor, Shie
%A Pineau, Joelle
%A Tamar, Aviv
%D 2016
%K bandits reinforcementlearning
%R 10.1561/2200000049
%T Bayesian Reinforcement Learning: A Survey
%U http://arxiv.org/abs/1609.04436
%X Bayesian methods for machine learning have been widely investigated, yielding
principled methods for incorporating prior information into inference
algorithms. In this survey, we provide an in-depth review of the role of
Bayesian methods for the reinforcement learning (RL) paradigm. The major
incentives for incorporating Bayesian reasoning in RL are: 1) it provides an
elegant approach to action-selection (exploration/exploitation) as a function
of the uncertainty in learning; and 2) it provides a machinery to incorporate
prior knowledge into the algorithms. We first discuss models and methods for
Bayesian inference in the simple single-step Bandit model. We then review the
extensive recent literature on Bayesian methods for model-based RL, where prior
information can be expressed on the parameters of the Markov model. We also
present Bayesian methods for model-free RL, where priors are expressed over the
value function or policy class. The objective of the paper is to provide a
comprehensive survey on Bayesian RL algorithms and their theoretical and
empirical properties.
@misc{ghavamzadeh2016bayesian,
abstract = {Bayesian methods for machine learning have been widely investigated, yielding
principled methods for incorporating prior information into inference
algorithms. In this survey, we provide an in-depth review of the role of
Bayesian methods for the reinforcement learning (RL) paradigm. The major
incentives for incorporating Bayesian reasoning in RL are: 1) it provides an
elegant approach to action-selection (exploration/exploitation) as a function
of the uncertainty in learning; and 2) it provides a machinery to incorporate
prior knowledge into the algorithms. We first discuss models and methods for
Bayesian inference in the simple single-step Bandit model. We then review the
extensive recent literature on Bayesian methods for model-based RL, where prior
information can be expressed on the parameters of the Markov model. We also
present Bayesian methods for model-free RL, where priors are expressed over the
value function or policy class. The objective of the paper is to provide a
comprehensive survey on Bayesian RL algorithms and their theoretical and
empirical properties.},
added-at = {2020-04-06T18:19:09.000+0200},
author = {Ghavamzadeh, Mohammad and Mannor, Shie and Pineau, Joelle and Tamar, Aviv},
biburl = {https://www.bibsonomy.org/bibtex/22f0d592e43a72ba84bb754585259cb17/cpankow},
description = {[1609.04436] Bayesian Reinforcement Learning: A Survey},
doi = {10.1561/2200000049},
interhash = {e9eddfe65cf9c6df19d1c59270e03735},
intrahash = {2f0d592e43a72ba84bb754585259cb17},
keywords = {bandits reinforcementlearning},
note = {cite arxiv:1609.04436},
timestamp = {2020-04-06T18:19:09.000+0200},
title = {Bayesian Reinforcement Learning: A Survey},
url = {http://arxiv.org/abs/1609.04436},
year = 2016
}