Selector-Actor-Critic and Tuner-Actor-Critic Algorithms for Reinforcement Learning

Wang, Zhengdao
Masadeh, Ala'eddin
Wang, Zhengdao
Kamal, Ahmed
Masadeh, Ala’eddin
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Journal Issue

This work presents two reinforcement learning (RL) architectures, which mimic rational humans in the way of analyzing the available information and making decisions. The proposed algorithms are called selector-actor-critic (SAC) and tuner-actor-critic (TAC). They are obtained by modifying the well known actor-critic (AC) algorithm. SAC is equipped with an actor, a critic, and a selector. The role of the selector is to determine the most promising action at the current state based on the last estimate from the critic. TAC is model based, and consists of a tuner, a model-learner, an actor, and a critic. After receiving the approximated value of the current state-action pair from the critic and the learned model from the model-learner, the tuner uses the Bellman equation to tune the value of the current state-action pair. Then, this tuned value is used by the actor to optimize the policy. We investigate the performance of the proposed algorithms, and compare with AC algorithm to show the advantages of the proposed algorithms using numerical simulations.

<p>This is a manuscript of a proceeding published as Masadeh, Ala'eddin, Zhengdao Wang, and Ahmed E. Kamal. "Selector-Actor-Critic and Tuner-Actor-Critic Algorithms for Reinforcement Learning." In <em>2019 11th International Conference on Wireless Communications and Signal Processing (WCSP)</em>. DOI: <a href="" target="_blank">10.1109/WCSP.2019.8928124</a>. Posted with permission.</p>
Reinforcement learning, model-based learning, model-free learning, actor-critic