WEKO3
アイテム
Forward and inverse reinforcement learning sharing network weights and hyperparameters
https://oist.repo.nii.ac.jp/records/2339
https://oist.repo.nii.ac.jp/records/23394d9b084f-bb75-459c-b65b-196512c1cf81
| 名前 / ファイル | ライセンス | アクション |
|---|---|---|
1-s2.0-S0893608021003221-main (2.1 MB)
|
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (https://creativecommons.org/licenses/by-nc-nd/4.0/)
|
| Item type | 学術雑誌論文 / Journal Article(1) | |||||
|---|---|---|---|---|---|---|
| 公開日 | 2021-10-19 | |||||
| タイトル | ||||||
| タイトル | Forward and inverse reinforcement learning sharing network weights and hyperparameters | |||||
| 言語 | en | |||||
| 言語 | ||||||
| 言語 | eng | |||||
| キーワード | ||||||
| 言語 | en | |||||
| 主題Scheme | Other | |||||
| 主題 | Reinforcement learning | |||||
| キーワード | ||||||
| 言語 | en | |||||
| 主題Scheme | Other | |||||
| 主題 | Inverse reinforcement learning | |||||
| キーワード | ||||||
| 言語 | en | |||||
| 主題Scheme | Other | |||||
| 主題 | Imitation learning | |||||
| キーワード | ||||||
| 言語 | en | |||||
| 主題Scheme | Other | |||||
| 主題 | Entropy regularization | |||||
| 資源タイプ | ||||||
| 資源タイプ識別子 | http://purl.org/coar/resource_type/c_6501 | |||||
| 資源タイプ | journal article | |||||
| 著者（英） |
Uchibe, Eiji
× Uchibe, Eiji× Doya, Kenji |
|||||
| 書誌情報 |
en : Neural Networks 巻 144, p. 138-153, 発行日 2021-08-20 |
|||||
| 抄録 | ||||||
| 内容記述タイプ | Other | |||||
| 内容記述 | This paper proposes model-free imitation learning named Entropy-Regularized Imitation Learning (ERIL) that minimizes the reverse Kullback–Leibler (KL) divergence. ERIL combines forward and inverse reinforcement learning (RL) under the framework of an entropy-regularized Markov decision process. An inverse RL step computes the log-ratio between two distributions by evaluating two binary discriminators. The first discriminator distinguishes the state generated by the forward RL step from the expert’s state. The second discriminator, which is structured by the theory of entropy regularization, distinguishes the state–action–next-state tuples generated by the learner from the expert ones. One notable feature is that the second discriminator shares hyperparameters with the forward RL, which can be used to control the discriminator’s ability. A forward RL step minimizes the reverse KL estimated by the inverse RL step. We show that minimizing the reverse KL divergence is equivalent to finding an optimal policy. Our experimental results on MuJoCo-simulated environments and vision-based reaching tasks with a robotic arm show that ERIL is more sample-efficient than the baseline methods. We apply the method to human behaviors that perform a pole-balancing task and describe how the estimated reward functions show how every subject achieves her goal. | |||||
| 出版者 | ||||||
| 出版者 | Elsevier Ltd | |||||
| ISSN | ||||||
| 収録物識別子タイプ | ISSN | |||||
| 収録物識別子 | 0893-6080 | |||||
| PubMed番号 | ||||||
| 関連タイプ | isIdenticalTo | |||||
| 識別子タイプ | PMID | |||||
| 関連識別子 | info:pmid/34492548 | |||||
| DOI | ||||||
| 関連タイプ | isIdenticalTo | |||||
| 識別子タイプ | DOI | |||||
| 関連識別子 | info:doi/10.1016/j.neunet.2021.08.017 | |||||
| 権利 | ||||||
| 権利情報 | © 2021 The Author(s). | |||||
| 関連サイト | ||||||
| 識別子タイプ | URI | |||||
| 関連識別子 | https://www.sciencedirect.com/science/article/pii/S0893608021003221?via%3Dihub | |||||
| 著者版フラグ | ||||||
| 出版タイプ | VoR | |||||
| 出版タイプResource | http://purl.org/coar/version/c_970fb48d4fbd8a85 | |||||
