Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation

Hideki SATOH

doi:10.1093/ietfec/e91-a.4.1169

Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation

Hideki SATOH

Full Text Views

0

Share
Cite this

Summary :

An orthonormal basis adaptation method for function approximation was developed and applied to reinforcement learning with multi-dimensional continuous state space. First, a basis used for linear function approximation of a control function is set to an orthonormal basis. Next, basis elements with small activities are replaced with other candidate elements as learning progresses. As this replacement is repeated, the number of basis elements with large activities increases. Example chaos control problems for multiple logistic maps were solved, demonstrating that the method for adapting an orthonormal basis can modify a basis while holding the orthonormality in accordance with changes in the environment to improve the performance of reinforcement learning and to eliminate the adverse effects of redundant noisy states.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E91-A No.4 pp.1169-1176

Publication Date: 2008/04/01

Publicized

Online ISSN: 1745-1337

DOI: 10.1093/ietfec/e91-a.4.1169

Type of Manuscript: PAPER

Category: Nonlinear Problems

Cite this

Copy

Hideki SATOH, "Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation" in IEICE TRANSACTIONS on Fundamentals, vol. E91-A, no. 4, pp. 1169-1176, April 2008, doi: 10.1093/ietfec/e91-a.4.1169.
Abstract: An orthonormal basis adaptation method for function approximation was developed and applied to reinforcement learning with multi-dimensional continuous state space. First, a basis used for linear function approximation of a control function is set to an orthonormal basis. Next, basis elements with small activities are replaced with other candidate elements as learning progresses. As this replacement is repeated, the number of basis elements with large activities increases. Example chaos control problems for multiple logistic maps were solved, demonstrating that the method for adapting an orthonormal basis can modify a basis while holding the orthonormality in accordance with changes in the environment to improve the performance of reinforcement learning and to eliminate the adverse effects of redundant noisy states.
URL: https://globals.ieice.org/en_transactions/fundamentals/10.1093/ietfec/e91-a.4.1169/_p

Copy

@ARTICLE{e91-a_4_1169,
author={Hideki SATOH, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation},
year={2008},
volume={E91-A},
number={4},
pages={1169-1176},
abstract={An orthonormal basis adaptation method for function approximation was developed and applied to reinforcement learning with multi-dimensional continuous state space. First, a basis used for linear function approximation of a control function is set to an orthonormal basis. Next, basis elements with small activities are replaced with other candidate elements as learning progresses. As this replacement is repeated, the number of basis elements with large activities increases. Example chaos control problems for multiple logistic maps were solved, demonstrating that the method for adapting an orthonormal basis can modify a basis while holding the orthonormality in accordance with changes in the environment to improve the performance of reinforcement learning and to eliminate the adverse effects of redundant noisy states.},
keywords={},
doi={10.1093/ietfec/e91-a.4.1169},
ISSN={1745-1337},
month={April},}

Copy

TY - JOUR
TI - Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1169
EP - 1176
AU - Hideki SATOH
PY - 2008
DO - 10.1093/ietfec/e91-a.4.1169
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E91-A
IS - 4
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - April 2008
AB - An orthonormal basis adaptation method for function approximation was developed and applied to reinforcement learning with multi-dimensional continuous state space. First, a basis used for linear function approximation of a control function is set to an orthonormal basis. Next, basis elements with small activities are replaced with other candidate elements as learning progresses. As this replacement is repeated, the number of basis elements with large activities increases. Example chaos control problems for multiple logistic maps were solved, demonstrating that the method for adapting an orthonormal basis can modify a basis while holding the orthonormality in accordance with changes in the environment to improve the performance of reinforcement learning and to eliminate the adverse effects of redundant noisy states.
ER -