A State Space Compression Method Based on Multivariate Analysis for Reinforcement Learning in High-Dimensional Continuous State Spaces

Hideki SATOH

doi:10.1093/ietfec/e89-a.8.2181

A State Space Compression Method Based on Multivariate Analysis for Reinforcement Learning in High-Dimensional Continuous State Spaces

Hideki SATOH

Full Text Views

0

Share
Cite this

Summary :

A state space compression method based on multivariate analysis was developed and applied to reinforcement learning for high-dimensional continuous state spaces. First, useful components in the state variables of an environment are extracted and meaningless ones are removed by using multiple regression analysis. Next, the state space of the environment is compressed by using principal component analysis so that only a few principal components can express the dynamics of the environment. Then, a basis of a feature space for function approximation is constructed based on orthonormal bases of the important principal components. A feature space is thus autonomously construct without preliminary knowledge of the environment, and the environment is effectively expressed in the feature space. An example synchronization problem for multiple logistic maps was solved using this method, demonstrating that it solves the curse of dimensionality and exhibits high performance without suffering from disturbance states.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E89-A No.8 pp.2181-2191

Publication Date: 2006/08/01

Publicized

Online ISSN: 1745-1337

DOI: 10.1093/ietfec/e89-a.8.2181

Type of Manuscript: PAPER

Category: Nonlinear Problems

Cite this

Copy

Hideki SATOH, "A State Space Compression Method Based on Multivariate Analysis for Reinforcement Learning in High-Dimensional Continuous State Spaces" in IEICE TRANSACTIONS on Fundamentals, vol. E89-A, no. 8, pp. 2181-2191, August 2006, doi: 10.1093/ietfec/e89-a.8.2181.
Abstract: A state space compression method based on multivariate analysis was developed and applied to reinforcement learning for high-dimensional continuous state spaces. First, useful components in the state variables of an environment are extracted and meaningless ones are removed by using multiple regression analysis. Next, the state space of the environment is compressed by using principal component analysis so that only a few principal components can express the dynamics of the environment. Then, a basis of a feature space for function approximation is constructed based on orthonormal bases of the important principal components. A feature space is thus autonomously construct without preliminary knowledge of the environment, and the environment is effectively expressed in the feature space. An example synchronization problem for multiple logistic maps was solved using this method, demonstrating that it solves the curse of dimensionality and exhibits high performance without suffering from disturbance states.
URL: https://globals.ieice.org/en_transactions/fundamentals/10.1093/ietfec/e89-a.8.2181/_p

Copy

@ARTICLE{e89-a_8_2181,
author={Hideki SATOH, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={A State Space Compression Method Based on Multivariate Analysis for Reinforcement Learning in High-Dimensional Continuous State Spaces},
year={2006},
volume={E89-A},
number={8},
pages={2181-2191},
abstract={A state space compression method based on multivariate analysis was developed and applied to reinforcement learning for high-dimensional continuous state spaces. First, useful components in the state variables of an environment are extracted and meaningless ones are removed by using multiple regression analysis. Next, the state space of the environment is compressed by using principal component analysis so that only a few principal components can express the dynamics of the environment. Then, a basis of a feature space for function approximation is constructed based on orthonormal bases of the important principal components. A feature space is thus autonomously construct without preliminary knowledge of the environment, and the environment is effectively expressed in the feature space. An example synchronization problem for multiple logistic maps was solved using this method, demonstrating that it solves the curse of dimensionality and exhibits high performance without suffering from disturbance states.},
keywords={},
doi={10.1093/ietfec/e89-a.8.2181},
ISSN={1745-1337},
month={August},}

Copy

TY - JOUR
TI - A State Space Compression Method Based on Multivariate Analysis for Reinforcement Learning in High-Dimensional Continuous State Spaces
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 2181
EP - 2191
AU - Hideki SATOH
PY - 2006
DO - 10.1093/ietfec/e89-a.8.2181
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E89-A
IS - 8
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - August 2006
AB - A state space compression method based on multivariate analysis was developed and applied to reinforcement learning for high-dimensional continuous state spaces. First, useful components in the state variables of an environment are extracted and meaningless ones are removed by using multiple regression analysis. Next, the state space of the environment is compressed by using principal component analysis so that only a few principal components can express the dynamics of the environment. Then, a basis of a feature space for function approximation is constructed based on orthonormal bases of the important principal components. A feature space is thus autonomously construct without preliminary knowledge of the environment, and the environment is effectively expressed in the feature space. An example synchronization problem for multiple logistic maps was solved using this method, demonstrating that it solves the curse of dimensionality and exhibits high performance without suffering from disturbance states.
ER -