Small Number of Hidden Units for ELM with Two-Stage Linear Model

Hieu Trung HUYNH; Yonggwan WON

doi:10.1093/ietisy/e91-d.4.1042

Small Number of Hidden Units for ELM with Two-Stage Linear Model

Hieu Trung HUYNH, Yonggwan WON

Full Text Views

0

Share
Cite this

Summary :

The single-hidden-layer feedforward neural networks (SLFNs) are frequently used in machine learning due to their ability which can form boundaries with arbitrary shapes if the activation function of hidden units is chosen properly. Most learning algorithms for the neural networks based on gradient descent are still slow because of the many learning steps. Recently, a learning algorithm called extreme learning machine (ELM) has been proposed for training SLFNs to overcome this problem. It randomly chooses the input weights and hidden-layer biases, and analytically determines the output weights by the matrix inverse operation. This algorithm can achieve good generalization performance with high learning speed in many applications. However, this algorithm often requires a large number of hidden units and takes long time for classification of new observations. In this paper, a new approach for training SLFNs called least-squares extreme learning machine (LS-ELM) is proposed. Unlike the gradient descent-based algorithms and the ELM, our approach analytically determines the input weights, hidden-layer biases and output weights based on linear models. For training with a large number of input patterns, an online training scheme with sub-blocks of the training set is also introduced. Experimental results for real applications show that our proposed algorithm offers high classification accuracy with a smaller number of hidden units and extremely high speed in both learning and testing.

Publication: IEICE TRANSACTIONS on Information Vol.E91-D No.4 pp.1042-1049

Publication Date: 2008/04/01

Publicized

Online ISSN: 1745-1361

DOI: 10.1093/ietisy/e91-d.4.1042

Type of Manuscript: PAPER

Category: Data Mining

Cite this

Copy

Hieu Trung HUYNH, Yonggwan WON, "Small Number of Hidden Units for ELM with Two-Stage Linear Model" in IEICE TRANSACTIONS on Information, vol. E91-D, no. 4, pp. 1042-1049, April 2008, doi: 10.1093/ietisy/e91-d.4.1042.
Abstract: The single-hidden-layer feedforward neural networks (SLFNs) are frequently used in machine learning due to their ability which can form boundaries with arbitrary shapes if the activation function of hidden units is chosen properly. Most learning algorithms for the neural networks based on gradient descent are still slow because of the many learning steps. Recently, a learning algorithm called extreme learning machine (ELM) has been proposed for training SLFNs to overcome this problem. It randomly chooses the input weights and hidden-layer biases, and analytically determines the output weights by the matrix inverse operation. This algorithm can achieve good generalization performance with high learning speed in many applications. However, this algorithm often requires a large number of hidden units and takes long time for classification of new observations. In this paper, a new approach for training SLFNs called least-squares extreme learning machine (LS-ELM) is proposed. Unlike the gradient descent-based algorithms and the ELM, our approach analytically determines the input weights, hidden-layer biases and output weights based on linear models. For training with a large number of input patterns, an online training scheme with sub-blocks of the training set is also introduced. Experimental results for real applications show that our proposed algorithm offers high classification accuracy with a smaller number of hidden units and extremely high speed in both learning and testing.
URL: https://globals.ieice.org/en_transactions/information/10.1093/ietisy/e91-d.4.1042/_p

Copy

@ARTICLE{e91-d_4_1042,
author={Hieu Trung HUYNH, Yonggwan WON, },
journal={IEICE TRANSACTIONS on Information},
title={Small Number of Hidden Units for ELM with Two-Stage Linear Model},
year={2008},
volume={E91-D},
number={4},
pages={1042-1049},
abstract={The single-hidden-layer feedforward neural networks (SLFNs) are frequently used in machine learning due to their ability which can form boundaries with arbitrary shapes if the activation function of hidden units is chosen properly. Most learning algorithms for the neural networks based on gradient descent are still slow because of the many learning steps. Recently, a learning algorithm called extreme learning machine (ELM) has been proposed for training SLFNs to overcome this problem. It randomly chooses the input weights and hidden-layer biases, and analytically determines the output weights by the matrix inverse operation. This algorithm can achieve good generalization performance with high learning speed in many applications. However, this algorithm often requires a large number of hidden units and takes long time for classification of new observations. In this paper, a new approach for training SLFNs called least-squares extreme learning machine (LS-ELM) is proposed. Unlike the gradient descent-based algorithms and the ELM, our approach analytically determines the input weights, hidden-layer biases and output weights based on linear models. For training with a large number of input patterns, an online training scheme with sub-blocks of the training set is also introduced. Experimental results for real applications show that our proposed algorithm offers high classification accuracy with a smaller number of hidden units and extremely high speed in both learning and testing.},
keywords={},
doi={10.1093/ietisy/e91-d.4.1042},
ISSN={1745-1361},
month={April},}

Copy

TY - JOUR
TI - Small Number of Hidden Units for ELM with Two-Stage Linear Model
T2 - IEICE TRANSACTIONS on Information
SP - 1042
EP - 1049
AU - Hieu Trung HUYNH
AU - Yonggwan WON
PY - 2008
DO - 10.1093/ietisy/e91-d.4.1042
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E91-D
IS - 4
JA - IEICE TRANSACTIONS on Information
Y1 - April 2008
AB - The single-hidden-layer feedforward neural networks (SLFNs) are frequently used in machine learning due to their ability which can form boundaries with arbitrary shapes if the activation function of hidden units is chosen properly. Most learning algorithms for the neural networks based on gradient descent are still slow because of the many learning steps. Recently, a learning algorithm called extreme learning machine (ELM) has been proposed for training SLFNs to overcome this problem. It randomly chooses the input weights and hidden-layer biases, and analytically determines the output weights by the matrix inverse operation. This algorithm can achieve good generalization performance with high learning speed in many applications. However, this algorithm often requires a large number of hidden units and takes long time for classification of new observations. In this paper, a new approach for training SLFNs called least-squares extreme learning machine (LS-ELM) is proposed. Unlike the gradient descent-based algorithms and the ELM, our approach analytically determines the input weights, hidden-layer biases and output weights based on linear models. For training with a large number of input patterns, an online training scheme with sub-blocks of the training set is also introduced. Experimental results for real applications show that our proposed algorithm offers high classification accuracy with a smaller number of hidden units and extremely high speed in both learning and testing.
ER -