The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
본 논문에서는 통계적 기계적 방법을 사용하여 온라인 학습 프레임워크에서 상관 입력을 사용한 학습의 일반화 성능을 분석적으로 조사합니다. 가우스 노이즈가 있는 선형 퍼셉트론으로 구성된 모델을 고려합니다. 먼저 경사법의 사례를 분석한다. 입력 간의 상관관계가 클수록, 입력 개수가 많을수록 학습률이 만족해야 하는 조건은 엄격해지고, 학습 속도는 느려진다는 것을 분석적으로 밝혔습니다. 둘째, 블록 직교 투영 학습을 대안 학습 규칙으로 취급하여 이론을 도출한다. 노이즈가 없는 경우 학습 속도는 상관 관계에 의존하지 않으며 업데이트에 사용되는 입력 수에 비례합니다. 학습 속도는 상관되지 않은 입력을 사용하는 그래디언트 방법과 동일합니다. 반면, 노이즈가 있는 경우에는 입력 간의 상관관계가 클수록 학습 속도가 느려지고 잔여 일반화 오류가 커집니다.
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Chihiro SEKI, Shingo SAKURAI, Masafumi MATSUNO, Seiji MIYOSHI, "A Theoretical Analysis of On-Line Learning Using Correlated Examples" in IEICE TRANSACTIONS on Fundamentals,
vol. E91-A, no. 9, pp. 2663-2670, September 2008, doi: 10.1093/ietfec/e91-a.9.2663.
Abstract: In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1093/ietfec/e91-a.9.2663/_p
부
@ARTICLE{e91-a_9_2663,
author={Chihiro SEKI, Shingo SAKURAI, Masafumi MATSUNO, Seiji MIYOSHI, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={A Theoretical Analysis of On-Line Learning Using Correlated Examples},
year={2008},
volume={E91-A},
number={9},
pages={2663-2670},
abstract={In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.},
keywords={},
doi={10.1093/ietfec/e91-a.9.2663},
ISSN={1745-1337},
month={September},}
부
TY - JOUR
TI - A Theoretical Analysis of On-Line Learning Using Correlated Examples
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 2663
EP - 2670
AU - Chihiro SEKI
AU - Shingo SAKURAI
AU - Masafumi MATSUNO
AU - Seiji MIYOSHI
PY - 2008
DO - 10.1093/ietfec/e91-a.9.2663
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E91-A
IS - 9
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - September 2008
AB - In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
ER -