The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
기존의 단순 라우팅 프로토콜(예: OSPF, RIP)은 유연성이 없고 특정 라우터에 패킷이 집중되어 혼잡이 발생하기 쉽다는 몇 가지 단점이 있습니다. 이러한 문제를 해결하기 위해 최근에는 머신러닝을 활용한 패킷 라우팅 방법이 제안되었습니다. 이러한 알고리즘에 비해 기계 학습 기반 방법은 효율적인 경로를 학습하여 지능적으로 라우팅 경로를 선택할 수 있습니다. 그러나 머신러닝 기반 방법은 학습 시간 오버헤드가 발생한다는 단점이 있습니다. 따라서 우리는 훈련 시간을 줄이기 위해 경량 기계 학습 알고리즘인 OS-ELM(Online Sequential Extreme Learning Machine)에 중점을 둡니다. OS-ELM을 이용한 강화학습에 대한 이전 연구가 존재하지만, 학습 정확도가 낮은 문제가 있습니다. 본 논문에서는 학습 성능 향상을 위해 우선순위화된 경험 재생 버퍼를 갖춘 OS-ELM QN(Q-Network)을 제안합니다. 네트워크 시뮬레이터를 이용한 심층 강화학습 기반 패킷 라우팅 방법과 비교됩니다. 실험 결과는 경험 재생 버퍼를 도입하면 학습 성능이 향상된다는 것을 보여줍니다. OS-ELM QN은 학습 속도 측면에서 DQN(Deep Q-Network)보다 2.33배 빠른 속도를 달성합니다. 패킷 전송 지연 시간과 관련하여 OS-ELM QN은 DQN과 비슷하거나 약간 열등하지만 혼잡을 분산시킬 수 있다는 점에서 대부분의 경우 OSPF보다 우수합니다.
Kenji NEMOTO
Keio University
Hiroki MATSUTANI
Keio University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Kenji NEMOTO, Hiroki MATSUTANI, "A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning" in IEICE TRANSACTIONS on Information,
vol. E106-D, no. 11, pp. 1796-1807, November 2023, doi: 10.1587/transinf.2022EDP7231.
Abstract: Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7231/_p
부
@ARTICLE{e106-d_11_1796,
author={Kenji NEMOTO, Hiroki MATSUTANI, },
journal={IEICE TRANSACTIONS on Information},
title={A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning},
year={2023},
volume={E106-D},
number={11},
pages={1796-1807},
abstract={Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.},
keywords={},
doi={10.1587/transinf.2022EDP7231},
ISSN={1745-1361},
month={November},}
부
TY - JOUR
TI - A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning
T2 - IEICE TRANSACTIONS on Information
SP - 1796
EP - 1807
AU - Kenji NEMOTO
AU - Hiroki MATSUTANI
PY - 2023
DO - 10.1587/transinf.2022EDP7231
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 11
JA - IEICE TRANSACTIONS on Information
Y1 - November 2023
AB - Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.
ER -