The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
RNN(Recurrent Neural Network)은 시간 및 순차 데이터와 관련된 다양하고 복잡한 작업에서 많은 최첨단 성능을 달성했습니다. 그러나 대부분의 RNN에는 훈련 및 추론 단계 모두에 많은 계산 능력과 엄청난 수의 매개 변수가 필요합니다. CANDECOMP/PARAFAC(CP), Tucker 분해 및 Tensor Train(TT)과 같은 여러 텐서 분해 방법이 포함되어 GRU(Gated Recurrent Unit) RNN을 다시 매개변수화합니다. 먼저, 다양한 수의 매개변수를 사용하여 시퀀스 모델링 작업에 대한 모든 텐서 기반 RNN 성능을 평가합니다. 실험 결과에 따르면 TT-GRU는 다른 분해 방법에 비해 다양한 매개 변수에서 최상의 결과를 얻었습니다. 나중에 우리는 제안된 TT-GRU를 음성 인식 작업으로 평가합니다. 우리는 DeepSpeech2 아키텍처 내에서 양방향 GRU 레이어를 압축했습니다. 실험 결과를 바탕으로 제안한 TT 형식 GRU는 압축되지 않은 GRU에 비해 GRU 매개 변수 수를 크게 줄이면서 성능을 유지할 수 있습니다.
Andros TJANDRA
Nara Institute of Science and Technology,RIKEN, Center for Advanced Intelligence Project AIP
Sakriani SAKTI
Nara Institute of Science and Technology,RIKEN, Center for Advanced Intelligence Project AIP
Satoshi NAKAMURA
Nara Institute of Science and Technology,RIKEN, Center for Advanced Intelligence Project AIP
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Andros TJANDRA, Sakriani SAKTI, Satoshi NAKAMURA, "Recurrent Neural Network Compression Based on Low-Rank Tensor Representation" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 2, pp. 435-449, February 2020, doi: 10.1587/transinf.2019EDP7040.
Abstract: Recurrent Neural Network (RNN) has achieved many state-of-the-art performances on various complex tasks related to the temporal and sequential data. But most of these RNNs require much computational power and a huge number of parameters for both training and inference stage. Several tensor decomposition methods are included such as CANDECOMP/PARAFAC (CP), Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated Recurrent Unit (GRU) RNN. First, we evaluate all tensor-based RNNs performance on sequence modeling tasks with a various number of parameters. Based on our experiment results, TT-GRU achieved the best results in a various number of parameters compared to other decomposition methods. Later, we evaluate our proposed TT-GRU with speech recognition task. We compressed the bidirectional GRU layers inside DeepSpeech2 architecture. Based on our experiment result, our proposed TT-format GRU are able to preserve the performance while reducing the number of GRU parameters significantly compared to the uncompressed GRU.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7040/_p
부
@ARTICLE{e103-d_2_435,
author={Andros TJANDRA, Sakriani SAKTI, Satoshi NAKAMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Recurrent Neural Network Compression Based on Low-Rank Tensor Representation},
year={2020},
volume={E103-D},
number={2},
pages={435-449},
abstract={Recurrent Neural Network (RNN) has achieved many state-of-the-art performances on various complex tasks related to the temporal and sequential data. But most of these RNNs require much computational power and a huge number of parameters for both training and inference stage. Several tensor decomposition methods are included such as CANDECOMP/PARAFAC (CP), Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated Recurrent Unit (GRU) RNN. First, we evaluate all tensor-based RNNs performance on sequence modeling tasks with a various number of parameters. Based on our experiment results, TT-GRU achieved the best results in a various number of parameters compared to other decomposition methods. Later, we evaluate our proposed TT-GRU with speech recognition task. We compressed the bidirectional GRU layers inside DeepSpeech2 architecture. Based on our experiment result, our proposed TT-format GRU are able to preserve the performance while reducing the number of GRU parameters significantly compared to the uncompressed GRU.},
keywords={},
doi={10.1587/transinf.2019EDP7040},
ISSN={1745-1361},
month={February},}
부
TY - JOUR
TI - Recurrent Neural Network Compression Based on Low-Rank Tensor Representation
T2 - IEICE TRANSACTIONS on Information
SP - 435
EP - 449
AU - Andros TJANDRA
AU - Sakriani SAKTI
AU - Satoshi NAKAMURA
PY - 2020
DO - 10.1587/transinf.2019EDP7040
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 2
JA - IEICE TRANSACTIONS on Information
Y1 - February 2020
AB - Recurrent Neural Network (RNN) has achieved many state-of-the-art performances on various complex tasks related to the temporal and sequential data. But most of these RNNs require much computational power and a huge number of parameters for both training and inference stage. Several tensor decomposition methods are included such as CANDECOMP/PARAFAC (CP), Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated Recurrent Unit (GRU) RNN. First, we evaluate all tensor-based RNNs performance on sequence modeling tasks with a various number of parameters. Based on our experiment results, TT-GRU achieved the best results in a various number of parameters compared to other decomposition methods. Later, we evaluate our proposed TT-GRU with speech recognition task. We compressed the bidirectional GRU layers inside DeepSpeech2 architecture. Based on our experiment result, our proposed TT-format GRU are able to preserve the performance while reducing the number of GRU parameters significantly compared to the uncompressed GRU.
ER -