The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
신경망은 의미 구성을 처리하는 효율성으로 인해 문장 유사성 측정 시스템에서 상당한 주목을 받아 왔습니다. 그러나 기존 신경망 방법은 입력에 묻혀 있는 가장 중요한 의미 정보를 캡처하는 데 충분히 효과적이지 않습니다. 이 문제를 해결하기 위해 가장 주목할만한 Attention 벡터를 유지하는 새로운 가중치 풀링 Attention 레이어가 제안되었습니다. 장단기 기억과 컨볼루션 신경망은 전체 문장 의미 표현의 풍부한 패턴을 축적하는 강력한 능력을 가지고 있다는 것이 이미 확립되었습니다. 먼저, 양방향 장단기 기억과 컨볼루션 신경망을 기반으로 한 샴 구조를 이용하여 문장 표현을 생성한다. 이어서, Weighted-pooling attention 레이어를 적용하여 attention 벡터를 얻습니다. 마지막으로 Attention Vector pair 정보를 활용하여 문장 유사성 점수를 계산합니다. 양방향 장단기 기억과 컨볼루션 신경망을 결합하여 정보 추출 및 학습 능력을 향상시키는 모델이 탄생했습니다. 조사에 따르면 제안된 방법은 의미론적 관련성 및 Microsoft 연구 의역 식별이라는 두 가지 작업에 대한 데이터 세트에 대한 최첨단 접근 방식보다 성능이 뛰어난 것으로 나타났습니다. 새로운 모델은 학습 능력을 향상시키고 유사성 정확도도 향상시킵니다.
Degen HUANG
Dalian University of Technology
Anil AHMED
Dalian University of Technology
Syed Yasser ARAFAT
University of Engineering and Technology (UET)
Khawaja Iftekhar RASHID
Dalian University of Technology
Qasim ABBAS
Dalian University of Technology
Fuji REN
Tokushima University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Degen HUANG, Anil AHMED, Syed Yasser ARAFAT, Khawaja Iftekhar RASHID, Qasim ABBAS, Fuji REN, "Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 10, pp. 2216-2227, October 2020, doi: 10.1587/transinf.2018EDP7410.
Abstract: Neural networks have received considerable attention in sentence similarity measuring systems due to their efficiency in dealing with semantic composition. However, existing neural network methods are not sufficiently effective in capturing the most significant semantic information buried in an input. To address this problem, a novel weighted-pooling attention layer is proposed to retain the most remarkable attention vector. It has already been established that long short-term memory and a convolution neural network have a strong ability to accumulate enriched patterns of whole sentence semantic representation. First, a sentence representation is generated by employing a siamese structure based on bidirectional long short-term memory and a convolutional neural network. Subsequently, a weighted-pooling attention layer is applied to obtain an attention vector. Finally, the attention vector pair information is leveraged to calculate the score of sentence similarity. An amalgamation of both, bidirectional long short-term memory and a convolutional neural network has resulted in a model that enhances information extracting and learning capacity. Investigations show that the proposed method outperforms the state-of-the-art approaches to datasets for two tasks, namely semantic relatedness and Microsoft research paraphrase identification. The new model improves the learning capability and also boosts the similarity accuracy as well.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2018EDP7410/_p
부
@ARTICLE{e103-d_10_2216,
author={Degen HUANG, Anil AHMED, Syed Yasser ARAFAT, Khawaja Iftekhar RASHID, Qasim ABBAS, Fuji REN, },
journal={IEICE TRANSACTIONS on Information},
title={Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention},
year={2020},
volume={E103-D},
number={10},
pages={2216-2227},
abstract={Neural networks have received considerable attention in sentence similarity measuring systems due to their efficiency in dealing with semantic composition. However, existing neural network methods are not sufficiently effective in capturing the most significant semantic information buried in an input. To address this problem, a novel weighted-pooling attention layer is proposed to retain the most remarkable attention vector. It has already been established that long short-term memory and a convolution neural network have a strong ability to accumulate enriched patterns of whole sentence semantic representation. First, a sentence representation is generated by employing a siamese structure based on bidirectional long short-term memory and a convolutional neural network. Subsequently, a weighted-pooling attention layer is applied to obtain an attention vector. Finally, the attention vector pair information is leveraged to calculate the score of sentence similarity. An amalgamation of both, bidirectional long short-term memory and a convolutional neural network has resulted in a model that enhances information extracting and learning capacity. Investigations show that the proposed method outperforms the state-of-the-art approaches to datasets for two tasks, namely semantic relatedness and Microsoft research paraphrase identification. The new model improves the learning capability and also boosts the similarity accuracy as well.},
keywords={},
doi={10.1587/transinf.2018EDP7410},
ISSN={1745-1361},
month={October},}
부
TY - JOUR
TI - Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention
T2 - IEICE TRANSACTIONS on Information
SP - 2216
EP - 2227
AU - Degen HUANG
AU - Anil AHMED
AU - Syed Yasser ARAFAT
AU - Khawaja Iftekhar RASHID
AU - Qasim ABBAS
AU - Fuji REN
PY - 2020
DO - 10.1587/transinf.2018EDP7410
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2020
AB - Neural networks have received considerable attention in sentence similarity measuring systems due to their efficiency in dealing with semantic composition. However, existing neural network methods are not sufficiently effective in capturing the most significant semantic information buried in an input. To address this problem, a novel weighted-pooling attention layer is proposed to retain the most remarkable attention vector. It has already been established that long short-term memory and a convolution neural network have a strong ability to accumulate enriched patterns of whole sentence semantic representation. First, a sentence representation is generated by employing a siamese structure based on bidirectional long short-term memory and a convolutional neural network. Subsequently, a weighted-pooling attention layer is applied to obtain an attention vector. Finally, the attention vector pair information is leveraged to calculate the score of sentence similarity. An amalgamation of both, bidirectional long short-term memory and a convolutional neural network has resulted in a model that enhances information extracting and learning capacity. Investigations show that the proposed method outperforms the state-of-the-art approaches to datasets for two tasks, namely semantic relatedness and Microsoft research paraphrase identification. The new model improves the learning capability and also boosts the similarity accuracy as well.
ER -