The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
고성능 DNN(심층 신경망) 기반 시스템은 엣지 환경에서 수요가 높습니다. 높은 계산 복잡성으로 인해 계산 리소스에 대한 엄격한 제한이 있는 엣지 장치에 DNN을 배포하는 것은 어렵습니다. 본 논문에서는 최근 제안된 매개변수 감소 기법인 Neural ODE(Ordinary Differential Equation)와 DSC(Depthwise Separable Convolution)를 결합하여 dsODENet이라는 간결하면서도 매우 정확한 DNN 모델을 도출합니다. Neural ODE는 ResNet과 ODE 간의 유사성을 활용하고 여러 계층에서 대부분의 가중치 매개변수를 공유하므로 메모리 소비가 크게 줄어듭니다. 우리는 이미지 분류 데이터 세트의 실제 사용 사례로 dsODENet을 도메인 적응에 적용합니다. 우리는 또한 전처리 및 후처리 레이어를 제외한 모든 매개변수와 기능 맵을 온칩 메모리에 매핑할 수 있는 dsODENet을 위한 리소스 효율적인 FPGA 기반 설계를 제안합니다. Xilinx ZCU104 보드에 구현되며 도메인 적응 정확도, 추론 속도, FPGA 리소스 활용도 및 소프트웨어 대비 속도 향상 속도 측면에서 평가됩니다. 결과는 dsODENet이 기본 Neural ODE 구현과 비교하여 비슷하거나 약간 더 나은 도메인 적응 정확도를 달성하는 동시에 전처리 및 후처리 레이어가 없는 전체 매개변수 크기가 54.2%에서 79.8%로 감소했음을 보여줍니다. 우리의 FPGA 구현은 추론 속도를 23.8배 가속화합니다.
Hiroki KAWAKAMI
Keio University
Hirohisa WATANABE
Keio University
Keisuke SUGIURA
Keio University
Hiroki MATSUTANI
Keio University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Hiroki KAWAKAMI, Hirohisa WATANABE, Keisuke SUGIURA, Hiroki MATSUTANI, "A Low-Cost Neural ODE with Depthwise Separable Convolution for Edge Domain Adaptation on FPGAs" in IEICE TRANSACTIONS on Information,
vol. E106-D, no. 7, pp. 1186-1197, July 2023, doi: 10.1587/transinf.2022EDP7149.
Abstract: High-performance deep neural network (DNN)-based systems are in high demand in edge environments. Due to its high computational complexity, it is challenging to deploy DNNs on edge devices with strict limitations on computational resources. In this paper, we derive a compact while highly-accurate DNN model, termed dsODENet, by combining recently-proposed parameter reduction techniques: Neural ODE (Ordinary Differential Equation) and DSC (Depthwise Separable Convolution). Neural ODE exploits a similarity between ResNet and ODE, and shares most of weight parameters among multiple layers, which greatly reduces the memory consumption. We apply dsODENet to a domain adaptation as a practical use case with image classification datasets. We also propose a resource-efficient FPGA-based design for dsODENet, where all the parameters and feature maps except for pre- and post-processing layers can be mapped onto on-chip memories. It is implemented on Xilinx ZCU104 board and evaluated in terms of domain adaptation accuracy, inference speed, FPGA resource utilization, and speedup rate compared to a software counterpart. The results demonstrate that dsODENet achieves comparable or slightly better domain adaptation accuracy compared to our baseline Neural ODE implementation, while the total parameter size without pre- and post-processing layers is reduced by 54.2% to 79.8%. Our FPGA implementation accelerates the inference speed by 23.8 times.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7149/_p
부
@ARTICLE{e106-d_7_1186,
author={Hiroki KAWAKAMI, Hirohisa WATANABE, Keisuke SUGIURA, Hiroki MATSUTANI, },
journal={IEICE TRANSACTIONS on Information},
title={A Low-Cost Neural ODE with Depthwise Separable Convolution for Edge Domain Adaptation on FPGAs},
year={2023},
volume={E106-D},
number={7},
pages={1186-1197},
abstract={High-performance deep neural network (DNN)-based systems are in high demand in edge environments. Due to its high computational complexity, it is challenging to deploy DNNs on edge devices with strict limitations on computational resources. In this paper, we derive a compact while highly-accurate DNN model, termed dsODENet, by combining recently-proposed parameter reduction techniques: Neural ODE (Ordinary Differential Equation) and DSC (Depthwise Separable Convolution). Neural ODE exploits a similarity between ResNet and ODE, and shares most of weight parameters among multiple layers, which greatly reduces the memory consumption. We apply dsODENet to a domain adaptation as a practical use case with image classification datasets. We also propose a resource-efficient FPGA-based design for dsODENet, where all the parameters and feature maps except for pre- and post-processing layers can be mapped onto on-chip memories. It is implemented on Xilinx ZCU104 board and evaluated in terms of domain adaptation accuracy, inference speed, FPGA resource utilization, and speedup rate compared to a software counterpart. The results demonstrate that dsODENet achieves comparable or slightly better domain adaptation accuracy compared to our baseline Neural ODE implementation, while the total parameter size without pre- and post-processing layers is reduced by 54.2% to 79.8%. Our FPGA implementation accelerates the inference speed by 23.8 times.},
keywords={},
doi={10.1587/transinf.2022EDP7149},
ISSN={1745-1361},
month={July},}
부
TY - JOUR
TI - A Low-Cost Neural ODE with Depthwise Separable Convolution for Edge Domain Adaptation on FPGAs
T2 - IEICE TRANSACTIONS on Information
SP - 1186
EP - 1197
AU - Hiroki KAWAKAMI
AU - Hirohisa WATANABE
AU - Keisuke SUGIURA
AU - Hiroki MATSUTANI
PY - 2023
DO - 10.1587/transinf.2022EDP7149
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2023
AB - High-performance deep neural network (DNN)-based systems are in high demand in edge environments. Due to its high computational complexity, it is challenging to deploy DNNs on edge devices with strict limitations on computational resources. In this paper, we derive a compact while highly-accurate DNN model, termed dsODENet, by combining recently-proposed parameter reduction techniques: Neural ODE (Ordinary Differential Equation) and DSC (Depthwise Separable Convolution). Neural ODE exploits a similarity between ResNet and ODE, and shares most of weight parameters among multiple layers, which greatly reduces the memory consumption. We apply dsODENet to a domain adaptation as a practical use case with image classification datasets. We also propose a resource-efficient FPGA-based design for dsODENet, where all the parameters and feature maps except for pre- and post-processing layers can be mapped onto on-chip memories. It is implemented on Xilinx ZCU104 board and evaluated in terms of domain adaptation accuracy, inference speed, FPGA resource utilization, and speedup rate compared to a software counterpart. The results demonstrate that dsODENet achieves comparable or slightly better domain adaptation accuracy compared to our baseline Neural ODE implementation, while the total parameter size without pre- and post-processing layers is reduced by 54.2% to 79.8%. Our FPGA implementation accelerates the inference speed by 23.8 times.
ER -