The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
전방향 이미지는 가상/증강 현실, 자율주행차, 로봇 시뮬레이터, 감시 시스템 등 다양한 애플리케이션에 사용되었습니다. 이러한 응용 분야의 경우 머리 장착형 디스플레이를 사용하여 시선 지점의 확률 분포를 나타내는 돌출 맵을 추정하여 전방향 이미지에서 중요한 영역을 감지하는 것이 유용할 것입니다. 본 논문에서는 다양한 방향과 화각의 전방향 영상에서 중첩된 2차원 평면 영상을 추출하여 전방향 영상에 대한 새로운 돌출맵 추정 모델을 제안한다. 2D 돌출 맵은 이미지 중앙에 확률이 높은 경향이 있는 반면(중앙 바이어스), 머리 장착형 디스플레이를 사용하는 경우 전방향 돌출 맵에서는 확률이 높은 영역이 수평 방향으로 나타납니다(적도 바이어스). 따라서 2D 평면 이미지 추출을 위해 Center-bias 레이어를 앙각에 따라 조정된 적도-바이어스 레이어로 대체하여 전방향 데이터 세트로 center-bias 레이어를 갖춘 2D 돌출 모델을 미세 조정했습니다. 돌출 데이터 세트에서 전방향 이미지의 제한된 가용성은 2D 돌출 맵의 실제값을 사용하여 다수의 훈련 이미지로 사전 훈련된 잘 확립된 2D 돌출 모델을 사용하여 보완할 수 있습니다. 또한, 본 논문에서는 가변적인 수용 영역을 갖는 다양한 크기의 객체를 검출하기 위해 다양한 화각에서 2차원 영상을 추출하는 다중 스케일 추정 방법을 제안한다. 각 객체에 최적의 척도를 부여하기 위해 통합 레이어에서 계산된 픽셀별 주의 가중치를 사용하여 다중 화각에서 추정된 돌출 맵을 통합했습니다. 제안된 방법은 전방향 돌출 맵에 대한 평가 지표가 포함된 공개 데이터 세트를 사용하여 평가되었습니다. 제안된 방법에 의해 돌출맵의 정확도가 향상됨을 확인하였다.
Takao YAMANAKA
Sophia University
Tatsuya SUZUKI
Sophia University
Taiki NOBUTSUNE
Sophia University
Chenjunlin WU
Sophia University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
부
Takao YAMANAKA, Tatsuya SUZUKI, Taiki NOBUTSUNE, Chenjunlin WU, "Multi-Scale Estimation for Omni-Directional Saliency Maps Using Learnable Equator Bias" in IEICE TRANSACTIONS on Information,
vol. E106-D, no. 10, pp. 1723-1731, October 2023, doi: 10.1587/transinf.2023EDP7055.
Abstract: Omni-directional images have been used in wide range of applications including virtual/augmented realities, self-driving cars, robotics simulators, and surveillance systems. For these applications, it would be useful to estimate saliency maps representing probability distributions of gazing points with a head-mounted display, to detect important regions in the omni-directional images. This paper proposes a novel saliency-map estimation model for the omni-directional images by extracting overlapping 2-dimensional (2D) plane images from omni-directional images at various directions and angles of view. While 2D saliency maps tend to have high probability at the center of images (center bias), the high-probability region appears at horizontal directions in omni-directional saliency maps when a head-mounted display is used (equator bias). Therefore, the 2D saliency model with a center-bias layer was fine-tuned with an omni-directional dataset by replacing the center-bias layer to an equator-bias layer conditioned on the elevation angle for the extraction of the 2D plane image. The limited availability of omni-directional images in saliency datasets can be compensated by using the well-established 2D saliency model pretrained by a large number of training images with the ground truth of 2D saliency maps. In addition, this paper proposes a multi-scale estimation method by extracting 2D images in multiple angles of view to detect objects of various sizes with variable receptive fields. The saliency maps estimated from the multiple angles of view were integrated by using pixel-wise attention weights calculated in an integration layer for weighting the optimal scale to each object. The proposed method was evaluated using a publicly available dataset with evaluation metrics for omni-directional saliency maps. It was confirmed that the accuracy of the saliency maps was improved by the proposed method.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2023EDP7055/_p
부
@ARTICLE{e106-d_10_1723,
author={Takao YAMANAKA, Tatsuya SUZUKI, Taiki NOBUTSUNE, Chenjunlin WU, },
journal={IEICE TRANSACTIONS on Information},
title={Multi-Scale Estimation for Omni-Directional Saliency Maps Using Learnable Equator Bias},
year={2023},
volume={E106-D},
number={10},
pages={1723-1731},
abstract={Omni-directional images have been used in wide range of applications including virtual/augmented realities, self-driving cars, robotics simulators, and surveillance systems. For these applications, it would be useful to estimate saliency maps representing probability distributions of gazing points with a head-mounted display, to detect important regions in the omni-directional images. This paper proposes a novel saliency-map estimation model for the omni-directional images by extracting overlapping 2-dimensional (2D) plane images from omni-directional images at various directions and angles of view. While 2D saliency maps tend to have high probability at the center of images (center bias), the high-probability region appears at horizontal directions in omni-directional saliency maps when a head-mounted display is used (equator bias). Therefore, the 2D saliency model with a center-bias layer was fine-tuned with an omni-directional dataset by replacing the center-bias layer to an equator-bias layer conditioned on the elevation angle for the extraction of the 2D plane image. The limited availability of omni-directional images in saliency datasets can be compensated by using the well-established 2D saliency model pretrained by a large number of training images with the ground truth of 2D saliency maps. In addition, this paper proposes a multi-scale estimation method by extracting 2D images in multiple angles of view to detect objects of various sizes with variable receptive fields. The saliency maps estimated from the multiple angles of view were integrated by using pixel-wise attention weights calculated in an integration layer for weighting the optimal scale to each object. The proposed method was evaluated using a publicly available dataset with evaluation metrics for omni-directional saliency maps. It was confirmed that the accuracy of the saliency maps was improved by the proposed method.},
keywords={},
doi={10.1587/transinf.2023EDP7055},
ISSN={1745-1361},
month={October},}
부
TY - JOUR
TI - Multi-Scale Estimation for Omni-Directional Saliency Maps Using Learnable Equator Bias
T2 - IEICE TRANSACTIONS on Information
SP - 1723
EP - 1731
AU - Takao YAMANAKA
AU - Tatsuya SUZUKI
AU - Taiki NOBUTSUNE
AU - Chenjunlin WU
PY - 2023
DO - 10.1587/transinf.2023EDP7055
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2023
AB - Omni-directional images have been used in wide range of applications including virtual/augmented realities, self-driving cars, robotics simulators, and surveillance systems. For these applications, it would be useful to estimate saliency maps representing probability distributions of gazing points with a head-mounted display, to detect important regions in the omni-directional images. This paper proposes a novel saliency-map estimation model for the omni-directional images by extracting overlapping 2-dimensional (2D) plane images from omni-directional images at various directions and angles of view. While 2D saliency maps tend to have high probability at the center of images (center bias), the high-probability region appears at horizontal directions in omni-directional saliency maps when a head-mounted display is used (equator bias). Therefore, the 2D saliency model with a center-bias layer was fine-tuned with an omni-directional dataset by replacing the center-bias layer to an equator-bias layer conditioned on the elevation angle for the extraction of the 2D plane image. The limited availability of omni-directional images in saliency datasets can be compensated by using the well-established 2D saliency model pretrained by a large number of training images with the ground truth of 2D saliency maps. In addition, this paper proposes a multi-scale estimation method by extracting 2D images in multiple angles of view to detect objects of various sizes with variable receptive fields. The saliency maps estimated from the multiple angles of view were integrated by using pixel-wise attention weights calculated in an integration layer for weighting the optimal scale to each object. The proposed method was evaluated using a publicly available dataset with evaluation metrics for omni-directional saliency maps. It was confirmed that the accuracy of the saliency maps was improved by the proposed method.
ER -