TFDEPTH: SELF-SUPERVISED MONOCULARDEPTH ESTIMATION WITH MULITI-SCALE SELECTIVE TRANSFORMER FEATURE FUSION

被引:0
作者
Hu, Hongli [1 ]
Miao, Jun [1 ,2 ]
Zhu, Guanghu [1 ]
Yan, Je [2 ]
Chu, Jun [3 ]
机构
[1] Nanchang Hangkong Univ, Sch Aeronaut Mfg Engn, Nanchang, Peoples R China
[2] Chinese Acad Sci, Key Lab Lunar & Deep Space Explorat, Beijing, Peoples R China
[3] Nanchang Hangkong Univ, Key Lab Jiangxi Prov Image Proc & Pattern Recognit, Nanchang 330063, Peoples R China
关键词
monocular depth estimation; multi-scale fusion; self-supervised learning; transformer;
D O I
10.105566/ias.2987
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Existing self -supervised models for monocular depth estimation suffer from issues such as discontinuity, blurred edges, and unclear contours, particularly for small objects. We propose a self -supervised monocular depth estimation network with multi -scale selective Transformer feature fusion. To preserve more detailed features, this paper constructs a multi -scale encoder to extract features and leverages the self -attention mechanism of Transformer to capture global contextual information, enabling better depth prediction for small objects. Additionally, the multi -scale selective fusion module (MSSF) is also proposed, which can make full use of multi -scale feature information in the decoding part and perform selective fusion step by step, which can effectively eliminate noise and retain local detail features to obtain a clear depth map with clear edges. Experimental evaluations on the KITTI dataset demonstrate that the proposed algorithm achieves an absolute relative error (Abs Rel) of 0.098 and an accuracy rate (delta) of 0.983. The results indicate that the proposed algorithm not only estimates depth values with high accuracy but also predicts the continuous depth map with clear edges.
引用
收藏
页码:139 / 149
页数:11
相关论文
共 50 条
  • [41] Self-supervised learning-based Multi-Scale feature Fusion Network for survival analysis from whole slide images
    Li, Le
    Liang, Yong
    Shao, Mingwen
    Lu, Shanghui
    Liao, Shuilin
    Ouyang, Dong
    COMPUTERS IN BIOLOGY AND MEDICINE, 2023, 153
  • [42] Hierarchical Multi-scale Architecture Search for Self-supervised Monocular Depth Estimation
    Ren, Jian
    Xie, Jin
    Jin, Zhong
    PATTERN RECOGNITION, ACPR 2021, PT II, 2022, 13189 : 447 - 461
  • [43] STP: Self-supervised transfer learning based on transformer for noninvasive blood pressure estimation using photoplethysmography
    Ma, Chenbin
    Zhang, Peng
    Zhang, Haonan
    Liu, Zeyu
    Song, Fan
    He, Yufang
    Zhang, Guanglei
    EXPERT SYSTEMS WITH APPLICATIONS, 2024, 249
  • [44] Scale-Aware Visual-Inertial Depth Estimation and Odometry Using Monocular Self-Supervised Learning
    Lee, Chungkeun
    Kim, Changhyeon
    Kim, Pyojin
    Lee, Hyeonbeom
    Kim, H. Jin
    IEEE ACCESS, 2023, 11 : 24087 - 24102
  • [45] IFDepth: Iterative fusion network for multi-frame self-supervised monocular depth estimation
    Wang, Lizhe
    Liang, Qi
    Che, Yu
    Wang, Lanmei
    Wang, Guibao
    KNOWLEDGE-BASED SYSTEMS, 2025, 318
  • [46] Self-supervised multi-scale pyramid fusion networks for realistic bokeh effect rendering
    Wang, Zhifeng
    Jiang, Aiwen
    Zhang, Chunjie
    Li, Hanxi
    Liu, Bo
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2022, 87
  • [47] Self-supervised spatial-temporal transformer fusion based federated framework for 4D cardiovascular
    Mazher, Moona
    Razzak, Imran
    Qayyum, Abdul
    Tanveer, M.
    Beier, Susann
    Khan, Tariq
    Niederer, Steven A.
    INFORMATION FUSION, 2024, 106
  • [48] Self-Supervised Monocular Depth Estimation Using Global and Local Mixed Multi-Scale Feature Enhancement Network for Low-Altitude UAV Remote Sensing
    Chang, Rong
    Yu, Kailong
    Yang, Yang
    REMOTE SENSING, 2023, 15 (13)
  • [49] Detaching and Boosting: Dual Engine for Scale-Invariant Self-Supervised Monocular Depth Estimation
    Jiang, Peizhe
    Yang, Wei
    Ye, Xiaoqing
    Tan, Xiao
    Wu, Meng
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2022, 7 (04): : 12094 - 12101
  • [50] A RECONSTRUCTION-BASED FEATURE ADAPTATION FOR ANOMALY DETECTION WITH SELF-SUPERVISED MULTI-SCALE AGGREGATION
    Zuo, Zuo
    Wu, Zongze
    Chen, Badong
    Zhong, Xiaopin
    2024 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, ICASSP 2024, 2024, : 5840 - 5844