Joint detection and tracking, which solves two fundamental vision challenges in a unified manner, is a challenging topic in computer vision. In this area, the proper use of spatial-temporal information in videos can help reduce local defects and improve the quality of feature representations. Although modeling low-level (usually pixel-wise) spatial-temporal information has been studied, instance-level spatial-temporal correlations (i.e., relations between semantic regions in which instances have occurred) have not been fully exploited. In comparison, modeling instance-level correlation is a more flexible and reasonable way to enhance feature representations. However, we have found that conventional instance-level relation learning that works for the separate tasks of detection or tracking is not effective in joint tasks in which a variety of scenarios may be presented. To try to resolve this problem, in this study, we effectively exploited instance-level spatial-temporal semantic information for joint detection and tracking via a joint relation learning pipeline with a novel relation learning mechanism called Similarity- and Quality-Guided Attention (SQGA). Specifically, we added task-specific SQGA relation modules before the corresponding task prediction heads to refine the instance feature representation using features of other reference instances in the neighboring frames; these features are aggregated on the basis of relational affinities. In particular, in SQGA, relational affinities were factorized to similarity and quality terms so that fine-grained supervision rules could be applied. Then we added task-specific attention losses for each SQGA relation module, resulting in a better feature aggregation for the corresponding task. Quantitative experiments based on several challenging multi-object tracking benchmarks showed that our approach was more effective than the baselines and provided competitive results compared with recent state-of-the-art methods.
机构:
Sun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Minist Educ, Key Lab Machine Intelligence & Adv Comp, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R ChinaSun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Ke, Bo
Zheng, Huicheng
论文数: 0引用数: 0
h-index: 0
机构:
Sun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Minist Educ, Key Lab Machine Intelligence & Adv Comp, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R ChinaSun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Zheng, Huicheng
Chen, Lvran
论文数: 0引用数: 0
h-index: 0
机构:
Sun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Minist Educ, Key Lab Machine Intelligence & Adv Comp, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R ChinaSun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Chen, Lvran
Yan, Zhiwei
论文数: 0引用数: 0
h-index: 0
机构:
Sun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Minist Educ, Key Lab Machine Intelligence & Adv Comp, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R ChinaSun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Yan, Zhiwei
Li, Ye
论文数: 0引用数: 0
h-index: 0
机构:
Sun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
Minist Educ, Key Lab Machine Intelligence & Adv Comp, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R ChinaSun Yat Sen Univ, Sch Data & Comp Sci, 135 West Xingang Rd, Guangzhou 510275, Guangdong, Peoples R China
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R China
Li, Xiaotong
Sun, Weifeng
论文数: 0引用数: 0
h-index: 0
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R China
Sun, Weifeng
Ji, Yonggang
论文数: 0引用数: 0
h-index: 0
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R China
Ji, Yonggang
Huang, Weimin
论文数: 0引用数: 0
h-index: 0
机构:
Mem Univ Newfoundland, Fac Engn & Appl Sci, St John, NF A1B 3X5, CanadaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao 266580, Peoples R China
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R China
Li, Xiaotong
Sun, Weifeng
论文数: 0引用数: 0
h-index: 0
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R China
Sun, Weifeng
Ji, Yonggang
论文数: 0引用数: 0
h-index: 0
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R China
Ji, Yonggang
Dai, Yongshou
论文数: 0引用数: 0
h-index: 0
机构:
China Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R ChinaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R China
Dai, Yongshou
Huang, Weimin
论文数: 0引用数: 0
h-index: 0
机构:
Mem Univ Newfoundland, Fac Engn & Appl Sci, St John, NF, CanadaChina Univ Petr East China, Coll Oceanog & Space Informat, Qingdao, Peoples R China
机构:
Hanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South KoreaHanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South Korea
Cho, Seung-Wan
Lim, Yeong-Hyun
论文数: 0引用数: 0
h-index: 0
机构:
Hanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South KoreaHanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South Korea
Lim, Yeong-Hyun
Seo, Kyung-Min
论文数: 0引用数: 0
h-index: 0
机构:
Hanyang Univ ERICA, Dept Ind & Management Engn, 55 Hanyangdaehak Ro, Ansan 15588, Gyeonggi Do, South KoreaHanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South Korea
Seo, Kyung-Min
Kim, Jungin
论文数: 0引用数: 0
h-index: 0
机构:
Hankyong Natl Univ, AI Res, 283 Samnam Ro, Pyeongtaek 17738, Gyeonggi Do, South KoreaHanyang Univ, Dept Ind & Management Engn, 222 Wangsimni Ro, Seoul 04763, South Korea