Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition

被引:0
|
作者
Zhu, Yixin [1 ]
Zhao, Yibiao [1 ]
Zhu, Song-Chun [1 ]
机构
[1] Univ Calif Los Angeles, Ctr Vis Cognit Learning & Art, Los Angeles, CA 90095 USA
来源
2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2015年
关键词
AFFORDANCES; GEOMETRY;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we present a new framework - task-oriented modeling, learning and recognition which aims at understanding the underlying functions, physics and causality in using objects as "tools". Given a task, such as, cracking a nut or painting a wall, we represent each object, e.g. a hammer or brush, in a generative spatio-temporal representation consisting of four components: i) an affordance basis to be grasped by hand; ii) a functional basis to act on a target object (the nut), iii) the imagined actions with typical motion trajectories; and iv) the underlying physical concepts, e.g. force, pressure, etc. In a learning phase, our algorithm observes only one RGB-D video, in which a rational human picks up one object (i. e. tool) among a number of candidates to accomplish the task. From this example, our algorithm learns the essential physical concepts in the task (e.g. forces in cracking nuts). In an inference phase, our algorithm is given a new set of objects (daily objects or stones), and picks the best choice available together with the inferred affordance basis, functional basis, imagined human actions (sequence of poses), and the expected physical quantity that it will produce. From this new perspective, any objects can be viewed as a hammer or a shovel, and object recognition is not merely memorizing typical appearance examples for each category but reasoning the physical mechanisms in various tasks to achieve generalization.
引用
收藏
页码:2855 / 2864
页数:10
相关论文
共 50 条
  • [41] Unsupervised learning of kb queries in task-oriented dialogs
    Raghu D.
    Gupta N.
    Mausam
    Transactions of the Association for Computational Linguistics, 2021, 9 : 374 - 390
  • [42] Task-oriented Dialogue System Based on Reinforcement Learning
    Song, Meina
    Chen, Zhongfu
    Niu, Peiqing
    Haihong, E.
    PROCEEDINGS OF 2019 IEEE 10TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS 2019), 2019, : 93 - 98
  • [43] One-Shot Learning for Task-Oriented Grasping
    Holomjova, Valerija
    Starkey, Andrew J.
    Yun, Bruno
    Meisner, Pascal
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2023, 8 (12) : 8232 - 8238
  • [44] Task-oriented contrastive learning for unsupervised domain adaptation
    Wei, Xing
    Wen, Bin
    Yang, Fan
    Liu, Yujie
    Zhao, Chong
    Hu, Di
    Luo, Hui
    EXPERT SYSTEMS WITH APPLICATIONS, 2023, 229
  • [45] Structural Learning: Attraction and Conformity in Task-Oriented Groups
    James A. Kitts
    Michael W. Macy
    Andreas Flache
    Computational & Mathematical Organization Theory, 1999, 5 (2): : 129 - 145
  • [46] Research on discourse role recognition in task-oriented collaborative dialogue
    Shan, Liqian
    Zhao, Hui
    Feng, Yuhui
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2024, 46 (03) : 5709 - 5721
  • [47] From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue
    Feng, Shutong
    Lubis, Nurul
    Ruppik, Benjamin
    Geishauser, Christian
    Heck, Michael
    Lin, Hsien-chin
    van Niekerk, Carel
    Vukovic, Renato
    Gasic, Milica
    24TH MEETING OF THE SPECIAL INTEREST GROUP ON DISCOURSE AND DIALOGUE, SIGDIAL 2023, 2023, : 85 - 103
  • [48] TIENet: task-oriented image enhancement network for degraded object detection
    Wang, Yudong
    Guo, Jichang
    Wang, Ruining
    He, Wanru
    Li, Chongyi
    SIGNAL IMAGE AND VIDEO PROCESSING, 2024, 18 (01) : 1 - 8
  • [49] TIENet: task-oriented image enhancement network for degraded object detection
    Yudong Wang
    Jichang Guo
    Ruining Wang
    Wanru He
    Chongyi Li
    Signal, Image and Video Processing, 2024, 18 : 1 - 8
  • [50] Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems
    Li, Ziming
    Kiseleva, Julia
    de Rijke, Maarten
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2020, 2020,