Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition

被引:0
|
作者
Zhu, Yixin [1 ]
Zhao, Yibiao [1 ]
Zhu, Song-Chun [1 ]
机构
[1] Univ Calif Los Angeles, Ctr Vis Cognit Learning & Art, Los Angeles, CA 90095 USA
来源
2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2015年
关键词
AFFORDANCES; GEOMETRY;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we present a new framework - task-oriented modeling, learning and recognition which aims at understanding the underlying functions, physics and causality in using objects as "tools". Given a task, such as, cracking a nut or painting a wall, we represent each object, e.g. a hammer or brush, in a generative spatio-temporal representation consisting of four components: i) an affordance basis to be grasped by hand; ii) a functional basis to act on a target object (the nut), iii) the imagined actions with typical motion trajectories; and iv) the underlying physical concepts, e.g. force, pressure, etc. In a learning phase, our algorithm observes only one RGB-D video, in which a rational human picks up one object (i. e. tool) among a number of candidates to accomplish the task. From this example, our algorithm learns the essential physical concepts in the task (e.g. forces in cracking nuts). In an inference phase, our algorithm is given a new set of objects (daily objects or stones), and picks the best choice available together with the inferred affordance basis, functional basis, imagined human actions (sequence of poses), and the expected physical quantity that it will produce. From this new perspective, any objects can be viewed as a hammer or a shovel, and object recognition is not merely memorizing typical appearance examples for each category but reasoning the physical mechanisms in various tasks to achieve generalization.
引用
收藏
页码:2855 / 2864
页数:10
相关论文
共 50 条
  • [21] Continual Learning in Task-Oriented Dialogue Systems
    Madotto, Andrea
    Lin, Zhaojiang
    Zhou, Zhenpeng
    Moon, Seungwhan
    Crook, Paul
    Liu, Bing
    Yu, Zhou
    Cho, Eunjoon
    Fung, Pascale
    Wang, Zhiguang
    2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 7452 - 7467
  • [22] Robustness Testing of Language Understanding in Task-Oriented Dialog
    Liu, Jiexi
    Takanobui, Ryuichi
    Wen, Jiaxin
    Wan, Dazhen
    Li, Hongguang
    Nie, Weiran
    Li, Cheng
    Peng, Wei
    Huang, Minlie
    59TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 11TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING, VOL 1 (ACL-IJCNLP 2021), 2021, : 2467 - 2480
  • [23] Review of Research on Task-Oriented Spoken Language Understanding
    Hou, Lixian
    Li, Yanling
    Li, Chengcheng
    Lin, Min
    2019 3RD INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE, AUTOMATION AND CONTROL TECHNOLOGIES (AIACT 2019), 2019, 1267
  • [24] Learning Folksonomies from Task-Oriented Dialogues
    Puppi Wanderley, Gregory Moro
    Paraiso, Emerson Cabrera
    30TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, VOLS I AND II, 2015, : 360 - 367
  • [25] Understanding User Satisfaction with Task-oriented Dialogue Systems
    Siro, Clemencia
    Aliannejadi, Mohammad
    de Rijke, Maarten
    PROCEEDINGS OF THE 45TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '22), 2022, : 2018 - 2023
  • [26] Accelerating Natural Language Understanding in Task-Oriented Dialog
    Ahuja, Ojas
    Desai, Shrey
    NLP FOR CONVERSATIONAL AI, 2020, : 46 - 53
  • [27] Task-oriented Grasping with Semantic and Geometric Scene Understanding
    Detry, Renaud
    Papon, Jeremie
    Matthies, Larry
    2017 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2017, : 3266 - 3273
  • [28] A Task-Oriented Approach for Cost-sensitive Recognition
    Mottaghi, Roozbeh
    Hajishirzi, Hannaneh
    Farhadi, An
    2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, : 2203 - 2211
  • [29] Task-Oriented Grasping Position Selection in Deformable Object Manipulation
    Huang, Jing
    Au, K. W. Samuel
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2023, 8 (02) : 776 - 783
  • [30] On modeling and utilizing chemical compound information with deep learning technologies: A task-oriented approach
    Lim, Sangsoo
    Lee, Sangseon
    Piao, Yinhua
    Choi, MinGyu
    Bang, Dongmin
    Gu, Jeonghyeon
    Kim, Sun
    COMPUTATIONAL AND STRUCTURAL BIOTECHNOLOGY JOURNAL, 2022, 20 : 4288 - 4304