TON-ViT: A Neuro-Symbolic AI Based on Task Oriented Network with a Vision Transformer

被引：1

作者：

Zhuo, Yupeng ^{[1
]}

Jiang, Nina ^{[1
]}

Kirkpatrick, Andrew W. ^{[2
]}

Couperus, Kyle ^{[3
,5
]}

Oanh Tran ^{[3
,5
]}

Beck, Jonah ^{[3
,5
]}

DeVane, DeAnna ^{[3
,5
]}

Candelore, Ross ^{[3
,4
]}

McKee, Jessica ^{[2
]}

Gorbatkin, Chad ^{[3
]}

Birch, Eleanor ^{[3
]}

Colombo, Christopher ^{[3
,5
]}

Duerstock, Bradley ^{[1
]}

Wachs, Juan ^{[1
]}

机构：

[1] Purdue Univ, Sch Ind Engn, W Lafayette, IN 47907 USA

[2] Univ Calgary, Calgary, AB, Canada

[3] Madigan Army Med Ctr, Joint Base Lewis McChord, WA USA

[4] William Beaumont Army Med Ctr, Ft Bliss, TX USA

[5] Geneva Fdn, Tacoma, WA USA

来源：

MEDICAL IMAGE UNDERSTANDING AND ANALYSIS, MIUA 2023 | 2024年 / 14122卷

关键词：

task graph; knowledge graph; semantic understanding; vision transformer; neuro-symbolic AI; medical procedures; KNOWLEDGE GRAPHS;

D O I：

10.1007/978-3-031-48593-0_12

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The objective of this paper is to present a neuro-symbolic AI based technique to represent field-medicine knowledge, referred as to TON-ViT. TON-ViT integrates a Deep Learning Model with an explicit symbolic manipulation, a task graph. This task graph describes the steps of each trauma resuscitation as denoted by a verb and noun pair. Through this representation, symbolic processing and manipulation on task graphs, we can find stereotypical procedures, regardless of style of the performer. Furthermore, we can use this technique to find differences in styles, errors, shortcuts and generate procedures never seen before. When used in combination with a transformer, it can help recognize actions in egocentric vision datasets. Last, through symbolic manipulations on the graph, it is possible to generate medical knowledge which the model has not seen before. We present preliminary results after testing the TON-ViT with the Trauma Thompson Dataset.

引用

页码：157 / 170

页数：14

共 26 条

[1]

Broughton M., 2021, PREPRINT, DOI [DOI 10.48550/ARXIV.2003, 10.48550/arXiv.2003., 10.48550/arXiv.2003]

[2] Knowledge graphs: Introduction, history, and perspectives [J].

Chaudhri, Vinay K. ;

Baru, Chaitanya ;

Chittar, Naren ;

Dong, Xin Luna ;

Genesereth, Michael ;

Hendler, James ;

Kalyanpur, Aditya ;

Lenat, Douglas B. ;

Sequeda, Juan ;

Vrandecic, Denny ;

Wang, Kuansan .

AI MAGAZINE, 2022, 43 (01) :17-29

[3] An automatic literature knowledge graph and reasoning network modeling framework based on ontology and natural language processing [J].

Chen, Hainan ;

Luo, Xiaowei .

ADVANCED ENGINEERING INFORMATICS, 2019, 42

[4]

Dosovitskiy A, 2021, Arxiv, DOI arXiv:2010.11929

[5]

Fensel Dieter, 2020, Knowledge graphs: Methodology, tools and selected use cases, P1, DOI [DOI 10.1007/978-3-030-37439-6_1, DOI 10.1007/978-3-030-37439-6]

[6] A Survey on Knowledge Graph-Based Recommender Systems [J].