Towards Multimodal Interaction with AI-Infused Shape-Changing Interfaces

被引：0

作者：

Gao, Chenfeng ^{[1
]}

Qian, Wanli ^{[1
]}

Liu, Richard ^{[1
]}

Hanocka, Rana ^{[1
]}

Nakagaki, Ken ^{[1
]}

机构：

[1] Univ Chicago, Chicago, IL 60637 USA

来源：

PROCEEDINGS OF THE 37TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY, UIST ADJUNCT 2024 | 2024年

关键词：

Shape-Changing Interface; Multimodal Interaction; Generative AI;

D O I：

10.1145/3672539.3686315

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present a proof-of-concept system exploring multimodal interaction with AI-infused Shape-Changing Interfaces. Our prototype integrates inFORCE, a 10x5 pin-based shape display, with AI tools for 3D mesh generation and editing. Users can create and modify 3D shapes through speech, gesture, and tangible inputs. We demonstrate potential applications including AI-assisted 3D modeling, adaptive physical controllers, and dynamic furniture. Our implementation, which translates text to point clouds for physical rendering, reveals both the potential and challenges of combining AI with shape-changing interfaces. This work explores how AI can enhance tangible interaction with 3D information and opens up new possibilities for multimodal shape-changing UIs.

引用

页数：3

共 24 条

[1] Grand Challenges in Shape-Changing Interface Research [J].

Alexander, Jason ;

Roudaut, Anne ;

Steimle, Juergen ;

Hornbaek, Kasper ;

Alonso, Miguel Bruns ;

Follmer, Sean ;

Merritt, Tim Robert .

PROCEEDINGS OF THE 2018 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI 2018), 2018,

[2]

Blackshaw M., 2011, Proceedings of CHI 2011, P1237, DOI [DOI 10.1145/1979742.1979754, 10.1145/2048259.2048272, DOI 10.1145/2048259.2048272]

[3]

Bolt R. A., 1980, Computer Graphics, V14, P262, DOI 10.1145/965105.807503

[4]

Chang Angel X., 2015, arXiv

[5] Shape-changing interfaces [J].

Coelho, Marcelo ;

Zigelbaum, Jamie .

PERSONAL AND UBIQUITOUS COMPUTING, 2011, 15 (02) :161-173

[6]

davidchiu21, 2006, Rendering

[7]

Dumas B, 2009, LECT NOTES COMPUT SC, V5440, P3, DOI 10.1007/978-3-642-00437-7_1

[8]

Goldstein Seth., 2004, Claytronics: A Scalable Basis For Future Robots

[9]

Honnibal M., 2017, IN PRESS

[10]

Ishii Hiroshi, 2012, Interactions, V19, P38, DOI 10.1145/2065327.2065337

← 1 2 3 →