TeleSHift
Telexisting TUI for Physical Collaboration & Interaction. (Best Demo Award, ACM UbiComp/ISWC '22)
Abstract
Current workflow on co-editing and simultaneous presentation of 3-D shapes is confined to on-screen manipulation, which causes loss of perceived information when presenting perceptual concepts or complex shapes between members. Thus, we create TeleSHift, a 3-D tangible user interface (TUI) with a telexisting communication framework for group-based collaboration and demonstration. In this work, we present a larger-scaled proof-of-concept prototype providing hands-on operation for shape-based interactions including multi-sided collaboration and one-to-many presentation. In contrast to previous works, we further extend the use of TUIs to support cooperative tasks with telexistence while enabling the linkage of manipulable bits to provide a better user experience and interactivity.
Role & Tools Incorporated
Led a research team of six and worked on media design with a focus on human-computer interaction. We explored 3-D tangible user interfaces for real-time multi-sided physical information presenting and editing. Developed concept of work, designed system structure and created a functional prototype. Technical tools include Arduino/ESP32, C++, Unity3D, C# ,and 3D Printing.
Problem & Background Research
Tangible User Interfaces (TUIs) enable humans to perceive spatial information more intuitively. Ever since its introduction, its applications have extended from simple static model-presenting tasks to providing touch-based feedback with a degree of interactivity. Even so, we consider that its applications have not been thoroughly explored and see the potential of TUIs in supporting the collaboration of physical prototyping and dimensional presentation.
When it comes to presenting 3-D shapes and structures with 2-D interfaces, especially during simultaneous editing, perceptual loss of accessed information would inevitably occur. The concept of telexistence is to make two separated objects in different physical spaces to sense co-presence via interaction. On top of that, we proposed a 3-D TUI to achieve more realistic feedback and thorough perception, while enhancing interaction experience on both dimensional information knowing and object-initiated interpersonal exchanges.
This work contributes by exploring the use of TUIs in physically manipulating and presenting 3-D shapes while also creating a functional, larger-scaled proof-of-concept prototype with telexisting abilities to provide parallel display and simultaneous altering of shape-based information for multi-sided physical collaboration.
The storyboard illustrating the concept of TeleSHift
Mechanism Overview
We present TeleSHift, which is a structure created by Telexisting shape-shifting substructures and used as a TUI. Our implementation of the substructure is constructed to have six extension arms able to perform 1-dimensional extend-interact movements. The arms, which consist of motorized slide potentiometers fixed onto the aluminum ridges, are each separated by 90 degrees to present deformation on the positive and negative of the x- y- and z-axes.
Furthermore, the extending arms are also used to connect with other substructures by magnets for swarming to shape-shift on a larger scale, creating TeleSHifts. The substructures’ control and communication are propelled with ESP32 and can exchange data with Google Realtime Firebase to update its current status including dimensions and jointing states, or fetch data to enable shape recovery and achieve synchronization with other connected TeleSHifts in different physical spaces.
Our larger scaled, proof-of-concept prototype of a substructure. Each consists of 3 ESP32 units, 3 TB6612FNG DC motor drivers, 6 Alps RS60N11M9A0E motorized slide potentiometers, electronic components along with other metal and 3-D printed PLA parts.
Two prototype substructures connecting with each other magnetically to form a TeleSHift. The connected substructures are able to sense the existence of each other and document connection positioning as well as its deformation status to Google Realtime Firebase.
Proposing Scenarios
Into Prototype Collaboration
When it comes to a group of designers physically in different spaces working together on a prototyping task creating an ergonomic mouse, a common workflow would be to rely on 2-D interfaces of computer-aided design tools for cooperation. However, a loss of perceived information, especially when collaborating simultaneously, would often occur when presenting perceptual concepts or complex shapes between the collaborating members. With TeleSHift, the designers can now collaborate on the mouse’s prototype at the same time: Not only could they present the deformation of the created shape, parallel knowing and simultaneous shape-shifting can be achieved when one manually modifies his or her TeleSHift by presenting the same deformation status on the others’ at the same time, and vice versa.
Into Dimensional Teaching
Another applicable scenario would be when a teacher wants to present the process on creating a basic structure of a chair to several students and asks them to create their own unique design with some high-cost or rare material. The teacher can manually demonstrate the shaping process with TeleSHift, while students can watch and learn about the structural change with their own TeleSHifts at the same time. Moreover, students’ manipulation of their devices would not affect both the other students and the teacher, which allows students to come up with their own creations. On the other hand, students can store the deformation status during their design at any time. When an undo process is required, they can return to the specific shape and redesign the chair.
Conference & Proceedings
Best Demo Award @ ACM UbiComp/ISWC '22
Andrew Chen, Tzu-Ling Yang, Shu-Yan Cheng, Po-Sheng Cheng, Tzu-Han Lin, and Kaiyuan Lin. 2022. TeleSHift: Telexisting TUI for Physical Collaboration & Interaction. In Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp/ISWC ’22 Adjunct), September 11–15, 2022, Cambridge, United Kingdom. ACM, New York, NY, USA, 4 pages. [ACM DL]
