SetterVision
Motion-based Tactical Training System for Volleyball Setters in Virtual Reality. (Accepted to ACM MM 2023)
Abstract
Volleyball, a sport characterized by unpredictable factors such as ball trajectory, teammate actions, and strategic positioning, presents a challenge when it comes to modeling and training due to its high levels of complexity. Successful gameplay relies on the coordinated efforts of all team members in the receiving, setting, and attacking phases. In real-life competitions, the setter’s on-ball ability and decision-making are particularly crucial to the team’s offensive success: To improve the training of setters in observing player movements while running and making informed attacking decisions, we propose the design of a virtual reality (VR) system which aims to enhance players’ setting skills and strategic thinking to achieve more successful offensive plays with a lower cost.
Role & Tools Incorporated
Work developed as a cross-lab/disciplinary project in cross-reality multimodal interaction with MiiS Lab at NTUST: Developed a virtual/mixed reality system for volleyball setter training with a group of seven. I worked on preliminary research, designed interaction methodology within virtual space, and implemented an immersive volleyball gameplay simulation system. Technical tools include Unity3D, AR/VR/XR, and C#.
System Overview
Tactic Board Module
To fully emulate current methodology of tactical training, we integrate a tactic board module to simulate plays of teammates’ positions and running trajectories. These plays can be created directly by the VR users within the VR environment or imported by professional coaches. With this feature, not only could setters be trained through playcalling, but coaches could also specify and customize set-pieces using the immersive gameplay simulation system, thereby improving the overall design of plays.
Immersive Gameplay Module
Once a play is selected, it could be thoroughly simulated in our gameplay system with virtual teammates running the given play to certain positions. The VR users have to utilize their court vision to run and arrive at the suitable position for setting within a limited time period. Once the VR users move to the correct position, they have to observe the positions and actions of their nearby teammates for setting using their decisionmaking ability. Moreover, the VR users have to perform correct setting actions with its corresponding passing teammate using their ball-handling skills. We modeled random realistic scenarios, including unready teammates, illegal actions, and ineligible attackers, to train situational awareness. Additionally, by switching between our tactic board and gameplay system, players can learn from both third-person videos and first-person perspectives, facilitating a more comprehensive understanding. Combining advanced simulations and realistic scenarios, our immersive gameplay system offers a solution that bridges the gap between on-paper strategies and on-court execution.
Motion Input Module
Unlike traditional emulation systems for sports, which usually rely on button-based state selection, our system utilizes motion-controlled inputs to replicate movements required during real gameplay. The Vive Wave XR Controller Plugin is applied to obtain the head movement of the VR users, and achieve hand tracking as well as gesture interaction when using our tactic board system in VR. Two additional Vive Wrist Trackers are used to acquire information from accelerometers and gyroscopes to understand wrist rotations and recognize which setter action (i.e., forward/backward set, jump forward/backward set) is performed. In this way, players can experience a higher level of immersiveness with more intuitive inputs, blurring the lines between virtual and real-world performances. This approach enables training in a more realistic manner for users.
Tactic Board System
Immersive Gameplay System
Setter Motion/Action
Conference & Proceedings
ACM Multimedia '23
Yu-Hsuan Chen, Chen-Wei Fu, Wei-Lun Huang, Ming-Cong Su, Hsin-Yu Huang, Andrew Chen, and Tse-Yu Pan. 2023. SetterVision: Motion-based Tactical Training System for Volleyball Setters in Virtual Reality. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29-November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 3 pages. [ACM DL]