Papers
| Monday, 18 March 2024 (Timezone: Orlando, Florida USA UTC-4) | |||
|---|---|---|---|
| MO1G | Tracking and Motion Capture | 14:00‑15:00 | Fantasia Ballroom G |
| MO1H | Social Applications | 14:00‑15:00 | Fantasia Ballroom H |
| MO1J | Embodiment, Avatars and Presence | 14:00‑15:00 | Fantasia Ballroom J |
| MO2G | Psycho- and Sociocultural Dimensions of Virtual Identities | 15:30‑17:00 | Fantasia Ballroom G |
| MO2H | Multimodal Input and Interaction | 15:30‑17:00 | Fantasia Ballroom H |
| MO2J | Perception and Cognition | 15:30‑17:00 | Fantasia Ballroom J |
| Tuesday, 19 March 2024 (Timezone: Orlando, Florida USA UTC-4) | |||
|---|---|---|---|
| TU1G | 3D Interaction and Touch | 8:30‑9:45 | Fantasia Ballroom G |
| TU1H | Multisensory Interfaces | 8:30‑9:45 | Fantasia Ballroom H |
| TU1J | Evaluating Immersion: UX and Interaction | 8:30‑9:45 | Fantasia Ballroom J |
| TU2G | Locomotion and Redirection | 13:30‑15:00 | Fantasia Ballroom G |
| TU2H | Projections | 13:30‑15:00 | Fantasia Ballroom H |
| TU2J | 3D Interaction and Teleoperation | 13:30‑15:00 | Fantasia Ballroom J |
| TU3G | 3D Interaction and Modality | 15:30‑17:00 | Fantasia Ballroom G |
| TU3H | Perception in Navigation, Locomotion and Redirection | 15:30‑17:00 | Fantasia Ballroom H |
| TU3J | User Experience | 15:30‑17:00 | Fantasia Ballroom J |
| Wednesday, 20 March 2024 (Timezone: Orlando, Florida USA UTC-4) | |||
|---|---|---|---|
| WE1G | 360 Video | 8:30‑9:45 | Fantasia Ballroom G |
| WE1H | Immersive Analytics and Visualization | 8:30‑9:45 | Fantasia Ballroom H |
| WE1J | Industrial and Sports Applications | 8:30‑9:45 | Fantasia Ballroom J |
| WE2G | 3D Authoring | 10:15‑11:15 | Fantasia Ballroom G |
| WE2H | Gaze | 10:15‑11:15 | Fantasia Ballroom H |
| WE2J | Collaboration | 10:15‑11:15 | Fantasia Ballroom J |
| WE3G | Haptics | 11:30‑12:30 | Fantasia Ballroom G |
| WE3H | Healthcare Applications | 11:30‑12:30 | Fantasia Ballroom H |
| WE3J | Human Factors and Ergonomics | 11:30‑12:30 | Fantasia Ballroom J |
| WE4G | Perception in AR, MR and Near-Eye Displays | 13:30‑15:00 | Fantasia Ballroom G |
| WE4H | Rendering and Displays | 13:30‑15:00 | Fantasia Ballroom H |
| WE4J | Experiences, Cybersickness and Presence | 13:30‑15:00 | Fantasia Ballroom J |
| Thursday, 21 March 2024 (Timezone: Orlando, Florida USA UTC-4) | |||
|---|---|---|---|
| TH1G | Distributed Systems and Telepresence | 8:30‑9:45 | Fantasia Ballroom G |
| TH1H | Multimodal Perception and Experiences | 8:30‑9:45 | Fantasia Ballroom H |
| TH1J | Depth and Distance Perception | 8:30‑9:45 | Fantasia Ballroom J |
| TH2G | Touch, Tangible, and Gesture Interfaces | 10:15‑11:15 | Fantasia Ballroom G |
| TH2H | Graphics and Crowds | 10:15‑11:15 | Fantasia Ballroom H |
| TH2J | Ethics in VR | 10:15‑11:15 | Fantasia Ballroom J |
| TH3G | Modeling and Simulation | 11:30‑12:30 | Fantasia Ballroom G |
| TH3H | Software | 11:30‑12:30 | Fantasia Ballroom H |
| TH3J | Localization and Tracking | 11:30‑12:30 | Fantasia Ballroom J |
| TH4G | Education Applications | 13:30‑15:00 | Fantasia Ballroom G |
| TH4H | Virtual Interaction and Embodiment | 13:30‑15:00 | Fantasia Ballroom H |
| TH4J | Locomotion and Navigation | 13:30‑15:00 | Fantasia Ballroom J |
Session: Tracking and Motion Capture (MO1G)
Date & Time: Monday, 18 March 2024, 14:00-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Manuela Chessa
Swift-Eye: Towards Anti-blink Pupil Tracking for Precise and Robust High-Frequency Near-Eye Movement Analysis with Event Cameras (Journal: P1220)
Tongyu Zhang, Shandong University; Yiran Shen, Shandong University; Guangrong Zhao, School of Software; Lin Wang, HKUST, GZ; Xiaoming Chen, Beijing Technology and Business University; Lu Bai, Shandong University; Yuanfeng Zhou, Shandong University
In this paper, we propose Swift-Eye, an offline precise and robust pupil estimation and tracking framework to support high-frequency near-eye movement analysis, especially when the pupil region is partially occluded. Swift-Eye is built upon the emerging event cameras to capture the high-speed movement of eyes in high temporal resolution. Then, a series of bespoke components are designed to generate high-quality near-eye movement video at a high frame rate over kilohertz and deal with the occlusion over the pupil caused by involuntary eye blinks. According to our extensive evaluations on EV-Eye, a large-scale public dataset for eye tracking using event cameras, Swift-Eye shows high robustness against significant occlusion.
GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance (Conference: P1136)
Shiyu Li, Technical University of Munich; Hannah Schieber, Friedrich-Alexander University; Niklas Corell, Friedrich-Alexander University Erlangen-Nürnberg; Bernhard Egger, Friedrich-Alexander-Universität Erlangen-Nürnberg; Julian Kreimeier, Technical University of Munich; Daniel Roth, Technical University of Munich
Guidance for assemblable parts is a promising field for the use of augmented reality. Augmented reality assembly guidance requires 6D object poses of target objects in real-time. To address this problem, we present Graph-based Object Tracking (GBOT), a novel graph-based single-view RGB-D tracking approach. The real-time markerless multi-object tracking is initialized via 6D pose estimation and updates the graph-based assembly poses. By utilizing relative poses of the individual assembly parts, we update the multi-state assembly graph. Quantitative experiments in synthetic data and further qualitative study in real test data show that GBOT can outperform existing work towards enabling context-aware augmented reality assembly guidance.
BOXRR-23: 4.7 Million Motion Capture Recordings from 105,000 XR Users (Journal: P1654)
Vivek C Nair, UC Berkeley; Wenbo Guo, Purdue University; Rui Wang, Carnegie Mellon University; James F. O'Brien, UC Berkeley; Louis Rosenberg, Unanimous AI; Dawn Song, UC Berkeley
Extended reality (XR) devices such as the Meta Quest and Apple Vision Pro have seen a recent surge in attention, with motion tracking "telemetry" data lying at the core of nearly all XR and metaverse experiences. Researchers are just beginning to understand the implications of this data for security, privacy, usability, and more, but currently lack large-scale human motion datasets to study. The BOXRR-23 dataset contains 4,717,215 motion capture recordings, voluntarily submitted by 105,852 XR device users from over 50 countries. BOXRR-23 is over 200 times larger than the largest existing motion capture research dataset and uses a new, highly efficient and purpose-built XR Open Recording (XROR) file format.
Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part (Invited Journal: P3005)
Lei Hu, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China & University of Chinese Academy of Sciences, Beijing, China; Zihao Zhang, Institute of Computing Technology, Chineses Academy of Sciences, Beijing, China; Chongyang Zhong, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China & University of Chinese Academy of Sciences, Beijing, China; Boyuan Jiang, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China & University of Chinese Academy of Sciences, Beijing, China; Shihong Xia, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China & University of Chinese Academy of Sciences, Beijing, China
Motion retargeting is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements, such as the source-target skeletons needing to have the same number of joints or share the same topology. To tackle this problem, we note that skeletons with different structure may have some common body parts despite the differences in joint numbers. Following this observation, we propose a novel, flexible motion retargeting framework. The key idea of our method is to regard the body part as the basic retargeting unit rather than directly retargeting the whole body motion. To enhance the spatial modeling capability of the motion encoder, we introduce a pose-aware attention network (PAN) in the motion encoding phase. The PAN is pose-aware since it can dynamically predict the joint weights within each body part based on the input pose, and then construct a shared latent space for each body part by feature pooling. Extensive experiments show that our approach can generate better motion retargeting results both qualitatively and quantitatively than state-of-the-art methods. Moreover, we also show that our framework can generate reasonable results even for a more challenging retargeting scenario, like retargeting between bipedal and quadrupedal skeletons because of the body part retargeting strategy and PAN. Our code is publicly available.
Session: Social Applications (MO1H)
Date & Time: Monday, 18 March 2024, 14:00-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Mary Whitton
Designing and Evaluating a VR Lobby for a socially enriching remote Opera watching experience (Journal: P1019)
Sueyoon Lee, Centrum Wiskunde & Informatica (CWI); Irene Viola, Centrum Wiskunde & Informatica (CWI); Silvia Rossi, Centrum Wiskunde & Informatica (CWI); Zhirui Guo, Centrum Wiskunde & Informatica (CWI); Ignacio Reimat, Centrum Wiskunde & Informatica (CWI); Kinga Ławicka, Centrum Wiskunde & Informatica (CWI); Alina Striner, Centrum Wiskunde & Informatica (CWI); Pablo Cesar, Centrum Wiskunde & Informatica (CWI)
In this paper, we design, implement, and evaluate a VR theatre lobby as a dynamic space for remote users to communicate and interact following their virtual opera experiences. We conducted an initial test with paired experts (N=10) in a highly realistic representation using our VR lobby prototype, developed based on the theoretical design concept. After refining the prototype for better usability and user experience, we ran a between-subject controlled study (N=40) to compare individuals' and pairs' user experience. The results of our mixed-methods analysis reveal the strength of our social VR lobby in connecting with other users, consuming the opera more deeply, and exploring new possibilities beyond what is common in real life.
Springboard, Roadblock, or "Crutch"?: How Transgender Users Leverage Voice Changers for Gender Presentation in Social Virtual Reality (Conference: P1377)
Kassie C Povinelli, University of Wisconsin-Madison; Yuhang Zhao, University of Wisconsin-Madison
Social virtual reality (VR) serves as a vital platform for TGNC individuals to explore their identities through avatars and build online communities. However, it presents a challenge: the disconnect between avatar embodiment and voice representation, often leading to misgendering and harassment. We interviewed 13 transgender and gender-nonconforming users, finding that using a voice changer not only reduces voice-related harassment, but also allows them to experience gender euphoria through their modified voice, motivating them to pursue voice training and medication to achieve desired voices. Furthermore, we identified technical barriers and possible improvements to voice changer technology for TGNC individuals.
Enhancing Positive Emotions through Interactive Virtual Reality Experiences: An EEG-Based Investigation (Conference: P1475)
Shiwei Cheng, Zhejiang University of Technology; Sheng Danyi, Computer Science; Yuefan Gao, Cyborg Intelligence Technology Company; Zhanxun DONG, Shanghai Jiao Tong University; Ting Han, Shanghai Jiao Tong University
Virtual reality (VR) holds potential to promote feelings of well-being by evoking positive emotions. Our study aimed to investigate the types of interaction behaviors in VR that effectively enhance positive emotions. An exploratory study (N = 22) was conducted on a virtual museum to study the impact of varying user autonomy and interaction behaviors on emotions. An individual emotion model based on electroencephalogram (EEG) was employed to predict the promotion of positive emotions. The results indicated that incorporating creative interaction functions increased positive emotions, with the extent of increase closely linked to the degree of user autonomy.
Self-Guided DMT: Exploring a Novel Paradigm of Dance Movement Therapy in Mixed Reality for Children with ASD (Journal: P1931)
Weiying Liu, School of Mechanical,Electrical&Information Engineering; Yanyan Zhang, Weihai Maternal & Child Health Care Hospital; Baiqiao Zhang, Shandong University; Qian qian Xiong, Shandong University; Hong Zhao, Shandong University; Sheng Li, Peking University; Juan Liu, Shandong University; Yulong Bian, School of Machanical, Electrical & Information Engineering
Children with Autism Spectrum Disorder (ASD) often face motor challenges. Traditional Dance Movement Therapy (DMT) lacks effectiveness. We propose Mixed Reality DMT with interactive virtual agents, offering immersive content and feedback. Our novel self-guided training paradigm creates virtual twin agents that resemble children with ASD using a single photo, aiding them during training. In an experiment involving 24 children with ASD, self-guidance through the twin agent significantly improved training performance, particularly in movement quality and target-related responses. This approach has clinical potential in medical treatment and rehabilitation for children with ASD.
Session: Embodiment, Avatars and Presence (MO1J)
Date & Time: Monday, 18 March 2024, 14:00-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Rebecca Fribourg
Measuring Embodiment: Movement Complexity and the Impact of Personal Characteristics (Invited Journal: P3004)
Tabitha C. Peck, Davidson College, Davidson, NC, USA; Jessica J. Good, Davidson College, Davidson, NC, USA
A user's personal experiences and characteristics may impact the strength of an embodiment illusion and affect resulting behavioral changes in unknown ways. This paper presents a novel re-analysis of two fully-immersive embodiment user-studies (n=189 and n=99) using structural equation modeling, to test the effects of personal characteristics on subjective embodiment. Results demonstrate that individual characteristics (gender, participation in science, technology, engineering or math - Experiment 1, age, video gaming experience - Experiment 2) predicted differing self-reported experiences of embodiment Results also indicate that increased self-reported embodiment predicts environmental response, in this case faster and more accurate responses within the virtual environment. Importantly, head-tracking data is shown to be an effective objective measure for predicting embodiment, without requiring researchers to utilize additional equipment.
Embodying a self-avatar with a larger leg: its impacts on motor control and dynamic stability (Journal: P1418)
Valentin Vallageas, Imaging and orthopedics research laboratory; Rachid Aissaoui, CHUM research center; Iris Willaert, Ecole de technologie superieure; David Labbe PhD, Ecole de technologie superieure
Several studies demonstrate that virtual reality users can embody avatars with altered morphologies, adapting their mental body map (body schema) crucial for planning movements. This study explores how embodying avatars with enlarged legs affects motor planning. Thirty participants embodied avatars with different leg sizes, combined with two different embodiment levels. Gait initiation tasks showed no significant biomechanical changes. Asynchronous stimuli reduced embodiment without affecting stability measures. Deforming avatars might subtly influence motor execution in rehabilitation. The study suggests the adaptability of the body schema to morphological modifications, with implications for individuals with impaired mobility.
Human Factors at Play: Understanding the Impact of Conditioning on Presence and Reaction Time in Mixed Reality (Journal: P1076)
Yasra Chandio, University of Massachusetts, Amherst; Victoria Interrante, University of Minnesota; Fatima Muhammad Anwar, UMASS Amherst
This study investigates the link between presence in mixed reality and reaction time, focusing on psychological and physiological human aspects. Presence, usually gauged using subjective surveys, is now found to align with objective metrics (reaction time). Our research delves into how human conditioning impacts this relationship. An exploratory study involving 60 users under varied conditioning scenarios (control, positive, negative) discovered a notable correlation (-0.64) between presence scores and reaction times, suggesting that the impact of human factors on reaction time correlates with its effect on presence. Our study takes another critical step toward using objective and systemic measures like reaction time as a presence measure.
Age and Realism of Avatars in Simulated Augmented Reality: Experimental Evaluation of Anticipated User Experience (Conference: P1922)
Veronika Mikhailova, Technische Universität Ilmenau; Christoph Gerhardt, Technische Universität Ilmenau; Christian Kunert, Technische Universität Ilmenau; Tobias Schwandt, Technische Universität Ilmenau; Florian Weidner, Technische Universität Ilmenau; Wolfgang Broll, Technische Universität Ilmenau; Nicola Döring, Technische Universität Ilmenau
The study investigates the social attractiveness of avatars in simulated augmented reality (AR) based on a sample of N=2086 age-diverse participants from Germany. In an online setting, participants evaluated avatars representing different age groups (younger, middle-aged, older) and levels of realism (low, medium, high). Results demonstrated a strong preference for younger, high-realism avatars as communication partners and for self-representation in AR. However, older adults showed a tendency to opt for avatars resembling their actual age. The study provides insights into social interactions in AR, highlighting age-related stereotypes in avatar-based communication and underscoring the need for a more inclusive avatar design.
Session: Psycho- and Sociocultural Dimensions of Virtual Identities (MO2G)
Date & Time: Monday, 18 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Eric Hodgson
On the Emergence of Symmetrical Reality (Conference: P1098)
Zhenliang Zhang, Beijing Institute for General Artificial Intelligence; Zeyu Zhang, Beijing Institute for General Artificial Intelligence; Ziyuan Jiao, Beijing Institute for General Artificial Intelligence; Yao Su, Beijing Institute for General Artificial Intelligence; Hangxin Liu, Beijing Institute for General Artificial Intelligence; Wei Wang, Beijing Institute for General Artificial Intelligence; Song-Chun Zhu, Beijing Institute for General Artificial Intelligence
In this paper, we introduce the symmetrical reality framework, which offers a unified representation encompassing various forms of physical-virtual amalgamations. This framework enables researchers to better comprehend how AI agents can collaborate with humans and how distinct technical pathways of physical-virtual integration can be consolidated from a broader perspective. We then delve into the coexistence of humans and AI, demonstrating a prototype system that exemplifies the operation of symmetrical reality systems for specific tasks, such as pouring water. Finally, we propose an instance of an AI-driven active assistance service that illustrates the potential applications of symmetrical reality.
Bring Your Own Character: A Holistic Solution for Automatic Facial Animation Generation of Customized Characters (Conference: P1518)
Zechen Bai, National University of Singapore; Peng Chen, Institute of Software, Chinese Academy of Sciences; Xiaolan Peng, Institute of software,Chinese Academy of Sciences; Lu Liu, Institute of Software, Chinese Academy of Sciences; Naiming Yao, Institute of Software, Chinese Academy of Sciences; Hui Chen, Institute of Software, Chinese Academy of Sciences
This paper introduces a comprehensive approach to automatically generate facial animations for customized characters, irrespective of their blendshape topologies and texture appearances. The method involves estimating blendshape coefficients from input images or videos using a deep learning model. The proposed toolkit incorporates this model, featuring user-friendly interfaces and a human-in-the-loop scheme. Evaluation results demonstrate the flexibility to support personalized virtual character models. The toolkit facilitates easy and efficient facial animation generation, yielding satisfactory quality. Human-in-the-loop involvement enhances solution performance.
The Influence of Environmental Context on the Creation of Cartoon-like Avatars in Virtual Reality (Conference: P1737)
Pauline Bimberg, University of Trier; Michael Feldmann, Trier University; Benjamin Weyers, Trier University; Daniel Zielasko, University of Trier
The user study presented in this paper explores the effects that being immersed in different virtual scenes has on a user's avatar-design behavior. For this purpose, we have developed a character creation tool that lets users configure their appearance in Virtual Reality. This tool has then been employed in a user study involving 33 participants, who were asked to configure a virtual avatar in a beach and a hospital environment. Our results show that the environment that participants were immersed in influenced their design behavior, with the beach environment leading to a more extensive use of accessories than the hospital scene.
The Impact of Avatar Stylization on Trust (Conference: P1858)
Ryan Canales, Reality Labs Research, Meta; Doug Roble, Reality Labs Research, Meta; Michael Neff, Reality Labs Research, Meta
Virtual Reality (VR) allows people to choose any avatar to represent themselves. How does this choice impact social interaction that often relies on the establishment of trust? Are people more likely to trust a highly realistic avatar or is there flexibility in representation? This work presents a study exploring this question using a high stakes medical scenario. Participants meet three different doctors with three different style levels: realistic, caricatured, and an in-between ``Mid'' level. Trust ratings are largely consistent across style levels, but participants were more likely to select doctors with the ``Mid'' level of stylization for a second opinion. There is a clear preference against one of the three doctor identities.
Influence of Virtual Shoe Formality on Gait and Cognitive Performance in a VR Walking Task (Conference: P2102)
Sebastian Oberdörfer, University of Würzburg; Sandra Birnstiel, Friedrich-Alexander-Universität Erlangen-Nürnberg; Marc Erich Latoschik, University of Würzburg
Shoes come in various degrees of formality and their structure can affect human gait. In our study, we embody 39 participants with a generic avatar of the user's gender wearing three different pairs of shoes as within condition. The shoes differ in degree of formality. We measure the gait during a 2-minute walking task during which participants wore the same real shoe and assess selective attention using the Stroop task. Our results show significant differences in gait between the tested virtual shoe pairs. We found small effects between the three shoe conditions with respect to selective attention. However, we found no significant differences with respect to correct items and response time in the Stroop task.
Stepping into the Right Shoes: The Effects of User-Matched Avatar Ethnicity and Gender on Sense of Embodiment in Virtual Reality (Journal: P1126)
Tiffany D. Do, University of Central Florida; Camille Isabella Protko, University of Central Florida; Ryan P. McMahan, University of Central Florida
In many consumer VR applications, users embody predefined characters that offer minimal customization options, frequently emphasizing storytelling over choice. We investigated if matching a user's ethnicity and gender with their virtual self-avatar affects their affects their sense of embodiment in VR. A 2x2 experiment with diverse participants (n=32) showed matching ethnicity increased overall sense embodiment, irrespective of gender, impacting sense of appearance, response, and ownership. Our findings highlight the significance of avatar-user alignment for a more immersive VR experience.
Session: Multimodal Input and Interaction (MO2H)
Date & Time: Monday, 18 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Mayra Donaji Barrera Machuca
Impact of multimodal instructions for tool manipulation skills on performance and user experience in an Immersive Environment (Conference: P1762)
Cassandre Simon, Univ Evry,Université Paris Saclay; Manel Boukli Hacene, Univ Evry, Université Paris Saclay; Flavien Lebrun, Univ Evry, Université Paris Saclay; Samir Otmane, Univ Evry, Université Paris Saclay; Amine Chellali, Univ Evry, Université Paris Saclay
This study explores the use of multimodal communication to convey instructions to learners on the amplitude of movements to perform in an immersive environment. The study aims to examine the impact of four modality combinations on performance, workload, and user experience. The results show that participants achieved higher accuracy with the visual-haptic and verbal-visual-haptic conditions. Moreover, they performed the movements faster, and their movement trajectories were closer to the reference trajectories in the visual-haptic condition. Finally, the most preferred verbal-visual-haptic combination enhanced the sense of presence, co-presence, social presence, and learning experience. No impact on workload was observed.
Robust Dual-Modal Speech Keyword Spotting for XR Headsets (Journal: P1317)
Zhuojiang Cai, Beihang University; Yuhan Ma, Beihang University; Feng Lu, Beihang University
While speech interaction finds widespread utility within the Extended Reality (XR) domain, conventional vocal speech keyword spotting systems continue to grapple with formidable challenges, including suboptimal performance in noisy environments, impracticality in situations requiring silence, and susceptibility to inadvertent activations when others speak nearby. These challenges, however, can potentially be surmounted through the cost-effective fusion of voice and lip movement information. Consequently, we propose a novel vocal-echoic dual-modal keyword spotting system for XR headsets. Experimental results demonstrate the promising performance of this dual-modal system across various challenging scenarios.
Modeling the Intent to Interact with VR using Physiological Features (Invited Journal: P3009)
Willy Nguyen, Universite Paris-Saclay, France; Klaus Gramann, TU Berlin, Germany; Lukas Gehrke, TU Berlin, Germany
Objective . Mixed-Reality (XR) technologies promise a user experience (UX) that rivals the interactive experience with the real-world. The key facilitators in the design of such a natural UX are that the interaction has zero lag and that users experience no excess mental load. This is difficult to achieve due to technical constraints such as motion-to-photon latency as well as false-positives during gesture-based interaction.
CAEVR: Biosignals-Driven Context-Aware Empathy in Virtual Reality (Journal: P1845)
Kunal Gupta, The University of Auckland; Yuewei Zhang, The University of Auckland; Tamil Selvan Gunasekaran, The University of Auckland; Nanditha Krishna, Amrita Vishwa Vidyapeetham; Yun Suen Pai, Keio University Graduate School of Media Design; Mark Billinghurst, The University of Auckland
This study examines the impact of Context-Aware Empathic VR (CAEVR) on users' emotions and cognition in VR. We developed personalized and generalized emotion recognition models using real-time electroencephalography, electrodermal activity, and heart rate variability data. These models were applied in a Context-Aware Empathic virtual agent and an Emotion-Adaptive VR environment. Results show increased positive emotions, cognitive load, and empathy towards the CAE agent. This suggests CAEVR's potential for enhancing user-agent interactions. The paper concludes with lessons and future research directions.
VRMN-bD: A Multi-modal Natural Behavior Dataset of Immersive Human Fear Responses in VR Stand-up Interactive Games (Conference: P1128)
He Zhang, Tsinghua University; Xinyang Li, Tsinghua University; Yuanxi Sun, Communication University of China; Xinyi Fu, Tsinghua University; Christine Qiu, KTH Royal Institute of Technology; John M. Carroll, Pennsylvania State University
This study focuses on analyzing fear in virtual reality (VR) using horror games. We collected multi-modal data (posture, audio, physiological signals) from 23 participants to understand fear responses. Our LSTM-based model achieved prediction accuracies of 65.31% (6-level fear classification) and 90.47% (2-level fear classification). We developed the VRMNbD, a unique multi-modal dataset focusing on natural human fear responses in VR interactive environments, surpassing existing datasets in data scale, collection method, and audience scope. This research contributes to advancements in immersive game development, scene creation, and virtual human-computer interactions by providing insights into fear emotions in VR environments.
Session: Perception and Cognition (MO2J)
Date & Time: Monday, 18 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Bobby Bodenheimer
ACHOO - Bless you! Sense of Presence can provoke Proactive Mucosal Immune Responses in Immersive Human-Agent Interactions (Conference: P1247)
Judith Katharina Keller, Universität Hamburg; Agon Kusari, Universität Hamburg; Sophie Czok, Universität Hamburg; Birgit Simgen, MVZ Volkmann Laboratory; Frank Steinicke, Universität Hamburg; Esther Diekhof, Universität Hamburg
Previous work suggests that the mere visual perception of disease cues can proactively enhance mucosal immune responses even without actual pathogen exposure. We present the first immersive immunological experiment, which investigates if social interactions with virtual agents in VR can lead to a mucosal immune response, in particular, a proactive release of secretory immunoglobin A (sIgA) in saliva. Therefore, we simulated a virtual bus stop scenario in which participants were required to closely approach and establish eye contact with ten agents. We found that sIgA secretion increased when agents sneezed as well as when they did not sneeze. In the latter, the increase was correlated with the perceived involvement and sense of presence.
Evaluating Text Reading Speed in VR Scenes and 3D Particle Visualizations (Journal: P1072)
Johannes Novotny PhD, VRVis Zentrum für Virtual Reality und Visualisierung; David H. Laidlaw, Brown University
We report on the effects of text size and display parameters on reading speed and legibility in three state-of-the-art VR displays. Two are head-mounted displays, and one is Brown’s CAVE-like YURT. Our two perception experiments uncover limits where reading speed declines as the text size approaches the so-called critical print sizes (CPS) of individual displays. We observe an inverse correlation between display resolution and CPS, revealing hardware-specific limitations on legibility beyond display resolution, making CPS an effective benchmark for VR devices. Additionally, we report on the effects of text panel placement, orientation, and occlusion-reducing rendering methods on reading speeds in volumetric particle visualization.
The Effects of Auditory, Visual, and Cognitive Distractions on Cybersickness in Virtual Reality (Invited Journal: P3014)
Rohith Venkatakrishnan, School of Computing, Clemson University, USA; Roshan Venkatakrishnan, School of Computing, Clemson niversity, USA; Balagopal Raveendranath, Department of Psychology, Clemson University, USA; Dawn M. Sarno, Department of Psychology, Clemson University, USA; Andrew C. Robb, School of Computing, Clemson University, USA; Wen-Chieh Lin, Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan; Sabarish V. Babu, School of Computing, Clemson University, USA
Cybersickness (CS) is one of the challenges that has hindered the widespread adoption of Virtual Reality (VR). Consequently, researchers continue to explore novel means to mitigate the undesirable effects associated with this affliction, one that may require a combination of remedies as opposed to a solitary stratagem. Inspired by research probing into the use of distractions as a means to control pain, we investigated the efficacy of this countermeasure against CS, studying how the introduction of temporally time-gated distractions affects this malady during a virtual experience featuring active exploration. Downstream of this, we discuss how other aspects of the VR experience are affected by this intervention. We discuss the results of a between-subjects study manipulating the presence, sensory modality, and nature of periodic and short-lived (5-12 seconds) distractor stimuli across 4 experimental conditions: (1) no-distractors (ND); (2) auditory distractors (AD); (3) visual distractors (VD); (4) cognitive distractors (CD). Two of these conditions (VD and AD) formed a yoked control design wherein every matched pair of ‘seers’ and ‘hearers’ was periodically exposed to distractors that were identical in terms of content, temporality, duration, and sequence. In the CD condition, each participant had to periodically perform a 2-back working memory task, the duration and temporality of which was matched to distractors presented in each matched pair of the yoked conditions. These three conditions were compared to a baseline control group featuring no distractions. Results indicated that the reported sickness levels were lower in all three distraction groups in comparison to the control group. The intervention was also able to both increase the amount of time users were able to endure the VR simulation, as well as avoid causing detriments to spatial memory and virtual travel efficiency. Overall, it appears that it may be possible to make users less consciously aware and bothered by the symptoms of CS, thereby reducing its perceived severity.
Investigating Personalization Techniques for Improved Cybersickness Prediction in Virtual Reality Environments (Journal: P1370)
Umama Tasnim, University of Texas at San Antonio; Rifatul Islam, Kennesaw State University; Kevin Desai, The University of Texas at San Antonio; John Quarles, University of Texas at San Antonio
Recent cybersickness research uses physiological data (HR, EDA) for prediction. However, the role of individual factors like age and gender in these models is unclear. Our study aims to fill this gap, advocating for personalized cybersickness prediction models for an inclusive virtual reality experience. We tested four personalization techniques: data grouping, transfer learning, early shaping, and sample weighing, using an open-source dataset. Results showed a marked improvement in prediction accuracy; for example, DeepTCN's early shaping reduced RMSE by 69.7% compared to the generic model. This underscores the potential of personalized models in enhancing cybersickness prediction, paving the way for future tailored reduction techniques.
Age and Gender Differences in the Pseudo-Haptic Effect on Computer Mouse Operation in a Desktop Environment (Invited Journal: P3011)
Yuki Ban, Graduate School of Frontier Sciences, University of Tokyo, Chiba, Japan; Yusuke Ujitoko, NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Atsugi, Japan
Pseudo-haptics is a method that can provide a haptic sensation without requiring a physical haptic device. The effect of pseudo-haptics is known to depend on the individual, but it is unclear which factors cause individual differences. As the first study establishing a calibration method for these differences in future research, we examined the differences in the pseudo-haptic effect on mouse cursor operation in a desktop environment depending on the age and gender of the user. We conducted an online experiment and collected data from more than 400 participants. The participants performed a task of lifting a virtual object with a mouse pointer. We found that the effect of pseudo-haptics was greater in younger or male participants than in older or female participants. We also found that the effect of pseudo-haptics, which varied with age and gender, can be explained by habituation to the mouse in daily life and the accuracy of detecting the pointer position using vision or proprioception. Specifically, the pseudo-haptic effect was higher for those who used the mouse more frequently and had higher accuracy in identifying the pointer position using proprioception or vision. The results of the present study not only indicate the factors that cause age and gender differences but also provide hints for calibrating these differences.
Multimodal Physiological Analysis of Impact of Emotion on Cognitive Control in VR (Journal: P1899)
Ming Li, Beihang University; Junjun Pan, Beihang University; Yu Li, Beijing Normal University; Yang Gao, Beihang university; Hong Qin, Stony Brook University; Yang Shen, Collaborative Innovation Center of Assessment for Basic Education Quality, Beijing Normal University
Cognitive control is perplexing to elucidate and can be influenced by emotions. Understanding individual cognitive control in VR is crucial for adaptive applications. In this study, we investigate the influence of emotions on cognitive control based on the arousal valence model. 26 participants are recruited, inducing emotions through VR videos and then performing related cognitive control tasks. Leveraging EEG, HRV, and EDA, we employ deep learning to categorize cognitive control levels. The experiment results demonstrate that high-arousal emotions significantly enhance users’ cognitive control abilities and achieve an accuracy of 84.52% in distinguishing between high and low cognitive control.
Session: 3D Interaction and Touch (TU1G)
Date & Time: Tuesday, 19 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Bruce Thomas
Exploring Bimanual Haptic Feedback for Spatial Search in Virtual Reality (Journal: P1812)
BoYu Gao, Jinan University; Tong Shao, Jinan University; Huawei Tu, La Trobe University; Qizi Ma, Jinan University; Zitao Liu, Jinan University; Teng Han, Institute of Software, Chinese Academy of Sciences
Spatial search tasks are common and crucial in many Virtual Reality (VR) applications. In this work, we explored bimanual haptic feedback with various combinations of haptic properties, where four types of bimanual haptic feedback were designed, for spatial search tasks in VR. Two experiments were designed to evaluate the effectiveness of bimanual haptic feedback on spatial direction guidance and search in VR. The results of the experiments showed that the proposed bimanual haptic feedback can provide more efficient and accurate performance than the baselines for spatial guidance and search in VR. Based on these findings, we have derived a set of design recommendations for spatial search using bimanual haptic feedback in VR.
FanPad: A Fan Layout Touchpad Keyboard for Text Entry in VR (Conference: P1301)
Jian Wu, Beihang University; Ziteng Wang, Beihang University; Lili Wang, Beihang University; Jiaheng Li, Beihang University; Yuhan Duan, Beihang University
Text entry poses a significant challenge in virtual reality (VR). This paper introduces FanPad, a novel solution that facilitates dual-hand text input within head-mounted displays (HMDs). FanPad accomplishes this by ingeniously mapping and curving the QWERTY keyboard onto the touchpads of both controllers. The curved key layout of FanPad is derived from the natural movement of the thumb when interacting with the touchpad, resembling an arc with a thumb-length fixed radius. We have conducted two comprehensive user studies to assess and evaluate the performance of our FanPad method. Notably, novices achieved a typing speed of 19.73 words per minute (WPM). The highest typing speed reached an impressive 24.19 WPM.
Eye-Hand Coordination Training: A Systematic Comparison of 2D, VR, and AR Screen Technologies and Task Motives (Conference: P1309)
Aliza Aliza, Kadir Has University; Irene zaugg, Colorado State University; Elif Çelik, Kadir Has University; Wolfgang Stuerzlinger, Simon Fraser University; Francisco Raul Ortega, Colorado State University; Anil Ufuk Batmaz, Concordia University; Mine Sarac, Kadir Has University
In this paper, we compare user motor performance with an EHCT setup in Augmented Reality (AR), Virtual Reality (VR), and on a 2D touchscreen display in a longitudinal study. Through a ten-day user study, we thoroughly analyzed the motor performance of twenty participants with five task instructions focusing on speed, error rate, accuracy, precision, and none. As a novel evaluation criterion, we also analyzed the participants’ performance in terms of effective throughput. The results showed that each task instruction has a different effect on one or more psychomotor characteristics of the trainee, which high- lights the importance of personalized training programs.
The Benefits of Near-field Manipulation and Viewing to Distant Object Manipulation in VR (Conference: P1663)
Sabarish V. Babu, Clemson University; Wei-An Hsieh, National Yang Ming Chiao Tung University; Jung-Hong Chuang, National Yang Ming Chiao Tung University
In this contribution, we propose to enhance two distant object manipulation techniques, BMSR and Scaled HOMER, via near-field scaled replica manipulation and viewing. In the proposed Direct BMSR, users are allowed to directly manipulate the target replica in their arm's reach space. In Scaled HOMER+NFSRV, Scaled HOMER is augmented with a near-field scaled replica view of the target object and its context. We conducted a between-subjects empirical evaluation of BMSR, Direct BMSR, Scaled HOMER, and Scaled HOMER+NFSRV. Our findings revealed that Direct BMSR and Scaled HOMER+NFSRV significantly outperformed BMSR and Scaled HOMER, respectively, in terms of accuracy.
Evaluating an In-Hand Ball-Shaped Controller for Object Manipulation in Virtual Reality (Conference: P1978)
Sunbum Kim, School of Computing, KAIST; Geehyuk Lee, School of Computing, KAIST
This study explored the use of a ball-shaped controller for object manipulation in virtual reality. We developed a ball-shaped controller with pressure-sensing capabilities featuring specifically designed interactions for object manipulation, including selection, translation, rotation, and scaling. We evaluated it on tasks involving both 6-DOF and 7-DOF object manipulation, including close and distant ranges. The results indicated that the ball-shaped controller performed similarly to VR controller methods for direct manipulation but excelled in reducing completion times and task load for distant object manipulation. Additionally, the ball-shaped controller minimized wrist and arm movements and was the preferred method among participants.
Session: Multisensory Interfaces (TU1H)
Date & Time: Tuesday, 19 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Daniel Roth
Exploring audio interfaces for vertical micro-guidance in augmented reality via hand-based feedback (Journal: P1062)
Renan L Martins Guarese, RMIT; Emma Pretty, RMIT; Aidan Renata, Royal Melbourne Institute of Technology; Debra Polson, Queensland University of Technology; Fabio Zambetta, RMIT University
This research proposes an evaluation of pitch-based sonification methods via user experiments in real-life scenarios, specifically vertical guidance, with the aim of standardizing the use of audio interfaces in AR in guidance tasks. Using literature on assistive technology for people who are visually impaired, we aim to generalize their applicability to a broader population and for different use cases. We propose and test sonification methods for vertical guidance in hand-navigation assessments with users without visual feedback. Including feedback from a visually impaired expert in digital accessibility, results outlined that methods that do not rely on memorizing pitch had the most promising accuracy and workload performances.
Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenes (Conference: P1075)
Anton Jeran Ratnarajah, University of Maryland, College Park; Dinesh Manocha, University of Maryland
We present an end-to-end binaural audio rendering approach (Listen2Scene) for VR and AR applications. We propose a novel neural-network-based binaural sound propagation method to generate acoustic effects for indoor 3D models of real environments. Any clean audio or dry audio can be convolved with the generated acoustic effects to render audio corresponding to the real environment. We have evaluated the accuracy of our approach with binaural acoustic effects generated using an interactive geometric sound propagation algorithm and captured real acoustic effects / real-world recordings. The demo videos, code and dataset are available online \footnote{\url{https://anton-jeran.github.io/Listen2Scene/}.
Influence of user's body in olfactory virtual environment generated by real-time CFD (Conference: P1513)
Masafumi Uda, School of Engineering; Takamichi Nakamoto, Tokyo Institute of Technology
We have developed a virtual olfactory environment using an olfactory display and computational fluid dynamics (CFD) simulation. Although CFD can calculate the odor distribution in the complicated geometry, its computational cost was expensive and did not work in real time in the previous study. In this study, real-time CFD based on GPU calculation was introduced to generate real-time olfactory VR environment. We investigated influence of the user's body with its location and orientation changing irregularly. The experimental result indicates the usefulness of considering the effect of the user's body since we cannot avoid that influence.
OdorAgent: Generate Odor Sequences for Movies Based on Large Language Model (Conference: P1942)
Yu Zhang, Tsinghua University; Peizhong Gao, Tsinghua University; Fangzhou Kang, Tsinghua Univerity; Jiaxiang Li, Tsinghua University; Jiacheng Liu, Tsinghua University; Qi Lu, Tsinghua University; YINGQING XU, Tsinghua University
Numerous studies have shown that integrating scents into movies enhances viewer engagement and immersion. However, creating such olfactory experiences often requires professional perfumers to match scents, limiting their widespread use. To address this, we propose OdorAgent which combines a LLM with a text-image model to automate video-odor matching. The generation framework is in four dimensions: subject matter, emotion, space, and time. We applied it to a specific movie and conducted user studies to evaluate and compare the effectiveness of different system elements. The results indicate that OdorAgent possesses significant scene adaptability and enables inexperienced individuals to design odor experiences for video and images.
EmoFace: Audio-driven Emotional 3D Face Animation (Conference: P1973)
Chang Liu, Shanghai Jiao Tong University; Qunfen Lin, Tencent Games; Zijiao Zeng, Tencent Games; Ye Pan, Shanghai Jiaotong University
EmoFace is a novel audio-driven methodology for creating facial animations with vivid emotional dynamics. It has the ability to generate dynamic facial animations with diverse emotions, synchronized lip movements, and natural blinks. Incorporating independent speech and emotion encoders, our approach establishes a robust link between audio, emotion, and facial controller rigs. Post-processing techniques enhance authenticity, focusing on blinks and eye movements. We also contribute an emotional audio-visual dataset and derive control parameters for each frames to drive MetaHuman models. Quantitative assessments and user studies validate the efficacy of our innovative approach.
Session: Evaluating Immersion: UX and Interaction (TU1J)
Date & Time: Tuesday, 19 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Doug Bowman
Task-based methodology to characterise immersive user experience with multivariate data (Conference: P1181)
Florent Robert, Université Côte d'Azur; Hui-Yin Wu, Centre Inria d'Université Côte d'Azur; Lucile Sassatelli, Université Côte d'Azur; Marco Winckler, Université Côte d'Azur
Virtual Reality technologies is promising for research, however, the evaluation of the user experience in immersive environments is daunting, the richness of the media presents challenges to synchronise context with behavioural metrics. We propose a task-based methodology that provides fine-grained descriptions and analyses of the experiential user experience in VR that (1) aligns low-level tasks with behavioural metrics, (2) defines performance components with baseline values to evaluate task performance, and (3) characterise task performance with multivariate user behaviour data. We find that the methodology allows us to better observe the experiential user experience by highlighting relations between user behaviour and task performance.
NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods (Journal: P1221)
Qiang Qu, The University of Sydney; hanxue Liang, University of Cambridge; Xiaoming Chen, Beijing Technology and Business University; Yuk Ying Chung, The University of Sydney; Yiran Shen, Shandong University
Neural View Synthesis (NVS) creates dense viewpoint videos from sparse images, but traditional metrics like PSNR and SSIM inadequately assess NVS and NeRF-synthesized scenes. Limited dense ground truth views in datasets like LLFF hinder full-reference quality evaluation. Addressing this, we introduce NeRF-NQA, a novel no-reference quality assessment method for NVS and NeRF scenes. It uniquely combines viewwise and pointwise evaluations to assess spatial and angular qualities. Extensive testing against 23 established visual quality methods demonstrates NeRF-NQA's superior performance in assessing NVS-synthesized scenes without reference data, marking a significant advancement in NVS quality assessment.
Exploring Controller-based Techniques for Precise and Rapid Text Selection in Virtual Reality (Conference: P1271)
Jianbin Song, Xi'an Jiaotong-Liverpool University; Rongkai Shi, Xi'an Jiaotong-Liverpool University; Yue Li, Xi'an Jiaotong-Liverpool University; BoYu Gao, Jinan University; Hai-Ning Liang, Xi'an Jiaotong-Liverpool University
Text selection, while common, can be difficult because the letters and words are too small and clustered together to allow precise selection. There has been limited exploration of techniques that support accurate and rapid text selection at the character, word, sentence, or paragraph levels in VR HMDs. We present three controller-based text selection methods: Joystick Movement, Depth Movement, and Wrist Orientation. They are evaluated against a baseline method via a user study with 24 participants. Results show that the three proposed techniques significantly improved the performance and user experience over the baseline, especially for the selection beyond the character level.
Evaluating the Effect of Binaural Auralization on Audiovisual Plausibility and Communication Behavior in Virtual Reality (Conference: P1872)
Felix Immohr, Technische Universität Ilmenau; Gareth Rendle, Bauhaus-Universität Weimar; Anton Benjamin Lammert, Bauhaus-Universität Weimar; Annika Neidhardt, University of Surrey; Victoria Meyer Zur Heyde, Technische Universität Ilmenau; Bernd Froehlich, Bauhaus-Universität Weimar; Alexander Raake, TU Ilmenau
Spatial audio has been shown to positively impact user experience in traditional communication media and presence in single-user VR. This work further investigates whether spatial audio benefits immersive communication scenarios. We present a study in which dyads communicate in VR under different auralization and scene arrangement conditions. A novel task is designed to increase the relevance of spatial hearing. Results are obtained through social presence and plausibility questionnaires, and through conversational and behavioral analysis. Although participants are shown to favor binaural over diotic audio in a direct comparison, no significant differences were observed from the other presented measures.
Eyes on the Task: Gaze Analysis of Situated Visualization for Collaborative Tasks (Conference: P1898)
Nelusa Pathmanathan, University of Stuttgart; Tobias Rau, University of Stuttgart; Xiliu Yang, Institute of Computational Design and Construction; Aimée Sousa Calepso, University of Stuttgart; Felix Amtsberg, Institute of Computational Design and Construction; Achim Menges, University of Stuttgart; Michael Sedlmair, University of Stuttgart; Kuno Kurzhals, University of Stuttgart
The use of augmented reality technology to support humans within complex and collaborative tasks has gained increasing importance. Analyzing collaboration patterns is usually done by conducting observations and interviews. We argue that eye tracking can be used to extract further insights and quantify behavior. To this end, we contribute a study that uses eye tracking to investigate participant strategies for solving collaborative sorting and assembly tasks. We compare participants' visual attention during situated instructions in AR and traditional paper-based instructions as a baseline. By investigating the performance and gaze behavior of the participants, different strategies for solving the provided tasks are revealed.
Session: Locomotion and Redirection (TU2G)
Date & Time: Tuesday, 19 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Evan Ruma Rosenberg
illumotion: An Optical-illusion-based VR Locomotion Technique for Long-Distance 3D Movement (Conference: P1841)
Zackary P. T. Sin, The Hong Kong Polytechnic University; Ye Jia, The Hong Kong Polytechnic University; Chen Li, The Hong Kong Polytechnic University; Hong Va Leong, The Hong Kong Polytechnic University; qing li, The Hong Kong Polytechnic University; Peter H.F. Ng, The Hong Kong Polytechnic University
Locomotion has a marked impact on user experience in VR, but common to-go techniques such as steering and teleportation have their limitations. Particularly, steering is prone to cybersickness, while teleportation trades presence for mitigating cybersickness. Inspired by how we manipulate a picture on a phone, we propose illumotion, an optical-illusion-based method that, we believe, can be an alternative for locomotion. Instead of zooming in a picture by pinching two fingers, we can move forward by “zooming” toward part of the 3D virtual scene with pinched hands. Results show that, compared with either teleportation, steering or both, illumotion has better performance, presence, usability, user experience and cybersickness alleviation.
Exploring Visual-Auditory Redirected Walking using Auditory Cues in Reality (Invited Journal: P3007)
Kumpei Ogawa; Kazuyuki Fujita; Shuichi Sakamoto; Kazuki Takashima; Yoshifumi Kitamura
We examine the effect of auditory cues occurring in reality on redirection. Specifically, we set two hypotheses: the auditory cues emanating from fixed positions in reality (Fixed sound, FS) increase the noticeability of redirection, while the auditory cues whose positions are manipulated consistently with the visual manipulation (Redirected sound, RDS) decrease the noticeability of redirection. To verify these hypotheses, we implemented an experimental environment that virtually reproduced FS and RDS conditions using binaural recording, and then we conducted a user study (N = 18) to investigate the detection thresholds (DTs) for rotational manipulation and the sound localization accuracy of the auditory cues under FS and RDS, as well as the baseline condition without auditory cues (No sound, NS). The results show, against the hypotheses, FS gave a wider range of DTs than NS, while RDS gave a similar range of DTs to NS. Combining these results with those of sound localization accuracy reveals that, rather than the auditory cues affecting the participants’ spatial perception in VR, the visual manipulation made their sound localization less accurate, which would be a reason for the increased range of DTs under FS. Furthermore, we conducted a follow-up user study (N = 11) to measure the sound localization accuracy of FS where the auditory cues were actually placed in a real setting, and we found that the accuracy tended to be similar to that of virtually reproduced FS, suggesting the validity of the auditory cues used in this study. Given these findings, we also discuss potential applications.
RedirectedDoors+: Door-Opening Redirection with Dynamic Haptics in Room-Scale VR (Journal: P1074)
Yukai Hoshikawa, Tohoku University; Kazuyuki Fujita, Tohoku University; Kazuki Takashima, Tohoku University; Morten Fjeld, Chalmers University of Technology; Yoshifumi Kitamura, Tohoku University
RedirectedDoors+ is a robot-based VR system that enhances the original RedirectedDoors by offering dynamic haptics during consecutive door openings. It utilizes door robots, a robot-positioning algorithm for just-in-time haptic feedback, and a user-steering algorithm for user navigation within limited areas. A simulation study, tested in six VR environments, reveals our system’s performance in relation to user walking speed, paths, and the number of door robots, leading to the derivation of usage guidelines. Additionally, a study with 12 participants confirms the system’s effectiveness in providing haptic feedback and redirecting users in confined spaces during a walkthrough application.
Virtual Steps: The Experience of Walking for a Lifelong Wheelchair User in Virtual Reality (Conference: P1509)
Atieh Taheri, University of California, Santa Barbara; Arthur Caetano, University of California, Santa Barbara; Misha Sra, UCSB
We co-designed a VR walking experience with a person with Spinal Muscular Atrophy who has been a lifelong wheelchair user. Over 9 days, we collected and analyzed data on this person's experience through a diary study to understand the required design elements. Given that they had only seen others walking and had not directly experienced it, determining which design parameters must be considered to match the virtual experience to their mental model was challenging. Generally, we found the experience of walking to be quite positive, providing a perspective from a higher vantage point than what was available in a wheelchair.
APF-S2T: Steering to Target Redirection Walking Based on Artificial Potential Fields (Journal: P1943)
Jun-Jie Chen, National Yang Ming Chiao Tung University; Huan-Chang Hung, National Yang Ming Chiao Tung University; Yu-Ru Sun, National Yang Ming Chiao Tung University; Jung-Hong Chuang, National Yang Ming Chiao Tung University
This paper introduced a novel APF-based redirected walking controller, called APF Steer-to-Target (APF-S2T). Different from previous APF-based controllers, APF-S2T locates the target sample with the lowest score within the user's walkable areas in both physical and virtual space. The score is defined based on the APF values and distance to the user. The direction from the user position to the target sample serves as the steering direction that will be used to set RDW gains. A comparative simulation-based evaluation reveals that APF-S2T performed better than the state-of-the-art controllers in terms of the number of reset and the average reset distance.
SafeRDW: Keep VR Users Safe When Jumping with Redirected Walking (Conference: P2110)
Sen-Zhe Xu, Tsinghua University; Kui Huang, Tsinghua University; Cheng-Wei Fan, Tsinghua University; Song-Hai Zhang, Tsinghua University
Existing redirected walking (RDW) algorithms typically focus on reducing collisions between users and obstacles during walking but overlook the safety when users perform significant actions such as jumping. This oversight can pose serious risks to users during VR exploration, especially when there are physical obstacles or boundaries near the virtual locations that require user jumping. We propose SafeRDW, the first RDW algorithm that takes the user's jumping safety into consideration. The proposed method can redirect users to safe physical locations when a jump is required in the virtual space, ensuring user safety. Simulation experiments and user study results both show that our method not only reduces the number of resets, but also significantly ensures user safety when they reach the jumping points in the virtual scene.
Session: Projections (TU2H)
Date & Time: Tuesday, 19 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Bruce Thomas
ViComp: Video Compensation for Projector-Camera Systems (Journal: P1096)
Yuxi Wang, Hangzhou Dianzi University; Haibin Ling, Stony Brook University; Bingyao Huang, Southwest University
Projector video compensation aims to cancel the geometric and photometric distortions caused by non-ideal projection surfaces and environments when projecting videos. This paper builds an online video compensation system that compensates frames and adjusts model parameters in parallel. By integrating an efficient deep learning-based compensation model, our system can be rapidly configured to unknown environments with good performance. Moreover, the proposed update strategy combining long-term and short-term memory mechanisms enables the compensation model to adapt to the target configuration and video content without manual intervention. Experiments show that our system significantly outperforms state-of-the-art baselines.
Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors (Journal: P1121)
Masaki Takeuchi, Osaka University; Hiroki Kusuyama, Osaka University; Daisuke Iwai, Osaka University; Kosuke Sato, Osaka University
Projection mapping (PM) typically requires a dark environment to achieve high-quality projections, limiting its practicality. In this paper, we overcome this limitation by replacing conventional room lighting with heterogeneous projectors. These projectors replicate environmental lighting by selectively illuminating the scene, excluding the projection target. Our contributions include a distributed projector optimization framework designed to effectively replicate environmental lighting and the incorporation of a large-aperture projector to reduce high-luminance emitted rays and hard shadows. Our findings demonstrate that our projector-based lighting system significantly enhances the contrast and realism of PM results.
Projection Mapping with a Brightly Lit Surrounding Using a Mixed Light Field Approach (Journal: P1606)
Masahiko Yasui, Tokyo Institute of Technology; Ryota Iwataki, Tokyo Institute of Technology; Masatoshi Ishikawa, Tokyo University of Science; Yoshihiro Watanabe, Tokyo Institute of Technology
Projection mapping (PM) exhibits suboptimal performance in well-lit environments because of the ambient light. This interference degrades the contrast of the projected images. To overcome these limitations, we introduce an innovative approach that leverages a mixed light field, blending traditional PM with ray-controllable ambient lighting. This methodological combination ensures that the projector exclusively illuminates the PM target, preserving the optimal contrast. Furthermore, we propose the integration of a kaleidoscopic array with integral photography to generate dense light fields for ray-controllable ambient lighting. Our optical simulations and the developed system collectively validate the effectiveness of our approach.
Real-time Seamless Multi-Projector Displays on Deformable Surfaces (Journal: P2001)
Muhammad Twaha Ibrahim, UC Irvine; Gopi Meenakshisundaram, University of California, Irvine; Aditi Majumder, UCI
Prior work on multi-projector displays on deformable surfaces have focused have focused mostly on small scale single projector displays. In this work, we present the first end-to-end solution for achieving a real-time, seamless display on deformable surfaces using mutliple unsychronized projectors without requiring any prior knowledge of the surface or device parameters. Using multiple projectors and RGB-D cameras, we provide the much desired aspect of scale to the displays on deformable and dynamic surfaces. This work has tremendous applications on mobile and expeditionary systems such as military or emergency operations in austere locations, or displays on inflatable objects for tradeshows/events and touring edutainment applications.
Towards Co-operative Beaming Displays: Dual Steering Projectors for Extended Projection Volume and Head Orientation Range (Journal: P2043)
Hiroto Aoki, The University of Tokyo; Takumi Tochimoto, Tokyo Institute of Technology; Yuichi Hiroi, Cluster Inc.; Yuta Itoh, The University of Tokyo
This study tackles trade-offs in existing near-eye displays (NEDs) by introducing a beaming display with dual steering projectors. While the traditional NED faces challenges in size, weight, and user limitations, the beaming display separates the NED into a steering projector (SP) and a passive headset. To overcome issues with a single SP, dual projectors are distributed to extend head orientation. A geometric model and calibration method for multiple projectors are proposed. The prototype achieves precision (1.8 ∼ 5.7 mm) and delay (14.46 ms) at 1m, projecting images in the passive headset's area (20 mm × 30 mm) and enabling multiple users with improved presentation features.
3D Gamut Morphing for Non-Rectangular Multi-Projector Displays (Invited Journal: P3006)
Mahdi Abbaspour Tehrani, Genentech., USA; Muhammad Twaha Ibrahim, University of California, Irvine, USA; Aditi Majumder, University of California, Irvine, USA; M. Gopi, University of California, Irvine, USA
In a spatially augmented reality system, multiple projectors are tiled on a complex shaped surface to create a seamless display on it. This has several applications in visualization, gaming, education and entertainment. The main challenges in creating seamless and undistorted imagery on such complex shaped surfaces are geometric registration and color correction. Prior methods that provide solutions for the spatial color variation in multi-projector displays assume rectangular overlap regions across the projectors that is possible only on flat surfaces with extremely constrained projector placement. In this paper, we present a novel and fully automated method for removing color variations in a multi-projector display on arbitrary shaped smooth surfaces using a general color gamut morphing algorithm that can handle any arbitrarily shaped overlap between the projectors and assures imperceptible color variations across the display surface.
Session: 3D Interaction and Teleoperation (TU2J)
Date & Time: Tuesday, 19 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Tim Weisker
Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation (Conference: P1006)
Shutaro Aoyama, Columbia University; Jen-Shuo Liu, Columbia University; Portia Wang, Columbia University; Shreeya Jain, Columbia University; Xuezhen Wang, Columbia University; Jingxi Xu, Columbia University; Shuran Song, Columbia University; Barbara Tversky, Columbia Teachers College; Steven Feiner, Columbia University
We present a prototype virtual reality user interface for robot teleoperation that supports high-level goal specification in remote assembly tasks. Users interact with virtual replicas of task objects. They asynchronously assign multiple goals in the form of 6DoF destination poses without needing to be familiar with specific robots and their capabilities, and manage and monitor the execution of these goals. The user interface employs two different spatiotemporal visualizations for assigned goals: one represents all goals within the user’s workspace, while the other depicts each goal within a separate world in miniature. We conducted a user study of the interface without the robot system to compare these visualizations.
Exploring Bi-Manual Teleportation in Virtual Reality (Conference: P1602)
Siddhanth Raja Sindhupathiraja, Indian Institute of Technology Delhi; A K M Amanat Ullah, University of British Columbia, Okanagan; William Delamare, ESTIA; Khalad Hasan, University of British Columbia
Enhanced hand tracking in modern VR headsets has popularized hands-only teleportation that allows instantaneous movement within VR environments. However, previous works on hands-only teleportation have not fully explored - the potential of bi-manual input (where each hand plays a distinct role), the influence of users’ posture (whether sitting or standing), or assessments based on human motor models (such as Fitts’ Law). To address these gaps, we conducted a user study (N=20) to compare the performance of bi-manual and uni-manual techniques in VR teleportation tasks using a proposed Fitts’ Law model, considering both the postures. Results showed that bi-manual techniques enable faster teleportation and are more accurate than other methods.
BaggingHook: Selecting Moving Targets by Pruning Distractors Away for Intention-Prediction Heuristics in Dense 3D Environments (Conference: P1733)
Paolo Boffi, King Abdullah University of Science and Technology (KAUST); Alexandre Kouyoumdjian, King Abdullah University of Science and Technology; Manuela Waldner, TU Wien; Pier Luca Lanzi, Politecnico di Milano; Ivan Viola, King Abdullah University of Science and Technology
This study presents two novel selection techniques: BaggingHook and AutoBaggingHook, based on distractor pruning and built upon the Hook intention-prediction heuristic. Our techniques reduce the number of targets in the environment, making them semi-transparent, to expedite heuristic convergence and reduce occlusion. BaggingHook allows manual distractor pruning, while AutoBaggingHook employs automated, score-based pruning. Results from a user study that compared both techniques to the Hook baseline show AutoBaggingHook was the fastest, while BaggingHook was preferred by most users for its greater user control. This work highlights the benefits of varying inputs in intention-prediction heuristics to improve performance and user experience.
SkiMR: Dwell-free Eye Typing in Mixed Reality (Conference: P1936)
Jinghui Hu, University of Cambridge; John J Dudley, University of Cambridge; Per Ola Kristensson, University of Cambridge
We present SkiMR: a dwell-free eye typing system that enables fast and accurate hands-free text entry on mixed reality headsets. SkiMR uses a statistical decoder to infer users' intended text based on users' eye movements on a virtual keyboard, bypassing the need for dwell timeouts. We conducted two studies with a HoloLens 2: the first (n=12) showed SkiMR's superiority in speed over traditional dwell-based and a hybrid method. The second study (n=16) focused on composition tasks with a refined system, revealing that users could compose text at 12 words per minute with a 1.1% corrected error rate. Overall, this work demonstrates the high potential for fast and accurate hands-free text entry for MR headsets.
Real-Virtual Objects: Exploring Bidirectional Embodied Tangible Interaction with a Virtual Human in World-Fixed Virtual Reality (Conference: P2080)
Lal "Lila" Bozgeyikli, University of Arizona
This paper explores bidirectional embodied tangible interaction between a human and a virtual human through shared objects that span the real-virtual boundary in world-fixed virtual reality. The shared objects extend from the real world into the virtual world (and vice versa). We discuss the novel interaction concept and implementation details and present the results of a between-subjects user study with 40 participants where we compared the developed novel real-virtual shared object interaction with a control version. The results showed that presence and co-presence were increased with the real-virtual object interaction, along with affective attraction to the virtual human and enjoyment of interaction.
Comparing Physics-based Hand Interaction in Virtual Reality: Custom Soft Body Simulation vs. Off-the-Shelf Integrated Solution (Conference: P2082)
Christos Lougiakis, National and Kapodistrian University of Athens; Jorge Juan González, PresenceXR; Giorgos Ganias, National and Kapodistrian University of Athens; Akrivi Katifori, Athena Research Center; Ioannis Panagiotis Ioannidis, Athena Research Center; Maria Roussou, National and Kapodistrian University of Athens
Physics-based hand interaction in VR has been explored extensively, but most solutions lack usability. CLAP stands out as a custom soft body simulation offering realistic hand interaction in VR. Despite CLAP's advantages, it imposes constraints. Introducing HPTK+, a software utilizing NVIDIA PhysX, aiming for free hand interactions in virtual environments. A user study with 27 participants compared both libraries, showing a preference for CLAP. However, no significant differences in other measures or performance, except variance, were observed. These findings offer insights into library suitability for specific tasks. HPTK+'s exclusive support for diverse interactions positions it for further research in physics-based VR interactions.
Session: 3D Interaction and Modality (TU3G)
Date & Time: Tuesday, 19 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Francisco Raul Ortega
PreVR: Variable-Distance Previews for Higher-Order Disocclusion in VR (Journal: P1012)
Shuqi Liao, Purdue University; Vetria L. Byrd, Purdue University; Voicu Popescu, Purdue University
The paper introduces PreVR, a method for allowing the user of a virtual reality application to preview a virtual environment around any number of corners. This way the user can gain line of sight to any part of the virtual environment, no matter how distant or how heavily occluded it is. PreVR relies on a multiperspective visualization that implements a high-order disocclusion effect with piecewise linear rays that bend multiple times as needed to reach the visualization target. PreVR was evaluated in a user study (N = 88) where it has brought significant virtual environment exploration efficiency gains compared to conventional virtual environment exploration.
Handwriting for Text Input and the Impact of XR Displays, Surface Alignments, and Sentence Complexities (Journal: P1242)
Florian Kern, University of Würzburg; Jonathan Tschanter, University of Würzburg; Marc Erich Latoschik, University of Würzburg
This article compares handwriting text input in VR and Video See-Through (VST) AR, facilitated by physically aligned and mid-air surfaces when writing simple and complex sentences. The results showed high usability with no significant impact of XR displays and surface alignments on text input performance. However, participants achieved higher input speeds and fewer errors for simple (17.85 WPM, 0.51% MSD ER) than complex sentences (15.07 WPM, 1.74% MSD ER). Physically aligned surfaces yielded higher learnability and lower physical demand. Mid-air surfaces and simple sentences led to enlarged and more connected cursive writing. We provide a phrase set and highlight benefits of XR controllers in pen-like postures with pressure-sensitive tips.
IntenSelect+: Enhancing Score-Based Selection in Virtual Reality (Journal: P1315)
Marcel Krüger, RWTH Aachen University; Tim Gerrits, RWTH Aachen University; Timon Römer, RWTH Aachen University; Torsten Wolfgang Kuhlen, RWTH Aachen University; Tim Weissker, RWTH Aachen University
Object selection in virtual environments is crucial, with the chosen technique significantly impacting system efficiency. IntenSelect, a scoring-based selection-by-volume method, outperforms other approaches, particularly for small spherical objects. However, it faces challenges in parameterization and flexibility. We introduce IntenSelect+, an enhanced version addressing these limitations. In a study with 42 users, comparing IntenSelect+, IntenSelect, and raycasting, we confirm the advantages of IntenSelect over raycasting and highlight significant improvements with IntenSelect+. Our findings indicate that IntenSelect+ is a promising enhancement for faster, more precise, and comfortable object selection in immersive virtual environments.
MusiKeys: Exploring Haptic-to-Auditory Sensory Substitution to Improve Mid-Air Text-Entry (Journal: P1421)
Alexander Krasner, Virginia Tech; Joseph L Gabbard, Virginia Tech
We investigated using auditory feedback in virtual reality mid-air typing to communicate the missing haptic feedback information typists normally receive when using a physical keyboard. We conducted a study with 24 participants, encompassing four mid-air virtual keyboards with increasing amounts of feedback information, along with a fifth physical keyboard as reference. Results suggest clicking feedback on key-press and key-release improves performance compared to no auditory feedback, consistent with literature. We find that audio can substitute information contained in haptic feedback, in that users can accurately perceive presented information. However, this understanding did not translate to significant differences in performance.
Investigating the Effects of Avatarization and Interaction Techniques on Near-field Mixed Reality Interactions with Physical Components (Journal: P1482)
Roshan Venkatakrishnan, Clemson University; Rohith Venkatakrishnan, Clemson University; Ryan Canales, Clemson University; Balagopal Raveendranath, Clemson University; Christopher Pagano, Clemson University; Andrew Robb, Clemson University; Wen-Chieh Lin, National Yang Ming Chiao Tung University; Sabarish V. Babu, Clemson University
Mixed reality experiences typically involve users interacting with a combination of virtual and physical components. In an attempt to understand how such interactions can be improved, we investigated how avatarization, the physicality of the interacting components, and interaction techniques affect the user experience. Results indicate that accuracy is more when the components are virtual rather than physical because of the increased salience of the task-relevant information. Furthermore, the relationship between avatarization and interaction techniques dictate how usable mixed reality interactions are deemed to be. This study provides key insights for optimizing mixed reality interactions towards immersive and effective user experiences.
Effect of Hand and Object Visibility in Navigational Tasks Based on Rotational and Translational Movements in Virtual Reality (Conference: P1245)
Amal Hatira, Kadir Has University; Zeynep Ecem Gelmez, Kadir Has University; Anil Ufuk Batmaz, Concordia University; Mine Sarac, Kadir Has University
In this paper, we investigate the effect of hand avatar and object visibility on navigational tasks using a VR headset. Participants navigated a cylindrical object through virtual obstacles using rotational or translational movements. The study used three visibility conditions for the hand avatar (opaque, transparent, and invisible) and two conditions for the object (opaque and transparent). Results showed that participants performed faster and with fewer collisions using invisible and transparent hands and opaque objects and preferred the combination of transparent hands and opaque objects. The findings could help researchers and developers determine the visibility/transparency conditions for precise navigational tasks.
Session: Perception in Navigation, Locomotion and Redirection (TU3H)
Date & Time: Tuesday, 19 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Anne Hélène Olivier
When vision lies - Navigating virtual environments with unreliable visual information (Conference: P1107)
Eden Or, Ben Gurion University; Shachar Maidenbaum, Ben Gurion University
What happens to human navigation when vision becomes actively unreliable? Will it impair user performance, be suppressed, or be used advantageously? While such scenarios are rare in the real world, this question has important implications for multisensory integration in extended reality applications - e.g. virtual walls that a user sees but can walk through. We tested this using mazes with reliable and unreliable sensory channels, which induced ghost walls (perceived but not boundaries) and invisible walls (no perception but boundaries). We found that unreliable vision led to better performance than without vision but that unreliable audition was suppressed.
Reaching Between Worlds: Calibration and Transfer of Perceived Affordances from Virtual to Real Environments (Conference: P1331)
Holly C Gagnon, University of Utah; Hunter C Finney, University of Utah; Jeanine Stefanucci, University of Utah; Bobby Bodenheimer, Vanderbilt University; Sarah Creem-Regehr, University of Utah
Accurate perception of action capabilities (affordance perception) is essential for interaction with real and virtual environments, but it is unknown what specific types of feedback are needed to improve affordance perception. This study examined whether perceived horizontal reachability in VR and AR improved with feedback, and if improvement transferred to the real world. Three types of feedback were studied: exploratory behavior, static outcome, and action outcome feedback. Our results indicate that exploratory behavior is sufficient for improved perceived reachability in VR, but in AR outcome feedback is necessary. In VR and AR outcome feedback was required for improvement in perceived reachability to transfer to the real world.
Examining Effects of Technique Awareness on the Detection of Remapped Hands in Virtual Reality (Journal: P1196)
Brett Benda, University of Florida; Benjamin Rheault, University of Florida; Yanna Lin, University of Florida; Eric Ragan, University of Florida
Input remapping techniques have been explored to allow users in virtual reality to exceed their own physical abilities, the limitations of physical space, or to facilitate interactions with real-world objects. Existing psychophysical methods can determine detection thresholds for these techniques, but they have known limitations. Our work evaluates a method for estimating detection that reduces these limitations and yields meaningful upper bounds. We apply this method to a well-explored hand motion scaling technique and demonstrate just how conservative these prior methods are. In unaware cases, users may detect their hand speed as abnormal at around 3.37 times the normal speed compared to a speed of 1.47 previous methods would suggest.
BiRD: Using Bidirectional Rotation Gain Differences to Redirect Users during Back-and-forth Head Turns in Walking (Journal: P2113)
Sen-Zhe Xu, Tsinghua University; Fiona Xiao Yu Chen, Tsinghua University; Ran Gong, Tsinghua University; Fang-Lue Zhang, Victoria University of Wellingtong; Song-Hai Zhang, Tsinghua University
Redirected walking (RDW) facilitates user navigation within expansive virtual spaces despite the constraints of limited physical spaces. It employs discrepancies between human visual-proprioceptive sensations, known as gains, to enable the remapping of virtual and physical environments. In this paper, we explore how to apply rotation gain while the user is walking. We propose to apply a rotation gain to let the user rotate by a different angle when reciprocating from a previous head rotation, to achieve the aim of steering the user to a desired direction. To apply the gains imperceptibly based on such a Bidirectional Rotation gain Difference (BiRD), we conduct both measurement and verification experiments on the detection thresholds of the rotation gain for reciprocating head rotations during walking. Unlike previous rotation gains which are measured when users are turning around in place (standing or sitting), BiRD is measured during users’ walking. Our study offers a critical assessment of the acceptable range of rotational mapping differences for different rotational orientations across the user's walking experience, contributing to an effective tool for redirecting users in virtual environments.
Evaluating Plausible Preference of Body-Centric Locomotion using Subjective Matching in Virtual Reality (Conference: P1882)
BoYu Gao, Jinan University; Haojun Zheng, Jinan University; Jingbo Zhao, China Agricultural University; Huawei Tu, La Trobe University; Hyungseok Kim, Konkuk University; Henry Been-Lirn Duh, Hong Kong Polytechnic University
Body-centric locomotion in Virtual Reality (VR) involves multiple factors, including the point of view, avatar representations, tracked body parts for locomotion control and transfer functions that map body movement to the displacement of the virtual viewpoint. This study employed the subjective matching method to evaluate the sense of plausible walking experience during body-centric locomotion in VR. A virtual locomotion experiment with these five factors based on subjective matching was conducted. The results could serve as the guidelines for virtual locomotion experience design that involves combinations of multiple factors and can help achieve a plausible walking experience in VR.
Analyzing Cognitive Demands and Detection Thresholds for Redirected Walking in Immersive Forest and Urban Environments (Conference: P1909)
Fariba Mostajeran, Universität Hamburg; Sebastian Schneider, Universität Hamburg; Gerd Bruder, University of Central Florida; Simone Kühn, Max Planck Institute for Human Development; Frank Steinicke, Universität Hamburg
The redirected walking technique allows users to explore larger spaces in immersive virtual environments (IVEs) via natural walking compared to their physical spaces. However, it may induce extra cognitive load for its users. On the other hand, it has been shown that the type of IVE can restore or diminish users' attentional capacities. In this paper, we examine the effects of two types of IVE (i.e., forest and urban) on redirected walking and its cognitive demands. The results of our frequentist and Bayesian analysis were consistent and suggest that redirected walking is robust to the variation of the tested IVEs.
Session: User Experience (TU3J)
Date & Time: Tuesday, 19 March 2024, 15:30-17:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Luciana Nedel
Who says you are so sick? An investigation on individual susceptibility to cybersickness triggers using EEG, EGG and ECG. (Journal: P1777)
Tian Nana, EPFL; Ronan Boulic, EPFL
In our study, we investigated the link between Electrocardiogram (ECG), Electrogastrogram (EGG), Electroencephalogram (EEG), and cybersickness susceptibility during rotational experiences. Combining objective measures and subjective assessments, we found EGG superior in detecting nausea symptoms, cautioning against sole reliance on ECG. Significant changes in EGG parameters were noted, but ECG alone may not reliably indicate cybersickness. Notably, EEG emerged as crucial for discerning individual differences in susceptibility to rotational triggers. Our findings extend to both periodic and aperiodic activities, highlighting EEG's potential in assessing cybersickness severity and individual susceptibility to cybersickness triggers.
A Study on Collaborative Visual Data Analysis in Augmented Reality with Asymmetric Display Types (Journal: P1875)
Judith Friedl-Knirsch, Technical University of Munich; Christian Stach, University of Applied Sciences Upper Austria; Fabian Pointecker, University of Applied Sciences Upper Austria; Christoph Anthes, University of Applied Sciences Upper Austria; Daniel Roth, Technical University of Munich
Collaboration is a key aspect of immersive visual data analysis, where augmented reality is useful for co-located scenarios. There are different types of technology available for augmented reality, which provide different premises for collaborative visual data analysis. In a mixed-methods user study, we combine handheld, optical see-through and video see-through displays to explore and understand the impact of these different device types on collaborative behaviour, user experience and usage patterns. We found that the different display types influenced how well participants could participate in the collaborative data analysis as well as differences in user experience and usage patterns.
DocuBits: VR Document Decomposition for Procedural Task Completion (Conference: P1484)
Geonsun Lee, University of Maryland; Jennifer Healey, Adobe Research; Dinesh Manocha, University of Maryland
DocuBits is a novel method for transforming monolithic instructional documents into small, interactive elements for VR. It enables users to create, position, and use these elements to monitor and share progress in collaborative VR learning environments. Two user studies, involving both individual and paired participants performing a chemistry lab task, demonstrated that DocuBits significantly improves usability and reduces perceived workload. In collaborative scenarios, it notably enhances social presence, collaborator awareness, and immersion. DocuBits offers valuable insights for integrating text-based instructions to support enhanced collaboration in VR settings.
Design and Validation of a Library of Active Affective Tasks for Emotion Elicitation in VR (Conference: P2072)
Jason Wolfgang Woodworth, University of Louisiana at Lafayette; Christoph W Borst, University of Louisiana at Lafayette
Emotion recognition models require datasets of physiological responses to stimuli designed to elicit targeted emotions. Many libraries of such stimuli have been created involving passive media such as images or videos. Virtual Reality, however, offers an opportunity to investigate uniquely active emotion elicitation stimuli that directly center the user in the experience. We leverage this to introduce a set of four active affective tasks in VR designed to quickly elicit targeted emotions without need for narrative understanding common to passive stimuli. We compare our tasks with selections from an existing library 360° videos and validate our approach by comparing self-reported emotional responses to the stimuli.
Dynamic Scene Adjustment Mechanism for Manipulating User Experience in VR (Conference: P2100)
Yi Li, Center for Future Media, the School of Computer Science and Engineering, University of Electronic Science and Technology of China; Zhitao Liu, UESTC; Li Yuan, Academy of Military Sciences; Haolan Tang, Center for Future Media, the School of Computer Science and Engineering, University of Electronic Science and Technology of China; YouTeng Fan, Center for Future Media, the School of Computer Science and Engineering, University of Electronic Science and Technology of China; Ning Xie, School of Computer Science and Engineering
Advancing VR tech creates realistic, controllable interactive environments, greatly impacting user experience. Concurrently, real-time user status monitoring advancements unlocked dynamic adjustments to VR environments through user real-time status and feedback. This paper introduces an interactive paradigm for VR environments called the Dynamic Scene Adjustment (DSA) mechanism, which can modify the VR environmental variables in real-time according to the user’s status and performance to enhance user engagement and experience. We adopted the perspective of visual environment variables, embedding the DSA mechanism into a music VR game with brain-computer interaction. Experimental results robustly support the rationality of the DSA approach.
Influence of Scenarios and Player Traits on Flow in Virtual Reality (Invited Journal: P3013)
Élise Lavoué, CNRS, Univ Lyon, LIRIS, France; Sophie Villenave, CNRS, Univ Lyon, LIRIS, France; Audrey Serna, CNRS, Univ Lyon, LIRIS, France; Clémentine Didier, CNRS, ENISE, LTDS, France; Patrick Baert, CNRS, Univ Lyon, LIRIS, France; Guillaume Lavoué, CNRS, Univ Lyon, LIRIS, France
Many studies have investigated how interpersonal differences between users influence their experience in Virtual Reality (VR) and it is now well recognized that user's subjective experiences and responses to the same VR environment can vary widely. In this study, we focus on player traits, which correspond to users' preferences for game mechanics, arguing that players react differently when experiencing VR scenarios. We developed three scenarios in the same VR environment that rely on different game mechanics, and evaluate the influence of the scenarios, the player traits and the time of practice of the VR environment on users' perceived flow. Our results show that 1) the type of scenario has an impact on specific dimensions of flow; 2) the scenarios have different effects on flow depending on the order they are performed, the flow preconditions being stronger when performed at last; 3) almost all dimensions of flow are influenced by the player traits, these influences depending on the scenario, 4) the Aesthetic trait has the most influences in the three scenarios. We finally discuss the findings and limitations of the present study that we believe has strong implications for the design of scenarios in VR experiences.
Session: 360 Video (WE1G)
Date & Time: Wednesday, 20 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Dirk Reiners
Avatar360: Emulating 6-DoF Perception in 360° Images through Avatar-Assisted Navigation (Conference: P1294)
Andrew Chalmers, Victoria University of Wellington; Faisal Zaman, Victoria University of Wellington; Taehyun James Rhee, Victoria University of Wellington
360° images provide a panoramic view into captured environments, placing users within an egocentric perspective. However, users are not able to experience 6-DoF navigation with translational movement. We propose Avatar360, a novel method to elicit 6-DoF perception in 360° panoramas. We seamlessly integrate a 3D avatar into the 360° panoramas, allowing users to navigate a 3D virtual landscape congruent with the 360° background. By aligning the exocentric perspective of the 360° panorama with the avatar's movements, we successfully replicate a sensation of 6-DoF navigation within 360° panoramas. A user study was conducted, showing that avatar-assisted navigation can convincingly elicit a user's sensation of movement within 360° panoramas.
Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360$^\circ$ Image Outpainting (Journal: P1228)
Hao Ai, Hong Kong University of Science and Technology (Guangzhou Campus); Zidong Cao, HKUST (GZ); Haonan Lu, OPPO; Chen Chen, OPPO research institute; jian ma, OPPO; Pengyuan Zhou, University of Science and Technology of China; Tae-Kyun Kim, Korea Advanced Institute of Science and Technology (KAIST); Pan Hui, The Hong Kong University of Science and Technology; Lin Wang, HKUST, GZ
360 images provide immersive and realistic environments for emerging virtual reality (VR) applications, such as virtual tourism, where users desire to create diverse panoramic scenes from a narrow FoV photo. In this paper, we propose a transformer-based 360 image outpainting framework called Dream360, which can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports, considering the spherical properties of 360 images. Dream360 comprises two key learning stages: (I) codebook-based panorama outpainting via Spherical-VQGAN, and (II) frequency-aware refinement with a novel frequency-aware consistency loss. We conducted a user study within 15 participants to interactively evaluate the generated results in VR.
NHVC: Neural Holographic Video Compression with Scalable Architecture (Conference: P1282)
Hyunmin Ban, Kyung Hee University; Seungmi Choi, Kyung Hee University; Jun Yeong Cha, Kyung Hee University; Yeongwoong Kim, Kyung Hee University; Hui Yong Kim, Kyung Hee Univ.
This paper presents a scalable, end-to-end trainable model for efficient phase hologram video generation and compression. The proposed model, Neural Holographic Video Compression (NHVC), combines an auto-encoder-based phase hologram generator with a latent coder and dual hyper-prior coders, achieving task scalability across hologram generation and compression. It significantly outperforms existing methods, evidenced by a 75.6% BD-Rate reduction over 'HoloNet + VVC' and delivering high-quality reconstruction. NHVC addresses key design questions in neural compression, offering solutions for motion estimation in the phase domain and variable rate support.
Jump Cut Effects in Cinematic Virtual Reality: Editing with the 30-degree Rule and 180-degree Rule (Conference: P1432)
Junjie Zhang, The Hong Kong University of Science and Technology; Lik-Hang Lee, Hong Kong Polytechnic University; Yuyang Wang, Hong Kong University of Science and Technology; Shan Jin, The Hong Kong University of Science and Technology (Guangzhou); Dan-Lu Fei, Hong Kong University of Science and Technology; Pan Hui, The Hong Kong University of Science and Technology
This study pioneers the examination of jump cuts' influence on presence, viewing experience, and edit quality in cinematic virtual reality (VR). Employing the 30-degree and 180-degree rules, our user study with thirteen participants reveals substantial improvements in presence, viewing experience, and edit quality. These findings offer crucial insights for VR content creators and editors seeking to elevate the effectiveness and immersiveness of VR experiences. In response to the escalating demand for top-tier VR content, this research provides valuable guidance for those navigating the evolving landscape of VR editing techniques.
Fumos: Neural Compression and Progressive Refinement for Continuous Point Cloud Video Streaming (Journal: P1559)
Zhicheng LIANG, The Chinese University of Hong Kong (Shenzhen); Junhua Liu, The chinese university of Hong Kong,Shenzhen; Mallesham Dasari, Carnegie Mellon University; Fangxin Wang, The Chinese University of Hong Kong, Shenzhen
Point cloud video (PCV) enhances VR/AR experiences with 6-DoF, offering photorealistic 3D scenes. However, its streaming demands high bandwidth, often exceeding commodity devices' capacities. To address the bandwidth challenge and unpredictable user FoV, we introduce, a novel system, Fumos, optimizing bandwidth and enhancing the user's quality of experience (QoE). Fumos features a neural compression framework (N-PCC) for efficient, high-fidelity data transmission, progressive refinement streaming for continuous high-quality playback, and system-level adaptation optimizing long-term user QoE. Our results show Fumos's superior performance, significantly outpacing current solutions in decoding speed and compression efficiency.
Session: Immersive Analytics and Visualization (WE1H)
Date & Time: Wednesday, 20 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Wolfgang Stuerzlinger
Reimagining TaxiVis through an Immersive Space-Time Cube metaphor and reflecting on potential benefits of Immersive Analytics for urban data exploration (Conference: P1133)
Jorge Wagner, Federal University of Rio Grande do Sul; Claudio Silva, New York University; Wolfgang Stuerzlinger, Simon Fraser University; Luciana Nedel, Federal University of Rio Grande do Sul (UFRGS)
To explore how a traditional visualization system could be adapted into an immersive framework, and how it could benefit from this, we decided to revisit a landmark paper presented ten years ago at IEEE VIS. TaxiVis, by Ferreira et al., enabled interactive spatio-temporal querying of a large dataset of taxi trips in New York City. Here, we reimagine how TaxiVis’ functionalities could be implemented and extended in a 3D immersive environment. Through reporting on our experience, and on the vision and reasoning behind our design decisions, we hope to contribute to the debate on how conventional and immersive visualization paradigms can complement each other and also how the exploration of urban datasets can be facilitated in the coming years.
VollyNaut: Pioneering Immersive Training for Inclusive Sitting Volleyball Skill Development (Conference: P1312)
Ut Gong, Zhejiang University; Hanze Jia, Zhejiang University; Yujie Wang, Zhejiang University; Tan Tang, Zhejiang University; Xiao Xie, Zhejiang University; Yingcai Wu, Zhejiang University
Participating in sports provides disabled individuals with social inclusion, health benefits, and confidence. Yet, popular para-sports such as sitting volleyball face limited access due to scarce dedicated courts. Addressing this, we conducted a pioneering VR para-sports training study and developed VolleyNaut, a VR system created in collaboration with professional coaches to realistically simulate daily training drills and ball pitches. Our user study included college volleyball players, as well as national sitting volleyball athletes and coaches, to assess VolleyNaut's engagement and training effectiveness. The results, incorporating both quantitative and qualitative data, showed consistently positive feedback across all groups.
Enhancing Tai Chi Training System: Towards Group-Based and Hyper-Realistic Training Experiences (Journal: P1474)
Feng Tian, Shanghai University; Shuting Ni, Shanghai University; Xiaoyue Zhang, Shanghai Film Academy,Shanghai University; Fei Chen, Shanghai University; Qiaolian Zhu, shanghai university; Chunyi Xu, Shanghai University; Yuzhi Li, Shanghai University
In this article, we propose a lightweight enhanced Tai Chi training system composed of multiple standalone virtual reality (VR) devices. The system aims to rapidly enhance movement precision and communication interest for learners. We objectively evaluate participants' action quality at different levels of immersion, including traditional coach guidance (TCG), VR, and mixed reality (MR), along with subjective measures like motion sickness, quality of interaction, social meaning, presence/immersion to comprehensively explore the system's feasibility. The results indicate VR performs the best in training accuracy, but MR provides superior social experience and relatively high accuracy.
Analyzing user behaviour patterns in a cross-virtuality immersive analytics system (Journal: P1631)
Mohammad Rajabi Seraji, Simon Fraser University; Parastoo Piray, Simon Fraser University; Vahid Zahednejad, Simon Fraser University; Wolfgang Stuerzlinger, Simon Fraser University
Motivated by the recently discovered benefits of Cross-virtuality Immersive Analytics (XVA) systems, we developed HybridAxes, which allows users to transition seamlessly between the desktop and a virtual environment. Our user study shows that users prefer AR for exploratory tasks and the desktop for detailed tasks, indicating that these modes of an XVA system complement each other in enhancing the data analysis experience. Despite minor challenges in mode-switching, the system was well-received for its user-friendliness and engagement. Our research offers design insights, valuable directions for future cross-virtuality visual analytics systems, and identifies potential areas for further study.
Whatever could be, could be: Visualizing Future Movement Predictions (Conference: P1843)
Chenkai Zhang, University of South Australia; Ruochen Cao, Taiyuan University of Technology; Andrew Cunningham, University of South Australia; James A. Walsh, University of South Australia
As technology grants us superhuman powers, looking into what the future may hold is no longer science fiction. We present our work evaluating visualizations of future predictions in the Football domain. We explore the problem space, examining what a future may be. Three visualizations (2 arrow lines, 5 arrow lines, and a heatmap) are introduced as representations. Whilst football is used as an example domain in this work, the findings aim to generalize to other scenarios that contain trajectory information. Two VR studies (2xn=24) examined the visualizations in various situations. Results show heatmap as the most effective and preferred by the vast majority of participants. Findings offer insights into future visualization.
Session: Industrial and Sports Applications (WE1J)
Date & Time: Wednesday, 20 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Ooi Chun Wei
Prototyping of Augmented Reality interfaces for air traffic alert and their evaluation using a Virtual Reality air-proximity simulator (Conference: P1621)
Angel Torres del Alamo, University of Castilla-La Mancha; Jose Pascual Molina Masso, University of Castilla-La Mancha; Arturo S. Garcia, University of Castilla-La Mancha; Pascual Gonzalez, University of Castilla-La Mancha
Mid-air collisions can occur, especially near airfields and in non-controlled airspaces. Technology typically helps pilots in the form of a flat display in the cockpit. AR could reach light and sport aviation in the next years thanks to new light and affordable AR glasses. In this work we rely on VR to prototype and test different AR interfaces for air traffic alert. Firstly, we proposed and tested four different HMD AR interfaces with 4 pilots in our own VR aircraft-proximity simulator. Then, the two best-scored interfaces were selected for a second evaluation, compared against Circular HUD (Alce et al.) and a fixed-mounted (FM) conventional HUD radar, tested by 4 additional pilots. Overall, pilots showed preference for our AR proposals.
Research Trends in Virtual Reality Music Concert Technology: A Systematic Literature Review (Journal: P1129)
Jieun Park, Korea Advanced Institute of Science and Technology; Youjin Choi, Korea Advanced Institute of Science and Technology; Kyung Myun Lee, Korea Advanced Institute of Science and Technology
Following COVID-19, there has been a significant interest in virtual reality (VR) music concerts as an innovative alternative to conventional live events. VR music concert research is rapidly expanding the diversity of research, requiring a unified understanding of the field. This systematic literature review, spanning 2018 to 2023, identifies trends in VR music concert technology through a PRISMA-based analysis of 27 selected papers. The studies were analyzed based on the research topic, interaction type, hardware used, and evaluation metrics. The review contributes to advancing the understanding of recent developments in VR music concert technology, shedding light on the diversification and potential of this emerging field.
Automatic Indoor Lighting Generation Driven by Human Activity Learned from Virtual Experience (Conference: P1148)
Jingjing Liu, Zhejiang University; Jianwen Lou, Zhejiang University; Youyi Zheng, Zhejiang University; Kun Zhou, Zhejiang University
A good indoor lighting solution should fit with people's habitual activity and have a low energy cost. However, it's challenging to capture and model human activity in reality due to its high complexity, let alone incorporating it into lighting planning. To solve this problem, we propose a novel framework for automatic indoor lighting generation driven by human activity learned from virtual experience. We first harnesses VR to simulate and model the user's daily activities within an indoor scene, and then devises a robust objective function which encompasses multiple activity-driven cost terms for lighting layout optimization. At last, an optimization algorithm is applied to search for the optimal lighting solution.
ViewR: Architectural-Scale Multi-User Mixed Reality with Mobile Head-Mounted Displays (Invited Journal: P3010)
Florian Schier, Computer Science, Dresden University of Technology, Dresden, Germany; Daniel Zeidler, Computer Science, Dresden niversity of Technology, Dresden, Germany; Krishnan Chandran, Computer Science, Dresden University of Technology, Dresden, Germany; Zhongyuan Yu, Computer Science, Dresden University of Technology, Dresden, Germany; Matthew McGinity, Computer Science, Dresden University of Technology, Dresden, Germany
The emergence of mobile head-mounted displays with robust ”inside-out” markerless tracking and video-passthrough permits the creation of novel mixed reality (MR) experiences in which architectural spaces of arbitrary size can be transformed into immersive multi-user visualisation arenas. Here we outline ViewR, an open-source framework for rapidly constructing and deploying architectural-scale multi-user MR experiences. ViewR includes tools for rapid alignment of real and virtual worlds, tracking loss detection and recovery, user trajectory visualisation and world state synchronisation between users with persistence across sessions. ViewR also provides control over the blending of the real and the virtual, specification of site-specific blending zones, and video-passthrough avatars, allowing users to see and interact with one another directly. Using ViewR, we explore the transformation of large architectural structures into immersive arenas by creating a range of experiences in various locations, with a particular focus on architectural affordances such as mezzanines, stairs, gangways and elevators. Our tests reveal that ViewR allows for experiences that would not be possible with pure virtual reality, and indicate that, with certain strategies for recovering from tracking errors, it is possible to construct large scale multi-user MR experiences using contemporary consumer virtual reality head-mounted displays.
Augmented Coach: Volumetric Motion Annotation and Visualization for Immersive Sports Coaching (Conference: P1828)
Jiqing Wen, Arizona State University; Lauren Gold, Arizona State University; Qianyu Ma, Arizona State University; Robert LiKamWa, Arizona State University
Traditional methods of remote sports coaching is challenged by the absence of spatial analysis capabilities, which hinders in-depth assessment of athletic performance. This paper introduces Augmented Coach, an immersive and interactive sports coaching system. Augmented Coach utilizes volumetric data to reconstruct the 3D representations of the athletes. As a result, coaches can not only view the resulting point cloud videos of the athletes performing athletic movements, but also employ the system's spatial annotation and visualization tools to gain insights into movement patterns and communicate with remote athletes.
Session: 3D Authoring (WE2G)
Date & Time: Wednesday, 20 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Yalong Yang
Instant Segmentation and Fitting of Excavations in Subsurface Utility Engineering (Journal: P1561)
Marco Stranner, Institute for Computer Graphics and Vision; Philipp Fleck, Graz University of Technology; Dieter Schmalstieg, Graz University of Technology; Clemens Arth, Graz University of Technology
AR for subsurface utility engineering (SUE) has benefited from recent advances in sensing hardware. In this work, we present a novel approach to automate the process of aligning existing SUE databases with measurements taken during excavation works, with the potential to correct the deviation from the as-planned to as-built documentation. Our segmentation algorithm performs infrastructure segmentation based on the live capture of an excavation on site. Our fitting approach correlates the inferred position and orientation with the existing digital plan and registers the as-planned model into the as-built state. We show the results of our proposed method on both synthetic data and a set of real excavations.
ConnectVR: A Trigger-Action Interface for Creating Agent-based Interactive VR Stories (Conference: P1811)
Mengyu Chen, University of California Santa Barbara; Marko Peljhan, University of California Santa Barbara; Misha Sra, UCSB
We present ConnectVR, a trigger-action interface to enable non-technical creators design agent-based narrative experiences. Our no-code authoring method specifically focuses on the design of narratives driven by a series of cause-effect relationships triggered by the player's actions. We asked 15 participants to use ConnectVR in a workshop study as well as two artists to extensively use our system to create VR narrative projects in a three-week in-depth study. Our findings shed light on the creative opportunities facilitated by ConnectVR's trigger-action approach, particularly its capability to establish chained behavioral effects between virtual agents.
HoloCamera: Advanced Volumetric Capture for Cinematic-Quality VR Applications (Journal: P2038)
Jonathan Heagerty, University of Maryland, College Park; Sida Li, University of Maryland; Eric Lee, University of Maryland; Shuvra Bhattacharyya, University of Maryland; Sujal Bista, University of Maryland; Barbara Brawn, University of Maryland; Brandon Y. Feng, University of Maryland; Susmija Jabbireddy, University of Maryland College Park; Joseph F. JaJa, University of Maryland; Hernisa Kacorri, University of Maryland; David Li, University Of Maryland; Derek T Yarnell, University of Maryland; Matthias Zwicker, University of Maryland, College Park; Amitabh Varshney, University of Maryland
HoloCamera is an innovative volumetric capture instrument comprising 300 high-resolution RGB cameras in a custom free-standing structure; it rapidly acquires, processes, and creates cinematic-quality virtual avatars and scenarios. The system employs 50 Jetson AGX Xavier boards to perform distributed computing, with each processing unit dedicated to driving six cameras; a Gigabit Ethernet network fabric seamlessly interconnects all compute boards, enabling precise camera synchronization and fast data transfer. The paper details construction design and approaches, techniques employed to achieve precise frame synchronization and calibrations of the cameras. We are releasing 30 high-fidelity light-field datasets to promote future research.
Toward More Comprehensive Evaluations of 3D Immersive Sketching, Drawing, and Painting (Invited Journal: P3002)
Mayra Donaji Barrera Machuca, Dalhousie University, Halifax, Canada; Johann Habakuk Israel, HTW-Berlin, Berlin, Germany; Daniel F. Keefe, University of Minnesota, Minneapolis, USA; Wolfgang Stuerzlinger, Simon Fraser University, Vancouver, Canada
To understand current practice and explore the potential for more comprehensive evaluations of 3D immersive sketching, drawing, and painting, we present a survey of evaluation methodologies used in existing 3D sketching research, a breakdown and discussion of important phases (sub-tasks) in the 3D sketching process, and a framework that suggests how these factors can inform evaluation strategies in future 3D sketching research. Existing evaluations identified in the survey are organized and discussed within three high-level categories: 1) evaluating the 3D sketching activity, 2) evaluating 3D sketching tools, and 3) evaluating 3D sketching artifacts. The new framework suggests targeting evaluations to one or more of these categories and identifying relevant user populations. In addition, building upon the discussion of the different phases of the 3D sketching process, the framework suggests to evaluate relevant sketching tasks, which may range from low-level perception and hand movements to high-level conceptual design. Finally, we discuss limitations and challenges that arise when evaluating 3D sketching, including a lack of standardization of evaluation methods and multiple, potentially conflicting, ways to evaluate the same task and user interface usability; we also identify opportunities for more holistic evaluations. We hope the results can contribute to accelerating research in this domain and, ultimately, broad adoption of immersive sketching systems.
Session: Gaze (WE2H)
Date & Time: Wednesday, 20 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Alexander Plopski
EyeShadows: Peripheral Virtual Copies for Rapid Gaze Selection and Interaction (Conference: P1153)
Jason Orlosky, Augusta University; Chang Liu, Kyoto University Hospital; Kenya Sakamoto, Osaka University; Ludwig Sidenmark, University of Toronto; Adam Mansour, Augusta University
In this paper, we present EyeShadows, an eye gaze-based selection system that takes advantage of peripheral copies of items that allow for quick selection and manipulation of an object or corresponding menus. This method is compatible with a variety of different selection tasks and controllable items, avoids the Midas touch problem, does not clutter the virtual environment, and is context sensitive. We have implemented and refined this selection tool for VR and AR, including testing with optical and video see-through displays. We demonstrate that EyeShadows can also be used for a wide range of AR and VR applications, including manipulation of sliders or analog elements.
Selection Performance and Reliability of Eye and Head Gaze Tracking Under Varying Light Conditions (Conference: P1189)
Alexander Marquardt, Institute of Visual Computing; Melissa Steininger, Institute of Visual Computing; Christina Trepkowski, Institute of Visual Computing; Martin Weier, RheinMain University of Applied Sciences; Ernst Kruijff, Bonn-Rhein-Sieg University of Applied Sciences
In this paper, we present two studies exploring the influence of environmental factors, particularly lighting conditions and spatial properties, on eye and head gaze tracking in Augmented Reality (AR). Our findings show that eye tracking tends to be faster but faces inconsistencies, particularly under dynamic lighting conditions. In contrast, head gaze tracking, though more consistent, experiences reduced accuracy in environments with fluctuating light levels. Additionally, the spatial properties of the environment significantly influence both tracking methods. Our findings emphasize the importance of these factors in choosing a tracking method and the need for AR systems to adapt dynamically to environmental changes.
Eye-hand Typing: Eye Gaze Assisted Finger Typing via Bayesian Processes in AR (Journal: P1726)
Yunlei Ren, University of Electronic Science and Technology of China; Yan Zhang, University of Electronic Science and Technology of China; Zhitao Liu, UESTC; Ning Xie, School of Computer Science and Engineering
In a factory environment, fast and accurate text input is crucial for operators' efficiency and task completion quality. However, the traditional AR keyboard may not meet this requirement, and the noisy environment is unsuitable for voice input. In this article, we introduce Eye-Hand Typing, an intelligent AR keyboard. We leverage the speed advantage of eye gaze and use a Bayesian process based on the information of gaze points to infer users' text input intentions. We improve the underlying keyboard algorithm without changing user input habits, thereby improving factory users' text input speed and accuracy. In summary, Eye-Hand Typing using gaze assist technology is an effective AR text input solution with great prospects in AR HMDs.
GazePuffer :Hands-Free Input Method Leveraging Puff Cheeks for VR (Conference: P1392)
Yunfei Lai, JiLin University; Minghui Sun, Jilin University; Zhuofeng Li, Jilin University
This paper introduces GazePuffer, a method that addresses the lack of a confirmation mechanism in gaze input by integrating cheek puffing with gaze. We explore the design space of cheek puffing gestures, presenting a set of nine gestures and their corresponding operations in virtual reality. We achieved a 93.8% accuracy rate in recognizing five cheek puffing gestures. In an experiment based on Fitts' Law, we compared GazePuffer's performance with two methods, demonstrating its performance is comparable to Gaze&Pinch and slightly superior to Gaze&Dwell in terms of efficiency. Finally, we showcase the practical applicability of GazePuffer in real-world VR interaction tasks.
Session: Collaboration (WE2J)
Date & Time: Wednesday, 20 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Kangsoo Kim
Googly Eyes: Exploring Effects of Displaying User’s Eye Movements Outward on a VR Head-Mounted Display on User Experience (Conference: P2097)
Evren Bozgeyikli, University of Arizona; Lal "Lila" Bozgeyikli, University of Arizona; Victor Gomes, University of Arizona
Head-mounted displays (HMDs) in virtual reality (VR) occlude the upper face of the wearing users, leading to decreased nonverbal communication cues toward outside users. In this paper, we discuss Googly Eyes, a high-fidelity prototype that displays an illustration of the HMD-wearing user’s eyes in real-time in front of an HMD. We designed and developed a collaborative asymmetrical co-located task performed by an HMD-wearing user and a non-HMD (tablet) user, and we conducted a between-subjects user study where we compared the Googly Eyes with a baseline HMD experience without any external eye depiction. In this paper, we discuss the system, task, user study details, and results along with implications for future studies.
Breaking the Isolation: Exploring the Impact of Passthrough in Shared Spaces on Player Performance and Experience in VR Exergames (Journal: P1232)
Zixuan Guo, Xi'an Jiaotong-Liverpool University; Hongyu Wang, Xi'an Jiaotong-Liverpool University; Hanxiao Deng, Xi'an Jiaotong-Liverpool University; Wenge Xu, Birmingham City University; Nilufar Baghaei, University of Queensland; Cheng-Hung Lo, Xi'an Jiaotong-Liverpool University; Hai-Ning Liang, Xi'an Jiaotong-Liverpool University
VR exergames boost physical activity but face challenges in shared spaces due to the presence of bystanders. Passthrough in VR enhances players' environmental awareness, offering a promising solution. This work explores its impact on player performance and experience in the following conditions: Space (Office vs. Corridor) and Passthrough Function (With vs. Without). Results show Passthrough improves performance and awareness while reducing immersion, especially benefiting higher self-consciousness players. Players typically favor open spaces that consider social acceptability issues, and Passthrough addresses concerns in both shared space types. Our findings offer insights for designing VR experiences in shared environments.
Exploring the Influence of Virtual Avatar Heads in Mixed Reality on Social Presence, Performance and User Experience in Collaborative Task (Journal: P1913)
Théo COMBE, Nantes Université, École Centrale Nantes, LS2N-PACCE, UMR 6004; Rebecca Fribourg, Nantes Université, École Centrale Nantes, LS2N-PACCE, UMR 6004; Lucas Detto, Nantes Université, ENSA Nantes, École Centrale Nantes, CNRS, AAU-CRENAU, UMR 1563; Jean-Marie Normand, Nantes Université, École Centrale Nantes, LS2N-PACCE, UMR 6004
In this paper, we investigate how displaying virtual avatars' heads on-top of the heads of participants in a VST Mixed Reality local collaborative task could improve their collaboration as well as social presence. To do so, we conducted a between-subject study (n=88) with two factors: the type of avatar (cartoon, realistic or no Avatar) and the level of facial expressions provided (high or low). The experiment involved two dyadic communication tasks involving more or less collaboration. Our results indicate that while adding an avatar's head does not necessarily improve social presence, the amount of facial expressions provided through it has an impact, and its realism influences self-rated performance and uncanny valley ratings.
Workspace Guardian: Investigating Awareness of Personal Workspace Between Co-Located Augmented Reality Users (Journal: P2052)
Bret Jackson, Macalester College; Linda Lor, Macalester College; Brianna C Heggeseth, Macalester College
As augmented reality (AR) systems proliferate and the technology gets smaller and less intrusive, we imagine a future where many AR users will interact in the same physical locations. While previous research has explored AR collaboration in these spaces, our focus is on co-located but independent work. In this paper, we explore co-located AR user behavior and investigate techniques for promoting awareness of personal workspace boundaries. Specifically, we compare three techniques: showing all virtual content, visualizing bounding box outlines of content, and a self-defined workspace boundary. The findings suggest that a self-defined boundary led to significantly more personal workspace encroachments.
Session: Haptics (WE3G)
Date & Time: Wednesday, 20 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Maud Marchal
Presentation of Finger-size Shapes by Combining Force Feedback and Electro-tactile Stimulation (Conference: P1264)
Yui Suga, The University of Electro-Communications; Izumi Mizoguchi, The University of Electro-Communications; Hiroyuki Kajimoto, The University of Electro-Communications
Perceiving shapes quickly and accurately is vital for effective interaction and immersion in VR. Force feedback devices often fail to convey precise geometric details like edges. We enhanced these devices with a compact, high-density cutaneous electrical stimulation mechanism to better represent subtle shapes. Our evaluation involved shape discrimination tests on four 7.5 mm thick column types, using reactive force and edge cues. We compared conditions using only force feedback, only cutaneous feedback, and a combination of both. The results showed that the combined approach significantly improved shape discrimination accuracy, leading to a more precise perception of fine geometric details.
Hap'n'Roll: A Scroll-inspired Device for Delivering Diverse Haptic Feedback with a Single Actuator (Conference: P1265)
Hiroki Ota, Nara Institute of Science and Technology; Daiki Hagimori, Nara Institute of Science and Technology; Monica Perusquia-Hernandez, Nara Institute of Science and Technolgy; Naoya Isoyama, Otsuma Women's University; Yutaro Hirao, Nara Institute of Science and Technology; Hideaki Uchiyama, Nara Institute of Science and Technology; Kiyoshi Kiyokawa, Nara Institute of Science and Technology
Hap'n'Roll is an innovative tactile device with a scroll-like mechanism and a single motor that manipulates a sheet to simulate various shapes and textures. This unique approach enables the presentation of multiple forms beyond cylindrical shapes and the conveyance of different textures to the user's fingertips. User studies confirm Hap'n'Roll's effectiveness, with high shape recognition accuracy, showcasing its potential in diverse haptic feedback applications.
ThermalGrasp: Enabling Thermal Feedback even while Grasping and Walking (Conference: P1440)
Alex Mazursky, University of Chicago; Jas Brooks, University of Chicago; Beza Desta, University of Chicago; Pedro Lopes, University of Chicago
We present ThermalGrasp, a solution for wearable thermal interfaces. Unlike conventional setups that hinder users from grasping or walking by attaching Peltier elements and bulky cooling systems directly to the palm or sole, ThermalGrasp relocates these components to non-essential areas, such as the dorsal hand or foot. Our approach facilitates heat conduction to/from the palm or sole via thin, compliant materials, allowing users to interact with real objects while experiencing thermal feedback. This strikes a balance between thermal and haptic realism, enabling users to, for instance, feel the heat of a torch-like prop in virtual reality without sacrificing the ability to grasp.
Immersive 3D Medical Visualization in Virtual Reality using Stereoscopic Volumetric Path Tracing (Conference: P1667)
Javier Taibo, Universidade da Coruña; Jose A. Iglesias-Guitian, Universidade da Coruña
Scientific visualization using Monte Carlo (MC) volumetric path tracing and physically-based lighting is applied in medical contexts under the term cinematic rendering (CR), and it benefits healthcare professionals with photorealistic 3D reconstructions. However, given its high computational cost, CR deployment in VR finds significant issues, such as limited interactivity and temporal flicker caused by noisy MC renderings. We present an immersive 3D medical visualization system capable of generating photorealistic and fully interactive stereoscopic renderings on head-mounted display (HMD) devices. Our method extends previous linear regression denoising to enable real-time stereoscopic volumetric path tracing within VR environments.
Session: Healthcare Applications (WE3H)
Date & Time: Wednesday, 20 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Regis Kopper
Virtual Reality Self Co-embodiment: An Alternative to Mirror Therapy for Post-Stroke Upper Limb Rehabilitation (Journal: P1297)
Rodrigo Cerecero Curiel, The University of Tokyo; Takuto Nakamura, The University of Tokyo; Hideaki Kuzuoka, The University of Tokyo; Takafumi Kanaya, The University of Tokyo; Cosima Prahm, University of Tuebingen; Keigo Matsumoto, The University of Tokyo
We introduce Virtual Reality Self Co-embodiment (VRSelfCo), a post-stroke upper limb rehabilitation method inspired by Mirror Therapy. By using motion tracking from a VR headset, we enable the paretic arm to control a digital avatar with any residual motion available. Then, we utilize the other arm's movement to complete any missing motion in the affected digital limb through self-supported virtual co-embodiment. By interacting with VRSelfCo, users actively engage the impaired area while perfroming rehabilitation chores. Our study compares task performance and embodiment between Mirror Therapy and our approach within a group of healthy individuals, revealing VRSelfCo is a feasible technique for testing with actual post-stroke patients.
Empowering Persons with Autism through Cross-Reality and Conversational Agents (Journal: P1180)
Franca Garzotto, Politecnico di Milano; Mattia Gianotti, Politecnico di Milano; Alberto Patti, Politecnico di Milano; Francesca Pentimalli, Politecnico di Milano; Francesco Vona, Politecnico di Milano
Autism Spectrum Disorder is a neurodevelopmental condition that can affect autonomy and independence. Our research explores the integration of Cross-Reality and Conversational Agents for Autistic persons to improve ability and confidence in everyday life situations. We combine two technologies of the Virtual-Real continuum. User experiences unfold from the simulation of tasks in VR to the execution of similar tasks supported by AR in real world. A speech-based Conversational Agent is integrated with both VR and AR. It provides contextualized help, promotes generalization, and stimulates users to apply what they learned in the virtual space. The paper presents the approach and describes an empirical study involving 17 young Autistic persons.
Understanding the Impact of Longitudinal VR Training on Users with Mild Cognitive Impairment Using fNIRS and Behavioral Data (Conference: P1223)
Jing Qu, Shandong University; Shantong Zhu, Shandong University; Yiran Shen, Shandong University; Yanjie Zhang, The Hong Kong Polytechnic University; Lingguo Bu, Shandong University
This study presents a VR-based cognitive training system for mild cognitive impairment (MCI) rehabilitation. It leverages multi-modal data to assess longitudinal training impact using brain activation, network connectivity, behavior indicators, and MoCA scores. A two-month experiment validates the feedback methodologies and explores training duration effects on rehabilitation efficacy. Results indicate significant positive outcomes for MCI rehabilitation and inform VR system refinement. The research highlights the importance of quantitative, longitudinal assessments in rehabilitation efficacy studies.
Reflecting on Excellence: VR Simulation for Learning Indirect Vision in Complex Bi-Manual Tasks (Conference: P1359)
Maximilian Kaluschke, University of Bremen; Rene Weller, University of Bremen; Myat Su Yin, Faculty of Information and Communication Techology; Benedikt Werner Hosp, University Clinics Tübingen; Farin Kulapichitr, University of Bremen; Siriwan Suebnukarn, Thammasat University; Peter Haddawy, Mahidol University; Gabriel Zachmann, University of Bremen
This study explores the efficacy of VR simulators in training dental surgery skills involving indirect vision through a mirror. To assess learning outcomes, a simulator tracked eye gaze and tool trajectories during the root canal access opening stage. Thirty fifth-year dental students underwent six training sessions, showing significant improvements in drilling on realistic plastic teeth. Students with better simulator performance demonstrated enhanced real-world test results. Eye tracking revealed correlations between correct mirror placement, continuous fixation on the tooth, and improved skills. The findings suggest that eye tracking offers valuable insights into bi-manual psychomotor skill development in indirect vision scenarios.
Session: Human Factors and Ergonomics (WE3J)
Date & Time: Wednesday, 20 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Isaac Cho
Hold Tight: Identifying Behavioral Patterns During Prolonged Work in VR through Video Analysis (Journal: P1354)
Verena Biener, Coburg University of applied sciences and arts; Forouzan Farzinnejad, Coburg University of applied sciences and arts; Rinaldo Schuster, Coburg University of applied sciences and arts; Seyedmasih Tabaei, Coburg University of applied sciences and arts; Leon Lindlein, Coburg University of applied sciences and arts; Jinghui Hu, University of Cambridge; Negar Nouri, Coburg University of applied sciences and arts; John J Dudley, University of Cambridge; Per Ola Kristensson, University of Cambridge; Jörg Müller, University of Bayreuth; Jens Grubert, Coburg University of Applied Sciences and Arts
Recent VR devices have been promoted as tools for knowledge work and research suggests VR can efficiently support certain knowledge worker tasks. As only a few studies have explored the effects of prolonged use of VR, we report on the results from an analysis of 559 hours of video material obtained in a prior study in which 16 participants worked in VR and a physical environment for five days each. We observed adaptation effects such as participants adjusting the HMD less at the last day, but also that wearing an HMD is disruptive to normal patterns of eating and drinking. Our findings demonstrate the value of long-term studies in VR and can be used to inform the design of better, more ergonomic VR systems as tools for knowledge work.
Investigating Whether the Mass of a Tool Replica Influences Virtual Training Learning Outcomes (Journal: P1408)
Julien Cauquis, CLARTE; Etienne Peillard, Lab-STICC; Lionel Dominjon, CLARTE; Thierry Duval, IMT Atlantique; Guillaume Moreau, IMT Atlantique
Designing an immersive training experience often involves creating a replica of a tool. Making a replica can make its mass different from that of the original tool. To investigate the impact of this difference, an immersive training experience was designed, with pre and post-training phases, for learning to use a rotary tool. 80 participants were divided into three groups: a control group using a replica with the same mass as the original tool, a group using a lighter replica, and a group using a heavier replica. Learning outcomes were comparable across all groups, with certain performance measures showing improvement.
Beyond the Wizard of Oz: Negative Effects of Imperfect Machine Learning to Examine the Impact of Reliability of Augmented Reality Cues on Visual Search Performance (Journal: P1529)
Aditya Raikwar, Colorado State University; Domenick Mifsud, Georgia Institute of Technology; Christopher Wickens, Colorado State University; Anil Ufuk Batmaz, Concordia University; Amelia C. Warden, Colorado State University; Brendan Kelley, Colorado State University; Benjamin A. Clegg, Montana State University; Francisco Raul Ortega, Colorado State University
Current Machine Learning algorithms aren't perfect at finding and providing visual cues. This might affect users' perceived reliability of the algorithm and, thus, search performance. Here, we examined the detrimental effects of automation bias caused by imperfect cues presented in the Augmented Reality head-mounted display using the YOLOv5 machine learning model. Two groups of 53 participants received either 100% or 88.9% accurate cues compared to a control group with no cues. The results show how cueing may increase performance and shorten search times. Additionally, the performance with imperfect automation was much worse than perfect automation and consistent with automation bias; participants were frequently enticed by incorrect cues.
Task and Environment-Aware Virtual Scene Rearrangement for Enhanced Safety in Virtual Reality (Journal: P1735)
Bing Ning, Beijing Institute of Technology; Mingtao Pei, Beijing Institute of Technology
Emerging VR applications have revolutionized user experiences by immersing individuals in digitally crafted environments.However, fully immersive experiences introduce new challenges, notably the risk of physical hazards when users are unaware of their surroundings. To address these challenges, we propose a novel approach that dynamically rearranges VR scenes according to users’ physical spaces, seamlessly embedding physical constraints and interaction tasks into the virtual environment. We design a computational model to optimize the rearranged scene through a cost function, ensuring collision-free interactions while maintaining visual fidelity and the goal of interaction tasks.
Session: Perception in AR, MR and Near-Eye Displays (WE4G)
Date & Time: Wednesday, 20 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Guillaume Moreau
Evaluating Transitive Perceptual Effects Between Virtual Entities in Outdoor Augmented Reality (Conference: P1020)
Juanita Benjamin, University of Central Florida; Austin Erickson, KBR, Air Force Research Lab; Matt Gottsacker, University of Central Florida; Gerd Bruder, University of Central Florida; Greg Welch, University of Central Florida
One of the most significant issues with augmented reality display systems is the perception of computer-generated content, which differs from human perception of real-world objects or entities. This can lead to distrust in these systems, which can have negative consequences for various applications such as distance and size judgments. We present a study that investigates how individuals perceive size, distance, and speed on 3D augmented reality objects in an outdoor setting. In this experiment, participants made estimated judgments about two types of entities: unfamiliar and familiar. Distance and speed were found to be underestimated, whereas size estimates were close to accurate.
Omnidirectional Virtual Visual Acuity: A User-centric Visual Clarity Metric for Virtual Reality Head-mounted Displays and Environments (Journal: P1251)
Jialin Wang, Xi'an Jiaotong-Liverpool University; Rongkai Shi, Xi'an Jiaotong-Liverpool University; Xiaodong Li, Xi'an Jiaotong-Liverpool University; Yushi Wei, Xi'an Jiaotong-Liverpool University; Hai-Ning Liang, Xi'an Jiaotong-Liverpool University
Users’ perceived image quality of VR HMDs depends on factors such as display resolution, optical system, structure, and user's visual acuity. Existing metrics like pixels per degree (PPD) have limitations and do not allow accurate comparison across different VR HMDs and applications. This work presents an end-to-end and user-centric visual clarity metric, omnidirectional virtual visual acuity (OVVA), that measures the virtual visual acuity of both central and non-central areas. OVVA provides an intuitive and accurate reference for VR HMDs and applications sensitive to visual accuracy of details and can be effectively used as a standard metric for comparing VR HMDs.
Comparing Synchronous and Asynchronous Task Delivery in Mixed Reality Environments (Journal: P1466)
Lara Sofie Lenz, ETH Zürich; Andreas Rene Fender, ETH Zürich; Julia Chatain, ETH Zurich; Christian Holz, ETH Zürich
Asynchronous digital communication is a widely applied and well-known form of information exchange. Purely digital messaging allows recipients to process them immediately (synchronous) or whenever they have time (asynchronous). Mixed Reality (MR) systems have the potential to not only handle digital interruptions but also interruptions in physical space, e.g., caused by co-workers in offices. However, the benefits of such MR systems previously remained untested. We conducted a user study (N=26) to investigate what impact the timing of task delivery in MR have on the participants' performance, workflow, and emotional state. Our results show that delaying interruptions has a significant impact on the perceived workload.
Perceptual Thresholds for Radial Optic Flow Distortion in Near-Eye Stereoscopic Displays (Journal: P1597)
Mohammad R. Saeedpour-Parizi, Meta; Niall L. Williams, University of Maryland, College Park; Timothy L Wong, Meta Platforms, Inc.; Phillip Guan, Meta Platforms, Inc.; Dinesh Manocha, University of Maryland; Ian M Erkelens, Facebook Reality Labs
We used a custom wide field of view, near-eye HMD simulator to measure how sensitive observers are to radial optic flow motion artifacts in stereoscopic displays, and to what extent we can leverage blinks to decrease observers' sensitivity. Results showed that visual sensitivity was reduced by a factor of 10 at the start and for ~70 ms after a blink was detected, which implies that a rapid radial optic flow distortion can go unnoticed during blinks. These results provide empirical data that can inform the engineering requirements for both hardware design and graphical correction algorithms for future varifocal near-eye displays.
Visual Illusion Created by a Striped Pattern through Augmented Reality for the Prevention of Tumbling on Stairs (Invited Journal: P3003)
Satoshi Miura, Department of Mechanical Engineering, Tokyo Institute of Technology, 2-12-1, Ookayama, Meguro-ku, Tokyo, Japan; Ryota Fukumoto, Department of Modern Mechanical Engineering, Waseda University, Nishi-Waseda, Tokyo, Japan; Naomi Okamura, Department of Modern Mechanical Engineering, Waseda University, Nishi-Waseda, Tokyo, Japan; Masakatsu G. Fujie, Department of Modern Mechanical Engineering, Waseda University, Nishi-Waseda, Tokyo, Japan; Shigeki Sugano, Department of Modern Mechanical Engineering, Waseda University, Nishi-Waseda, Tokyo, Japan
A fall on stairs can be a dangerous accident. An important indicator of falling risk is the foot clearance, which is the height of the foot when ascending stairs or the distance of the foot from the step when descending. We developed an augmented reality system with a holographic lens using a visual illusion to improve the foot clearance on stairs. The system draws a vertical striped pattern on the stair riser as the participant ascends the stairs to create the illusion that the steps are higher than the actual steps, and draws a horizontal striped pattern on the stair tread as the participant descends the stairs to create the illusion of narrower stairs. We experimentally evaluated the accuracy of the system and fitted a model to determine the appropriate stripe thickness. Finally, participants ascended and descended stairs before, during, and after using the augmented reality system. The foot clearance significantly improved, not only while the participants used the system but also after they used the system compared with before.
A SharpView Font with Enhanced Out-of-Focus Text Legibility for Augmented Reality Systems (Conference: P1061)
Mohammed Safayet Arefin, Colorado State University; Carlos Montalto, University of Costa Rica; Alexander Plopski, TU Graz; J. Edward Swan II, Mississippi State University
In optical see-through augmented reality system, users are required to frequently change their eye focus from one distance to another to view virtual and real information at different distances, and only one piece of information is in focus while the other is out of focus. This paper introduces a novel SharpView font, which looks sharper and more legible than standard fonts when seen out of focus. Our method models out-of-focus blur with Zernike mathematical formulations, develops a focus correction algorithm, and proposes a novel sharpness quantification algorithm. Simulation and camera-based measurement showed that SharpView font is significantly sharper than standard fonts.
Session: Rendering and Displays (WE4H)
Date & Time: Wednesday, 20 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Shohei Mori
Divide-Conquer-and-Merge: Memory- and Time-Efficient Holographic Displays (Conference: P1457)
Zhenxing Dong, Shanghai Jiao Tong University; Jidong Jia, Shanghai Jiao Tong University; Yan Li, Shanghai Jiao Tong University; Yuye Ling, Shanghai Jiao Tong University
We proposed a divide-conquer-and-merge strategy to address the memory and computational capacity scarcity in ultra-high-definition CGH generation. By integrating our strategy into existing SOTA methods, HoloNet and CCNNs, we achieved significant reductions in GPU memory usage during the training period by 64.3% and 12.9%, respectively. Furthermore, we observed substantial speed improvements in hologram generation, with an acceleration of up to 3× and 2×, respectively. Particularly, we successfully trained and inferred 8K definition holograms on an NVIDIA GeForce RTX 3090 GPU for the first time in simulations. Furthermore, we conducted full-color optical experiments to verify the effectiveness of our method.
StainedSweeper: Compact, Variable-Intensity Light-Attenuation Display with Sweeping Tunable Retarders (Journal: P2042)
Yuichi Hiroi, Cluster Metaverse Lab; Takefumi Hiraki, Cluster Metaverse Lab; Yuta Itoh, The University of Tokyo
We propose StainedSweeper, a compact light attenuation display (LAD) that achieves both the wide color gamut and the variable intensity with a single SLM. Our system synchronously controls a pixel-wise Digital Micromirror Device (DMD) and a nonpixel polarizing color filter to pass light when each pixel is the desired color. By sweeping this control at high speed, the human eye perceives images in a time-multiplexed, integrated manner. To achieve this, we develop the OST-HMD design using a reflective Solc filter as a polarized color filter, a color reproduction algorithm, and a proof-of-concept prototype.
Retinotopic Foveated Rendering (Conference: P1137)
Yan Zhang, Shanghai Jiao Tong University; Keyao You, Shanghai Jiao Tong University; Xiaodan Hu, NAIST; Hangyu Zhou, Shanghai Jiao Tong University; Kiyoshi Kiyokawa, Nara Institute of Science and Technology; Xubo Yang, SHANGHAI JIAO TONG UNIVERSITY
Existing foveated rendering (FR) works based on the radially symmetric regression model of human visual acuity. However, horizontal-vertical asymmetry and vertical meridian asymmetry of the human visual system have been evidenced by retinotopy research of neuroscience, suggesting the radially asymmetric regression of visual acuity. In this paper, we begin with the fMRI data and then introduce a radially asymmetric regression model for leveraging the rendering precision of FR. Our user study demonstrates retinotopic foveated rendering (RFR) provides participants with perceptually equal image quality compared to typical FR methods while reducing fragments shading by 27.2% averagely, leading to the acceleration of 1/6 for graphics rendering.
CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency (Conference: P1386)
Hanxin Zhu, University of Science and Technology of China; Zhibo Chen, University of Science and Technology of China
We propose CMC, a novel method for few-shot novel view synthesis via cross-view multiplane consistency. Our key insight is that by forcing the same spatial points to be sampled repeatedly in different input views, we can strengthen the interactions between views and therefore alleviate the overfitting problem. To achieve this, we build the neural networks on layered representations (\textit{i.e.}, multiplane images), and the sampling point can thus be resampled on multiple discrete planes. Furthermore, to regularize the unseen target views, we constrain the rendered colors and depths from different input views to be the same. Experiments demonstrate that our proposed method can achieve better synthesis quality over state-of-the-art methods
Expressive Talking Avatars (Journal: P1970)
Ye Pan, Shanghai Jiaotong University; Shuai Tan, Shanghai Jiao Tong University; Shengran Cheng, Shanghai Jiaotong University; Qunfen Lin, Tencent Games; Zijiao Zeng, Tencent Games; Kenny Mitchell, Edinburgh Napier University
We build the Emotional Talking Avatar Dataset which is a talking-face video corpus featuring 6 different stylized characters talking with 7 different emotions. Together with the dataset, we also release an emotional talking avatar generation method which enables the manipulation of emotion. We validated the effectiveness of our dataset and our method in generating audio-based puppetry examples, including comparisons to state-of-the-art techniques and a user study. Finally, various applications of this method are discussed in the context of animating avatars in VR.
Neural Bokeh: Learning Lens Blur for Computational Videography and Out-of-Focus Mixed Reality (Conference: P2106)
David Mandl, Graz University of Technology; Shohei Mori, Graz University of Technology; Peter Mohr, Graz University of Technology; Yifan (Evan) Peng, The University of Hong Kong; Tobias Langlotz, University of Otago; Dieter Schmalstieg, Graz University of Technology; Denis Kalkofen, Graz University of Technology
We present Neural Bokeh, a deep learning approach for synthesizing convincing out-of-focus effects with applications in Mixed Reality (MR) image and video compositing. Unlike existing approaches that solely learn the amount of blur for out-of-focus areas, our approach captures the overall characteristic of the bokeh to enable the seamless integration of rendered scene content into real images, ensuring a consistent lens blur over the resulting MR composition. Our method learns spatially varying blur shapes, which mimics the characteristics of the physical lens. We envision a variety of applications, including visual enhancement of image and video compositing containing creative utilization of out-of-focus effects.
Session: Experiences, Cybersickness and Presence (WE4J)
Date & Time: Wednesday, 20 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Victoria Interrante
Understanding Interaction and Breakouts of Safety Boundaries in Virtual Reality Through Mixed-Method Studies (Conference: P1106)
Wen-Jie Tseng, Technical University of Darmstadt; Petros Dimitrios Kontrazis, Telecom paris; Eric Lecolinet, Institut Polytechnique de Paris; Samuel Huron, Télécom Paris, Institut Polytechnique de Paris; Jan Gugenheimer, TU-Darmstadt
Virtual Reality (VR) technologies facilitate immersive experiences within domestic settings. Despite the prevalence of safety boundaries implemented by commercial VR products to mitigate collisions, empirical insights into how people perceive and interact with these safety features remain underexplored. This paper presents two mixed-method design studies (an online survey and a lab study) to investigate attitudes, behavior, and reasons when interacting with safety boundaries. Our empirical findings reveal VR participants sometimes break out of safety boundaries based on their real-world spatial information. Finally, we discuss improving future VR safety mechanisms by supporting participants' real-world spatial information using landmarks.
PetPresence: Investigating the Integration of Real-World Pet Activities in Virtual Reality (Journal: P1504)
Ningchang Xiong, City University of Hong Kong; Qingqin Liu, City University of Hong Kong; Kening Zhu, City University of Hong Kong
In this paper, we investigate the integration of real-world pet activities into immersive VR interaction. Our pilot study showed that the active pet movements, especially dogs, could negatively impact users' performance and experience in immersive VR. We proposed three different types of pet integration, namely semitransparent real-world portal, non-interactive object in VR, and interactive object in VR. Our user-study results showed that compared to the baseline condition without any pet-integration technique, the approach of integrating the pet as interactive objects in VR yielded significantly higher participant ratings in perceived realism, joy, multisensory engagement, and connection with their pets in VR.
Text2VRScene: Exploring the Paradigm of Automated Generation System for VR Experience From the Ground Up (Conference: P1662)
Zhizhuo Yin, Hong Kong University of Science and Technology (Guangzhou); Yuyang Wang, Hong Kong University of Science and Technology; Theodoros Papatheodorou, Hong Kong University of Science and Technology (Guangzhou); Pan Hui, The Hong Kong University of Science and Technology
This paper explores how to generate VR scenes from text by incorporating LLMs and various generative models into an automated system. This paper first identifies the possible limitations of LLMs for an automated system and proposes a systematic framework to mitigate them. A VR scene generation system named Text2VRScene is developed based on the proposed framework with well-designed prompts. A series of tests are carried out to validate the effectiveness of this system. The results show that the framework contributes to improving the reliability of the system and the quality of the generated VR scenes and illustrate the promising performance of the system in generating satisfying VR scenes with a clear theme.
The Effect of Directional Airflow toward Vection and Cybersickness (Conference: P2117)
Seunghoon Park, Korea University; Seungwoo Son, Korea University; Jungha Kim, Korea University; Gerard Jounghyun Kim, Korea University
Our study investigates the impact of directional airflow on cybersickness and user experience in virtual navigation. Findings reveal that airflow aligned with navigation enhances vection, while inconsistent airflow has no effect. However, airflow's refreshing quality significantly reduces cybersickness overall, regardless of its direction. These insights can guide the design of more dynamic and comfortable virtual navigation experiences
The Effects of Secondary Task Demands on Cybersickness in Active Exploration Virtual Reality Experiences (Journal: P1769)
Rohith Venkatakrishnan, Clemson University; Roshan Venkatakrishnan, Clemson University; Balagopal Raveendranath, Clemson University; Ryan Canales, Clemson University; Dawn M. Sarno, Clemson University; Andrew Robb, Clemson University; Wen-Chieh Lin, National Yang Ming Chiao Tung University; Sabarish V. Babu, Clemson University
During navigation, users often engage in additional tasks that require attentional resources. This work investigated how the attentional demands of secondary tasks performed during exploration affect cybersickness in virtual reality. We manipulated a secondary task's demand across two levels and studied its effects on sickness in two provocative experiences. Results revealed that increased secondary task demand generally exacerbated sickness levels, further vitiating spatial memory and navigational performance. In light of research demonstrating the use of distractions to counteract sickness, our results suggest the existence of a threshold beyond which distractions can reverse from being sickness-reducing to sickness-inducing.
Excuse Me: Large Groups in Small Rooms (Conference: P1348)
Ephraim Schott, Bauhaus-Universität Weimar; Tony Jan Zoeppig, Bauhaus-Universität Weimar; Anton Benjamin Lammert, Bauhaus-Universität Weimar; Bernd Froehlich, Bauhaus-Universität Weimar
Standing in a large crowd can be uncomfortable and usually results in other users obstructing the view of the virtual environment. In this paper, we present four techniques designed to improve the user’s view in crowded environments. Inspired by related work on various transparency techniques, as well as observed user behavior in crowded scenarios, this paper addresses the visibility problem by locally manipulating the appearance of other users. A user study with 24 participants found that the transparency technique was advantageous for quick search tasks. However, in a realistic museum setting, no clear favorite could be determined as the techniques make different trade-offs and users weighted these aspects differently.
Session: Distributed Systems and Telepresence (TH1G)
Date & Time: Thursday, 21 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Andrew Chalmers
Universal Access for Social XR Across Devices: The Impact of Immersion on the Experience in Asymmetric Virtual Collaboration (Conference: P1281)
Christian Merz, University of Würzburg; Christopher Göttfert, University of Würzburg; Carolin Wienrich, University of Würzburg; Marc Erich Latoschik, University of Würzburg
This work examines how immersion in extended reality (XR) affects user experience (UX), focusing on presence, self-perception, other perception, and task perception in social XR contexts. We tested three device configurations in an asymmetric collaborative sorting task. Findings show that presence and self-perception, i.e., embodiment were lower with lower immersion. However, other-perception, i.e., co-presence and social presence were independent of immersion level. Our results suggest that while device choice within asymmetric collaborative XR influences self-perception and presence, it does not significantly impact other key UX aspects like other-perception, supporting the potential for universal access in social XR experiences.
CloVR: Fast-Startup Low-Latency Cloud VR (Journal: P1201)
Yuqi Zhou, Purdue University; Voicu Popescu, Purdue University
This paper introduces CloVR, a cloud VR system ensuring rapid loading for swift user engagement during session startup or after teleportation. The server reduces the original VE to a compact representation through near-far partitioning. The server sends the client an environment map for the far region, along with near region geometry, enabling local, low-latency rendering of high-quality frames. The near region starts out small and grows progressively, with strict visual continuity, minimizing startup time. CloVR's low-latency and fast-startup benefits were confirmed in a study with 8 participants using Quest 2 VR headsets, supported by a laptop server for a collaborative VR app with a 25 million triangle VE.
A3RT: Attention-Aware AR Teleconferencing with Life-Size 2.5D Video Avatars (Conference: P1243)
Xuanyu Wang, Xi'an Jiaotong University; Weizhan Zhang, Xi'an Jiaotong University; Hongbo Fu, City University of Hong Kong
In video-avatar-based multiparty AR teleconferencing, non-verbal cues indicating "who is looking at whom" are always lost or misdelivered. Existing solutions to the awareness of such non-verbal cues lack immersion, are less feasible for everyday usage, or lose remote users' authentic appearances. In this paper, we decompose such attention awareness into the lookee's awareness and the onlooker's awareness, and propose the "Attention Circle" layout and the "rotatable 2.5D video avatar with attention thumbnail" visualization to address them. We implement A3RT, a proof-of-concept prototype that empowers attention-aware 2.5D-video-avatar-based multiparty AR teleconferencing, and conduct evaluations to verify its effectiveness.
MeshReduce: Scalable and Bandwidth Efficient 3D Scene Capture (Conference: P1817)
Tao Jin, Carnegie Mellon University; Mallesham Dasari, Carnegie Mellon University; Connor Smith, NVIDIA; Kittipat Apicharttrisorn, Nokia Bell Labs; Srinivasan Seshan, Carnegie Mellon University; Anthony Rowe, Carnegie Mellon University
This paper introduces MeshReduce, a system for bandwidth-efficient and scalable 3D scene capture and streaming. It addresses the high latency, bandwidth, and scalability issues of traditional methods by employing sensor-side compute nodes for per-sensor mesh reconstruction, decimation, and compression. MeshReduce also introduces an innovative network rate control. MeshReduce efficiently merges independent mesh reconstructions, ensuring adaptive, high-quality streaming. Demonstrated with Azure Kinect depth camera and a custom LiDAR + 360° camera setup, MeshReduce offers a scalable and efficient solution for 3D scene capture and rendering.
Object Cluster Registration of Dissimilar Rooms Using Geometric Spatial Affordance Graph to Generate Shared Virtual Spaces (Conference: P2081)
Seonji Kim, KAIST; Dooyoung Kim, KAIST; Jae-eun Shin, KAIST; Woontack Woo, KAIST
We propose Object Cluster Registration (OCR) using a Geometric Spatial Affordance Graph to support user interaction with multiple objects in a shared space generated from two dissimilar rooms. Previous research on generating a shared virtual space has only considered objects individually and aimed at maximizing the area, leading to limited interactions and neglecting usability. The proposed method extracts optimal object cluster pairs to align dissimilar rooms in generating shared space. The evaluation showed that object correlation preservation was higher when OCR was used with no significant difference in the area size. This suggests that factoring in the relationship between objects does not compromise the objective of maximizing area.
Session: Multimodal Perception and Experiences (TH1H)
Date & Time: Thursday, 21 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: John Quarles
The Differential Effects of Multisensory Attentional Cues on Task Performance in VR Depending on the Level of Cognitive Load and Cognitive Capacity (Journal: P1040)
Sihyun Jeong, KAIST; Jinwook Kim, KAIST; Jeongmi Lee, KAIST
Devising attentional cues that optimize VR task performance has become crucial. We investigated how the effects of attentional cues on task performance are modulated by the levels of cognitive load and cognitive capacity. Participants engaged in dual tasks under different levels of cognitive load while an attentional cue (visual, tactile, or visuotactile) was presented. The results showed that multi-sensory attentional cues are generally more effective than uni-sensory cues in enhancing task performance, but the benefit of multi-sensory cues increases with higher cognitive load and lower cognitive capacity. These findings provide practical implications for designing attentional cues to enhance VR task performance.
AMMA: Adaptive Multimodal Assistants Through Automated State Tracking and User Model-Directed Guidance Planning (Conference: P1803)
Jackie (Junrui) Yang, Stanford University; Leping Qiu, University of Toronto; Emmanuel Angel Corona-Moreno, Stanford University; Louisa Shi, Tsinghua University; Hung Bui, Stanford University; Monica Lam, Stanford University; James A. Landay, Stanford University
Novel technologies such as augmented reality and computer perception lay the foundation for smart task guidance assistants. However, real-world interaction requires adaptation to users’ mistakes, environments, and communication preferences. We propose Adaptive Multimodal Assistants (AMMA), a software architecture for adaptive guidance interfaces with a user action state tracker generated from step-by-step instructions and a guidance planner based on an adapting user model. With this, AMMA can adapt its guidance orders and modalities on the fly. We showed the viability of AMMA in an adaptive cooking assistant in VR. A user study with it showed that AMMA can reduce the task completion time and manual communication methods changes.
Modeling the Impact of Head-Body Rotations on Audio-Visual Spatial Perception for Virtual Reality Applications (Journal: P1307)
Edurne Bernal-Berdun, Universidad de Zaragoza - I3A; Mateo Vallejo, Universidad de Zaragoza - I3A; Qi Sun, New York University; Ana Serrano, Universidad de Zaragoza; Diego Gutierrez, Universidad de Zaragoza
Proper synchronization of visual and auditory feedback is crucial for perceiving a coherent and immersive virtual reality (VR) experience. We investigate how audio-visual offsets and rotation velocities impact users' directional localization acuity during natural head-body rotations. Using psychometric functions, we model perceptual disparities and identify offset detection thresholds. Results show that target localization accuracy is affected by perceptual audio-visual disparities during head-body rotations when there is a stimuli-head relative motion. We showcase with a VR game how a compensatory approach based on our study can enhance localization accuracy by up to 40%. Similarly, we provide guidelines for enhancing VR content creation.
"May I Speak?": Multi-modal Attention Guidance in Social VR Group Conversations (Journal: P1485)
Geonsun Lee, University of Maryland; Dae Yeol Lee, Dolby Laboratories; Guan-Ming Su, Dolby Labs; Dinesh Manocha, University of Maryland
In this paper, we introduce a unique multi-modal attention guidance method to improve turn-taking in virtual reality (VR) meetings. Addressing the limited field of view and lack of gesture tracking in VR, our method helps users notice new speakers more easily. It provides customized cues based on participant engagement levels, enhancing meeting dynamics. We developed a prototype using light as a guidance tool and spatial audio for an immersive experience, directing attention to new speakers. Our evaluation study showed our method's superiority over existing ones in response time, conversation satisfaction, and user preference, offering insights for future VR social attention guidance research.
A User Study on Sharing Physiological cues in VR Assembly tasks (Conference: P1821)
Prasanth Sasikumar, National University of Singapore; Ryo Hajika, The University of Auckland; Tamil Selvan Gunasekaran, The University of Auckland; Kunal Gupta, The University of Auckland; Yun Suen Pai, Keio University Graduate School of Media Design; Huidong Bai, The University of Auckland; Suranga Nanayakkara, Department of Information Systems and Analytics, National University of Singapore; Mark Billinghurst, University of South Australia
Understanding your partner's emotions is crucial for success in collaborative settings, particularly in remote collaboration. Our solution involves a visual representation enabling the inference of emotional patterns from physiological data, potentially influencing communication styles. A study explored the impact of this visual representation in remote collaboration, revealing minimal variation in workload linked to observing physiological cues. However, participants predominantly favored monitoring their partner's attentional state. Results indicate that most participants allocated only a small portion of their attention to partner's physiological cues, often unsure how to interpret and use the obtained information.
Session: Depth and Distance Perception (TH1J)
Date & Time: Thursday, 21 March 2024, 8:30-9:45 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Daisuke Iwai
An Empirical Evaluation of the Calibration of Auditory Distance Perception under Different Levels of Virtual Environment Visibilities (Conference: P1669)
Wan-Yi Lin, National Yang Ming Chiao Tung University; Rohith Venkatakrishnan, University of Florida; Roshan Venkatakrishnan, University of Florida; Sabarish V. Babu, Clemson University; Christopher Pagano, Clemson University; Wen-Chieh Lin, National Yang Ming Chiao Tung University
We investigated if perceptual learning through carryover effects of calibration occurs in different levels of a virtual environment’s visibility. Users performed an auditory depth judgment task over several trials in which they walked where they perceived an aural sound to be. This task was sequentially performed in the pretest, calibration, and posttest phases. Feedback on the perceptual accuracy of distance estimates was only provided in the calibration phase. We found that auditory depth estimates, obtained using an absolute measure, can be calibrated to become more accurate and that environments visible enough to reveal their extent may contain visual information that users attune to in scaling aurally perceived depth.
Assessing Depth Perception in VR and Video See-Through AR: A Comparison on Distance Judgment, Performance, and Preference (Journal: P1027)
Franziska Westermeier, Human-Computer Interaction Group, University of Würzburg; Larissa Brübach, University of Würzburg; Carolin Wienrich, University of Würzburg; Marc Erich Latoschik, University of Würzburg
This article investigates depth perception differences in Virtual Reality (VR) and Video See-Through Augmented Reality (VST AR). Thirty-two participants perform depth-related tasks in a physical office room and its virtual replica. Our results show higher rates of depth misjudgment, longer task completion times, and increased head movements in VST AR, likely due to conflicting virtual and physical depth cues. Surprisingly, a preference for the VST AR experience was evident among participants. This paradox is discussed considering the visual dominance of physical content and referential power in VST AR, fostering a strong sense of spatial presence and plausibility.
Exploring Depth-based Perception Conflicts in Virtual Reality through Error-Related Potentials (Conference: P1698)
Haolin Gao, Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology; Kang Yue, Institute of Software, Chinese Academy of Sciences; Songyue Yang, Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology; Yu Liu, Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology; Mei Guo, Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology; Yue Liu, Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology
Previous research has indicated that human’s perception of depth in virtual reality (VR) differs from the real world. In this paper, we investigated users’ depth perception conflicts at varying depths within near-field in VR using Error-Related Potentials (ErrPs) from electroencephalography (EEG) data. We employed a 3 (target depth) × 2 (conflict or non-conflict) experimental design, mimicking depth perception conflicts within the near-field. We collected questionnaire results, performance data, and EEG data from users. Our results showed that a significant effect of target depth on conflict perception, and the amplitude of an ErrPs component increases with the degree of depth perception conflict.
Low-Latency Ocular Parallax Rendering and Investigation of Its Effect on Depth Perception in Virtual Reality (Journal: P1203)
Yuri Mikawa, NTT Communication Science Laboratories; Taiki Fukiage, NTT Communication Science Laboratories
Recently, ocular parallax, a small parallax generated by eye rotation, has received considerable attention for its impact on depth perception in VR/AR displays. However, the substantial latency of head-mounted displays (HMDs) has made it challenging to accurately assess its true effect during free eye movements. We propose a high-speed (360 Hz) and low-latency (4.8 ms) ocular parallax rendering system with a custom-built eye tracker. Using this proposed system, we investigated the latency requirements necessary for achieving perceptually stable ocular parallax rendering along with its effects on binocular fusion and monocular depth perception under free viewing conditions.
Invisible Mesh: Effects of X-Ray Vision Metaphors on Depth Perception in Optical-See-Through Augmented Reality (Conference: P1791)
Haoyu Tan, University of Minnesota; Tongyu Nie, University of Minnesota; Evan Suma Rosenberg, University of Minnesota
This paper investigates the impact of X-ray vision metaphors on distance estimation in optical-see-through augmented reality in action space. A within-subjects study was conducted to evaluate depth judgments across five conditions, including a novel "invisible mesh" technique. Although quantitative results regarding the impact of different X-ray vision metaphors on distance perception were inconclusive, participant feedback revealed a diversity of strategies and preferences. Overall, the findings suggest that no single metaphor was considered universally superior. This research contributes to understanding of X-ray vision techniques and informs the design considerations for AR systems aiming to enhance depth perception and user experience.
Session: Touch, Tangible, and Gesture Interfaces (TH2G)
Date & Time: Thursday, 21 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Rob Lindeman
A Comparative Usability Study of Physical Multi-touch versus Virtual Desktop-Based Spherical Interfaces (Conference: P1255)
Nikita Soni, University of Illinois Chicago; Oluwatomisin Obajemu, University of Florida; Katarina Jurczyk, University of Florida; Chaitra Peddireddy, University of Florida; Maeson Vallee, University of Florida; Ailish Tierney, University of Florida; Niloufar Saririan, University of Florida; Cameron John Zuck, University of Florida; Kathryn A. Stofer, University of Florida; Lisa Anthony, University of Florida
Physical multi-touch spherical displays offer a hands-on experience for visualizing global data but are often too costly for learning institutions. As an alternative, virtual spherical interfaces like Google Earth have been used, but it's unclear if they offer a similar usability experience. We ran a within-subjects study with 21 users to compare usability experiences of physical and virtual spherical interfaces. Our results showed no significant difference in usability or task completion time between two platforms. Participants noted differences in effort and motor demand. Our research implies that, in resource-constrained settings, a virtual globe can be a viable alternative for a physical sphere from a usability perspective.
Supporting Text Entry in Virtual Reality with Large Language Models (Conference: P1851)
Liuqing Chen, Zhejiang University; Yu Cai, Zhejiang University; Ruyue Wang, Zhejiang University; Shixian Ding, Zhejiang University; Yilin Tang, Zhejiang University; Preben Hansen, Stockholm University; Lingyun Sun, Zhejiang University
This study investigates improving text entry efficiency in virtual reality by utilizing large language models. It introduces three LLM-assisted methods: Simplified Spelling, Content Prediction, and Keyword-to-Sentence Generation, aligning with user cognition and English text predictability at various levels. Tested on a VR prototype, these methods significantly reduce manual keystrokes by 16.4% to 49.9%, boosting efficiency by 21.4% to 76.3%. These methods lessen task loads without increasing manual corrections, enhancing overall usability. Long-term observations show that user proficiency with these methods further enhances text entry efficiency.
HandyNotes: using the hands to create semantic representations of contextually aware real-world objects (Conference: P1880)
Clément Quere, Université Côte d'Azur; Aline Menin, Université Côte d'Azur; Raphaël Julien, Université Côte d'Azur; Hui-Yin Wu, Centre Inria d'Université Côte d'Azur; Marco Winckler, Université Côte d'Azur
This paper uses Mixed Reality (MR) technologies to provide a seamless integration of digital information in physical environments through human-made annotations. Creating digital annotations of physical objects evokes many challenges for performing (simple) tasks such as adding digital notes and connecting them to real-world objects. For that, we have developed an MR system using the Microsoft HoloLens2 to create semantic representations of contextually-aware real-world objects while interacting with holographic virtual objects. User interaction is enhanced with use of fingers as placeholders for menu items. We demonstrate our approach through two real-world scenarios. We also discuss the challenges for using MR technologies.
Leveraging Tendon Vibration to Enhance Pseudo-Haptic Perceptions in VR (Invited Journal: P3008)
Yutaro Hirao, Tomohiro Amemiya, Takuji Narumi, Ferran Argelaguet, Anatole Lécuyer
Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this paper proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on the range and resolution of pseudo-haptics. The first experiment investigated the effect of tendon vibration on the detection threshold of the discrepancy between visual and physical motion. The results indicated that vibrations applied to the inner tendons of the wrist and elbow increased the threshold, suggesting that tendon vibration can augment the applicable visual motion gain by approximately 13\% without users detecting the visual/physical discrepancy. Furthermore, the results demonstrate that tendon vibration acts as noise on haptic motion cues. The second experiment assessed the impact of tendon vibration on the resolution of pseudo-haptics by determining the just noticeable difference in pseudo-weight perception. The results suggested that the tendon vibration does not largely compromise the resolution of pseudo-haptics. The third experiment evaluated the equivalence between the weight perception triggered by tendon vibration and that by visual motion gain, that is, the point of subjective equality. The results revealed that vibration amplifies the weight perception and its effect was equivalent to that obtained using a gain of 0.64 without vibration, implying that the tendon vibration also functions as an additional haptic cue. Our results provide design guidelines and future work for enhancing pseudo-haptics with tendon vibration.
Session: Graphics and Crowds (TH2H)
Date & Time: Thursday, 21 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Julien Pettre
Foveated Fluid Animation in Virtual Reality (Conference: P1120)
Yue Wang, Shanghai Jiao Tong University; Yan Zhang, Shanghai Jiao Tong University; Xuanhui Yang, Shanghai Jiao Tong University; Hui Wang, Shanghai Jiao Tong University; Dongxu Liu, Shanghai Jiao Tong University; Xubo Yang, SHANGHAI JIAO TONG UNIVERSITY
In this paper, we propose a novel foveated fluid simulation method that optimizes physics-based fluid simulation by leveraging the human visual system's natural foveation for VR. Dividing the simulation into foveal, peripheral, and boundary regions, our system dynamically allocates computational resources, balancing accuracy and efficiency. Implemented with a multi-scale approach, our method achieves a 2.27x speedup while maintaining perceptual quality. Subjective studies validate its effectiveness, and we explore the impact of metrics like particle radius and viewing distance on visual effects. Our work pioneers real-time foveated fluid simulation in VR, advancing efficiency and realism for fluid-based VR applications.
DreamSpace: Dreaming Your Room Space with Text-Driven Holistic Texture Propagation (Conference: P1172)
Bangbang Yang, Bytedance Inc; Wenqi Dong, Zhejiang University; Lin Ma, PICO MixedReality, Bytedance Inc; Wenbo Hu, PICO MixedReality, Bytedance Inc; Xiao Liu, PICO MixedReality, Bytedance Inc; Zhaopeng Cui, Zhejiang University; Yuewen Ma, PICO MixedReality, Bytedance Inc
DreamSpace allows users to personalize their own spaces' appearances with text prompts and delivers immersive VR experiences on HMD devices. Specifically, given a real-world captured room, we generate enchanting and holistic mesh textures based on the user's textual inputs, while ensuring semantic consistency and spatial coherence, such as the sofa still retains its recognizable form as a sofa, but in fantasy styles.
Exploring the Role of Expected Collision Feedback in Crowded Virtual Environments (Conference: P1712)
Haoran Yun, Universitat Politècnica de Catalunya; Jose Luis Ponton, Universitat Politècnica de Catalunya; Alejandro Beacco, Universitat Politècnica de Catalunya; Carlos Andujar, Universitat Politècnica de Catalunya; Nuria Pelechano, Universitat Politècnica de Catalunya
This study explores how expected collision feedback affects user interaction and perception in virtual environments populated by dynamic virtual humans. While real-world locomotion is influenced by collision risks, such risks are often absent in virtual settings. The paper examines the effectiveness of auditory cues, tactile vibrations, and the expectation of physical bumps in enhancing collision realism in virtual crowds. Results show that anticipated collision risk significantly alters participant behavior—both in overall navigation and local movement—and strengthens the sense of presence and copresence. Auditory cues notably impact navigation and copresence, whereas tactile feedback primarily affects local movements.
With or Without You: Effect of Contextual and Responsive Crowds on VR-based Crowd Motion Capture (Journal: P1727)
Tairan Yin, INRIA; Ludovic Hoyet, Inria; Marc Christie, IRISA; Marie-Paule R. Cani, Ecole Polytechnique, IP Paris; Julien Pettré, Inria
Capturing real crowd motions is challenging. VR helps by immersing users in simulated or motion-capture-based crowds. Users' motions can extend the crowd size using Record-and-Replay (2R), but these methods have limitations affecting data quality. We introduce contextual crowds, combining crowd simulation and 2R for consistent data. We present two strategies: Replace-Record-Replay (3R), where simulated agents are replaced by user data, and Replace-Record-Replay-Responsive (4R), where agents gain responsive capabilities. Evaluated in VR-replicated real-world scenarios, these paradigms yield more natural user behaviors, enhancing captured crowd data consistency.
Session: Ethics in VR (TH2J)
Date & Time: Thursday, 21 March 2024, 10:15-11:15 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Daniel Medeiros
A Scientometric History of IEEE VR (Conference: P1815)
Richard Skarbez, La Trobe University; Dai Jiang, La Trobe University
As of IEEE VR 2023, there have been 30 installments of the IEEE Virtual Reality conference (VR) or its predecessor, the Virtual Reality Annual International Symposium (VRAIS). As such, it seems an opportune time to reflect on the intellectual history of the conference, and by extension, the VR research community. This article uses scientometric techniques to undertake such an intellectual history, using co-word analysis and citation analysis to identify core themes and trends in VR research over time. We identify the papers that have stood the test of time, the most esteemed authors and researchers in the IEEE VR community, and the topics that have shaped our field to date.
Measurement of Empathy in Virtual Reality with Head-Mounted Displays: A Systematic Review (Journal: P1514)
Yongho Lee, Electronics and Telecommunications Research Institute; Heesook Shin, Electronics and Telecommunications Research Institute; Youn-Hee Gil, Electronics and Telecommunications Research Institute
We present a systematic review of 111 papers that measure the impact of virtual experiences created through head-mounted displays (HMDs) on empathy. Our goal was to analyze the conditions and the extent to which virtual reality (VR) enhances empathy. To achieve this, we categorized the relevant literature according to measurement methods, correlated human factors, viewing experiences, topics, and participants. In our meta-analysis, we found that emotional empathy increased temporarily after the VR experience and returned to its original level over time, whereas cognitive empathy remained enhanced. We also found that storytelling and personal characteristics are more important than the composition of the VR scene.
HardenVR: Harassment Detection in Social Virtual Reality (Conference: P1633)
Na Wang, George Mason University; Jin Zhou, George Mason University; Jie Li, EPAM; Bo Han, George Mason University; Fei Li, George Mason University; Songqing Chen, George Mason University
Despite the promising prospects of social VR, there is a growing concern about the harassment issue. Existing protections against harassment are highly limited. The deficiency of studies further complicates the situation. To address these challenges, we first build a customized platform to collect data about users’ social interaction behaviors. An analysis of the collected dataset reveals harassment detection depends not only on users’ actions but also on the spatial and temporal relationships. In order to accurately discern harassment, we propose the novel context-aware framework HardenVR, which employs a transformer-based model to learn hand actions and relative poses. The experiments show its accuracy as high as 98.26%.
Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience (Journal: P1998)
Ethan Wilson, University of Florida; Azim Ibragimov, University of Florida; Michael J Proulx, Meta Reality Labs Research; Sai Deep Tetali, Meta Platforms Inc; Kevin Butler, University of Florida; Eakta Jain, University of Florida
We investigate the feasibility of eye tracking systems in immersive VR. Prior research has shown that eye tracking data can be used for re-identification attacks. We investigate multiple real-time privacy mechanisms for eye tracking data in order to prevent unwanted user re-identification while still enabling novel interactions. Our novel evaluation methodology balances privacy metrics with user-centric evaluation, and considers multiple adversarial threat scenarios. We find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance.
Session: Modeling and Simulation (TH3G)
Date & Time: Thursday, 21 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Ben Lok
Interactive Molecular Docking Improved With Human Spatial Perception Using Virtual Reality (Journal: P2030)
Shivam Mishra, California State University, Los Angeles; Missael Corro-Flores, California State University, Los Angeles; Negin Forouzesh, California State University, Los Angeles; David M. Krum, California State University, Los Angeles
This manuscript proposes a novel method to guide adaptive steered molecular dynamics (ASMD) simulations using optimal trajectories collected from interactive molecular dynamics in virtual reality (iMD-VR). The authors outline an experimental protocol for setting up an iMD-VR simulation of a protein-ligand system, specifically HIV-1 protease and the ligand Amprenavir. Users manipulate the system to obtain optimal binding trajectories. These trajectories are then used to calculate a force constant which serves as input for ASMD simulations to steer the ligand towards the binding site. Briefly, the proposed interactive molecular application in 3D demonstrated the advantage of using VR application characteristics.
ARpenSki: Augmenting Ski Training with Direct and Indirect Postural Visualization (Conference: P1204)
Takashi Matsumoto, Tokyo Institute of Technology; Erwin Wu, Tokyo Institute of Technology; Chen-Chieh Liao, Tokyo Institute of Technology; Hideki Koike, Tokyo Institute of Technology
Alpine skiing is a popular winter sport, and several systems have been proposed to enhance training and improve efficiency. However, many existing systems rely on simulation-based environments, which suffer from drawbacks such as a gap between real skiing and the lack of body ownership. To address these limitations, we present ARpenSki, a novel augmented reality (AR) ski training system that employs a see-through head mounted display (HMD) to deliver augmented visual training cues that may be applied on real slopes. The proposed AR system provides a transparent view of the lower half of the field of vision, where we implemented three different AR-based direct and indirect postural visualization methods.
Generating Virtual Reality Stroke Gesture Data from Out-of-Distribution Desktop Stroke Gesture Data (Conference: P1300)
Linping Yuan, The Hong Kong University of Science and Technology; Boyu Li, Hong Kong University of Science and Technology (GuangZhou); Jindong Wang, Microsoft Research Asia; Huamin Qu, The Hong Kong University of Science and Technology; Wei Zeng, The Hong Kong University of Science and Technology (Guangzhou)
This paper exploits ubiquitous desktop strokes as an input source for generating VR stroke gestures. The commonalities between desktop and VR strokes allow the generation of additional dimensions in VR strokes. However, distribution shifts not only exist between desktop and VR datasets but also within each dataset. To build models capable of generalizing to unseen distributions, we propose a novel architecture based on cGANs, with the generator encompassing three steps: discretizing the output space, characterizing latent distributions, and learning conditional domain-invariant representations. We illustrate the applicability of the enriched VR datasets through two applications: VR stroke classification and stroke prediction.
Exploring Augmented Reality's Role in Enhancing Spatial Perception for Building Facade Retrofit Design for Non-experts (Conference: P1589)
John Sermarini, University of Central Florida; Robert A. Michlowitz, University of Central Florida; Joseph LaViola, University of Central Florida; Lori C. Walters, University of Central Florida; Roger Azevedo, University of Central Florida; Joseph T. Kider Jr., University of Central Florida
This paper investigates the decision-making outcomes and cognitive-physical load implications of integrating a Building Information Modeling-driven Augmented Reality (AR) system into retrofitting design and how movement is best leveraged to understand daylighting impacts. We conducted a study with 128 non-expert participants, who were asked to choose a window facade to improve an interior space. We found no significant difference in the overall decision-making outcome between those who used an AR tool or a conventional desktop approach and that greater eye movement in AR was related to non-experts better balancing the complicated impacts facades have on daylight, aesthetics, and energy.
Session: Software (TH3H)
Date & Time: Thursday, 21 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Takefumi Hiraki
DreamCodeVR: Towards Democratizing Behavior Design in Virtual Reality with Speech-Driven Programming (Conference: P1685)
Daniele Giunchi, University College London; Nels Numan, University College London; Elia Gatti, University College London; Anthony Steed, University College London
VR content creation remains a complex and challenging task, requiring specialized skills and knowledge. Powered by large language models (LLMs), DreamCodeVR is designed to assist users, irrespective of their coding skills, in crafting basic object behavior in VR environments by translating spoken language into code within an active application. Our preliminary user study indicated that the system can support elementary programming tasks. However, it also uncovered a wide range of challenges and areas for future research, which we detail in an extensive discussion.
VR.net: A Real-world Large-scale Dataset for Virtual Reality Motion Sickness Research (Journal: P1422)
Elliott Wen, The University of Auckland; Chitralekha Gupta, National University of Singapore; Prasanth Sasikumar, National University of Singapore; Mark Billinghurst, University of South Australia; James Wilmott, Meta; Emily Skow, Meta; Arindam Dey, Meta; Suranga Nanayakkara, The University of Auckland
This paper introduces VR.net, a dataset with 165 hours of gameplay videos from 100 real-world games spanning 10 genres, evaluated by 500 participants. The dataset assigns 24 motion sickness-related labels per video frame. They are automatically extracted from 3D engines' rendering pipelines. VR.net's substantial scale, accuracy, and diversity present unmatched opportunities for VR motion sickness research and beyond.
ProtoColVR: Requirements Gathering and Collaborative Rapid Prototyping of VR Training Simulators for Multidisciplinary Teams (Journal: P1720)
Vivian Gómez, Universidad de los Andes; Pablo Figueroa, Universidad de los Andes
ProtoColVR is a methodology and plugin for collaborative and rapid prototyping of virtual reality training simulators. It leverages current technologies, involves stakeholders in design and development, and implements simulator creation through multiple iterations. Open-source tools and free environments like Twine and Unity are integrated. Is a result of experienced in two projects with Hospital and our Navy, ProtoColVR has undergone testing in a development Jam, providing valuable insights. These include the ability to create functional prototypes in multidisciplinary teams, enhance communication among different roles, and streamline requirements gathering while improving understanding of the virtualized environment.
PLUME : Record, Replay, Analyze and Share User Behavior in 6-DoF XR Experiences (Journal: P1746)
Charles Javerliat, Centrale Lyon, Univ Lyon, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205; Sophie Villenave, Centrale Lyon, Univ Lyon, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205; Pierre Raimbaud, Centrale Lyon, Univ Lyon, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205; Guillaume Lavoué, Centrale Lyon, Univ Lyon, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205
Exploiting behavioral data, such as user movements, gaze, actions, and physiological signals is essential to understanding the user experience during XR experiments. However, it remains challenging in 6DoF XR scenarios. We introduce PLUME, an open-source software toolbox designed to ease and democratize the collection, sharing, and analysis of such data. PLUME Recorder is a drag-n-drop Unity plugin that allows for the exhaustive record of XR behavioral data, including synchronous physiological signals, in a compact and interoperable format. PLUME Viewer is a standalone application that enables offline interactive replay and visual analysis. PLUME Python provides compatibility with existing analysis workflows.
Session: Localization and Tracking (TH3J)
Date & Time: Thursday, 21 March 2024, 11:30-12:30 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Yuichi Hiroi
Force-regulated Elastic Linear Objects Tracking for Virtual and Augmented Reality (Conference: P1810)
Yusheng Luo, Southeast University; Lifeng Zhu, Southeast University; Aiguo Song, Southeast University
We present a method for interactive tracking of elastic rods, which are commonly encountered in daily life. Although humans are sensitive to shape changes of rods, estimating external forces responsible for the deformation is not intuitive. We develop a particle filter-based framework and employ the Cosserat rod model to regulate noisy points, turning the inverse physics problem into a forward simulation and search problem. The method can simultaneously digitalize shapes and external forces on real-world elastic rods. We demonstrate virtual and augmented reality applications to facilitate the interaction with elastic linear objects. The tracking performance is also validated with experiments.
Corr-Track: Category-Level 6D Pose Tracking with Soft-Correspondence Matrix Estimation (Journal: P1150)
Xin Cao, Shandong University; Jia Li, Shandong University; Panpan Zhao, Shandong University; Jiachen Li, Zhejiang University; Xueying Qin, Shandong University
Category-level pose tracking methods can continuously track the poses of objects without requiring prior knowledge of the specific shape of the tracked instances. This feature is particularly advantageous in augmented reality and virtual reality applications. We propose Corr-Track, a novel category-level 6D pose tracking method, designed to accurately predict the poses of previously unseen objects within the same category from depth video streams. Corr-Track introduces a soft correspondence matrix and establishes effective constraints through direct spatial point-to-point correspondence representation. Additionally, Corr-Track employs a "point cloud expansion" strategy to effectively address the problem of "point cloud shrinkage" problem.
100-Phones: A Large VI-SLAM Dataset for Augmented Reality Towards Mass Deployment on Mobile Phones (Journal: P1188)
Guofeng Zhang, Zhejiang University; Jin Yuan, Sensetime; Haomin Liu, Sensetime; Zhen Peng, SenseTime; Chunlei Li, Sensetime; Zibin Wang, Sensetime; Hujun Bao, Zhejiang Univeristy
Visual-inertial SLAM (VI-SLAM) is a key technology for Augmented Reality (AR). The current VI-SLAM methods still face robustness challenges when deployed on mid- and low-end smartphones. Existing VI-SLAM datasets use either very ideal sensors or only a limited number of devices for data collection, which cannot reflect the capability gaps that VI-SLAM methods need to solve. This work proposes 100-Phones, the first VI-SLAM dataset covering a wide range of devices. Through analysis and experiments on the collected data, we conclude that the quality of visual-inertial data vary greatly among the mainstream phones, and the current VI-SLAM methods still have serious robustness issues when it comes to mass deployment on mobile phones.
StageAR: Markerless Mobile Phone Localization for AR in Live Events (Conference: P1816)
Tao Jin, Carnegie Mellon University; Shengxi Wu, Carnegie Mellon University; Mallesham Dasari, Carnegie Mellon University; Kittipat Apicharttrisorn, Nokia Bell Labs; Anthony Rowe, Carnegie Mellon University
Localizing mobile phone users for AR in dynamic theater and concert settings is challenging due to changing staging and lighting. Visual markers disrupt aesthetics, and static maps aren't reliable amidst visual changes. Our study explores techniques using sparse infrastructure to adapt to environmental changes for robust AR tracking. Our basic method uses fixed cameras to filter unreliable feature points from a static model. For tougher settings, our technique generates dynamic 3D feature maps for real-time scene adaptation, allowing precise mobile phone localization without markers. We assess StageAR's various techniques. Our top method rivals the accuracy of large static markers while remaining unseen by the audience.
Session: Education Applications (TH4G)
Date & Time: Thursday, 21 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom G
Session Chair: Pablo Figueroa
HyperXRC: Hybrid In-Person + Remote Extended Reality Classroom - A Design Study (Conference: P1011)
Siyu Huang, Purdue University; Voicu Popescu, Purdue University
This paper investigates HyperXRC, a hybrid classroom design that accommodates both local and remote students. The instructor wears an extended reality (XR) headset that shows the local classroom and the local students, as well as remote students modeled with live video sprites placed in empty classroom seats. A controlled user study reveals that instructors are more likely to detect remote student actions in the HyperXRC condition (59%) than in a conventional videoconferencing condition (36%), and that local students prefer the lecture when the instructor does not wear the XR headset.
Understanding Online Education in Metaverse: Systems and User Experience Perspectives (Conference: P1212)
Ruizhi Cheng, George Mason University; Erdem Murat, George Mason University; Lap-Fai Yu, George Mason University; Songqing Chen, George Mason University; Bo Han, George Mason University
Virtual reality (VR) is becoming increasingly popular in online education. However, there is currently no in-depth investigation of the user experience of VR-based online education and the comparison of it with video-conferencing-based counterparts. To fill these critical gaps, we conduct multiple sessions of two courses in a university with 10 and 37 participants on Mozilla Hubs (Hubs for short), a social VR platform, and let them compare the classroom experience on Hubs with Zoom, a popular video-conferencing application. Besides employing traditional analytical methods to understand user experience, we benefit from a measurement study of Hubs to corroborate our findings and systematically detect its performance bottlenecks.
Visuo-Haptic VR and AR Guidance for Dental Nerve Block Education (Journal: P1344)
Sara Samuel, Columbia University; Carmine Elvezio, Columbia University; Salaar Khan, Columbia University; Laureen Zubiaurre Bitzer, Columbia University; Letty Moss-Salentijn, Columbia University; Steven Feiner, Columbia University
The inferior alveolar nerve block (IANB) is a critical dental anesthetic injection that dental students frequently learn to administer through videos and practice on silicone molds or other students. To reduce discomfort and improve clinical outcomes, we created a VR headset-based IANB educational application combining a layered 3D anatomical model, dynamic visual guidance, and force feedback to emulate interaction with tissue, and a companion mobile AR app. We performed a study that showed that compared to students who used only traditional study materials, students who used our system were more confident administering their first clinical injections, had less need for syringe readjustment, and had greater success in numbing patients.
Learning Middle-Latitude Cyclone Formation up in the Air: Student Learning Experience, Outcomes, and Perceptions in a CAVE-enabled Meteorology Class (Journal: P1613)
Hao He, Emporia State University; Xinhao Xu, University of Missouri; Shangman Li, University of Missouri-Columbia; Fang Wang, University of Missouri; Isaac Schroeder, University of Missouri-Columbia; Eric M. Aldrich, University of Missouri; Scottie D Murrell Mr, University of Missouri; Lanxin Xue, University of Missouri-Columbia; Yuanyuan Gu, University of Missouri-Columbia
Cave Automatic Virtual Environments (CAVE) are less studied due to the high cost and complexity of system integration. In this study, we studied how CAVE impacted learners’ learning outcomes and how learners perceived their learning experiences and outcomes. The results indicated that their learning outcomes increased after using CAVE, and their perceptions of immersion, presence, and engagement significantly correlated with each other. Learners showed a great fondness of and satisfaction with the learning experience, group collaboration, and effectiveness of the CAVE-enabled class. The learners’ learning experiences in CAVE could be further improved if we provided more interaction and reduced cybersickness. Implications are discussed.
An Overview of Enhancing Distance Learning Through Emerging Augmented and Virtual Reality Technologies (Invited Journal: P3001)
Elizabeth Childso, University of Maryland USA; Ferzam Mohammado, University of Maryland USA; Logan Stevenso, University of Maryland USA; Hugo Burbelo, University of Maryland USA; Amanuel Awoke, University of Maryland USA; Nicholas Rewkowski, University of Maryland USA; Dinesh Manocha, University of Maryland USA
Although distance learning presents a number of interesting educational advantages as compared to in-person instruction, it is not without its downsides. We first assess the educational challenges presented by distance learning as a whole and identify 4 main challenges that distance learning currently presents as compared to in-person instruction: the lack of social interaction, reduced student engagement and focus, reduced comprehension and information retention, and the lack of flexible and customizable instructor resources. After assessing each of these challenges in-depth, we examine how AR/VR technologies might serve to address each challenge along with their current shortcomings, and finally outline the further research that is required to fully understand the potential of AR/VR technologies as they apply to distance learning.
Beyond Slideshows-Investigating the Impact of Immersive Virtual Reality on Science Learning (Conference: P1787)
Fabian Froehlich, NYU; Chris Hovey, NYU; Sameen Reza, New York University; Jan Plass, New York University
We investigated the effectiveness of immersive VR (VR) for science learning by comparing an VR environment with traditional learning. One group learned about cell biology with a headmounted display using interactive simulation modules in VR, the other with a slideshow presenting the same materials. This study focused on the research question: "Does learning in immersive VR designed to take advantage of VR affordances lead to higher learning outcomes and affective outcomes compared to traditional instruction?" In an experimental between-subject control group design (N= 63), we measured students’ recall and affect. Results indicate that the VR group scored significantly higher than the control group on some dimensions of the post assessment.
Session: Virtual Interaction and Embodiment (TH4H)
Date & Time: Thursday, 21 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom H
Session Chair: Frank Steinicke
Investigating the Correlation Between Presence and Reaction Time in Mixed Reality (Invited Journal: P3012)
Yasra Chandio, University of Massachusetts Amherst, USA; Noman Bashir, University of Massachusetts Amherst, USA; Victoria Interrante, University of Minnesota Twin Cities, USA; Fatima M. Anwar, University of Massachusetts Amherst, USA
Measuring presence is critical to improving user involvement and performance in Mixed Reality (MR). Presence , a crucial aspect of MR, is traditionally gauged using subjective questionnaires, leading to a lack of time-varying responses and susceptibility to user bias. Inspired by the existing literature on the relationship between presence and human performance, the proposed methodology systematically measures a user's reaction time to a visual stimulus as they interact within a manipulated MR environment. We explore the user reaction time as a quantity that can be easily measured using the systemic tools available in modern MR devices. We conducted an exploratory study (N=40) with two experiments designed to alter the users' sense of presence by manipulating place illusion and plausibility illusion . We found a significant correlation between presence scores and reaction times with a correlation coefficient -0.65, suggesting that users with a higher sense of presence responded more swiftly to stimuli. We develop a model that estimates a user's presence level using the reaction time values with high accuracy of up to 80%. While our study suggests that reaction time can be used as a measure of presence, further investigation is needed to improve the accuracy of the model.
In case of doubt, one follows one's self: the implicit guidance of the embodied self-avatar (Journal: P1274)
Loën Boban, École Polytechnique Fédérale de Lausanne (EPFL); Ronan Boulic, École Polytechnique Fédérale de Lausanne (EPFL); Bruno Herbelin, École Polytechnique Fédérale de Lausanne (EPFL)
In virtual reality, the sense of embodiment is explained by multi-sensory integration mechanisms where the avatar feedback is successfully merged with bodily perceptions. It was previously observed that, in cases of small or progressive temporal and spatial manipulations of the avatars’ movements, participants may spontaneously follow their avatar’s movement. This study extends observations of this phenomenon to large movement manipulations and shows that, in case of doubt about which movement to perform during a recall task, participants can be influenced by their avatar’s movements, despite their awareness about both the avatar movement disruption and on the possible influence it had on their choice.
The Utilitarian Virtual Self - Using Embodied Personalized Avatars to Investigate Moral Decision-Making in Semi-Autonomous Vehicle Dilemmas (Journal: P1619)
Anca Salagean, University of Bath; Michelle Wu, University of Bath; George Fletcher, University of Bath; Darren Cosker, Microsoft; Danaë Stanton Fraser, University of Bath
Embodied personalized avatars can enhance ecological validity in moral decision-making research. We tested whether avatar personalization and motor control impact moral decision-making, physiological reactions, reaction times and embodiment. Seventeen participants took part in moral dilemmas as semi-autonomous vehicle drivers. Participants took a utilitarian approach by performing harmful actions only to maximize outcomes. There was higher physiological arousal (SCRs; heart rate) for personalized compared to generic avatars, and higher SCRs in motor control versus no motor control conditions. Motor control led to slower reaction times, suggesting more elaborate decision-making processes. Embodiment was also higher for personalized avatars.
Context-Aware Head-and-Eye Motion Generation with Diffusion Model (Conference: P1564)
Yuxin Shen, Yangtze Delta Region Academy of Beijing Institute of Technology; Manjie Xu, Beijing institute of technology; Wei Liang, Beijing Institute of Technology
We introduce a novel two-stage approach to generate context-aware head-and-eye motions across diverse scenes. By harnessing the capabilities of advanced diffusion models, our approach adeptly produces contextually appropriate eye gaze points, further leading to the generation of natural head-and-eye movements. Utilizing Head-Mounted Display (HMD) eye-tracking technology, we also present a comprehensive dataset, which captures human eye gaze behaviors in tandem with associated scene features. We show that our approach consistently delivers intuitive and lifelike head-and-eye motions and demonstrates superior performance in terms of motion fluidity, alignment with contextual cues, and overall user satisfaction.
Beyond Looks: A Study on Agent Movement and Audiovisual Spatial Coherence in Augmented Reality (Conference: P1781)
Stephanie Arevalo Arboleda, Ilmenau University of Technology; Christian Kunert, Technische Universität Ilmenau; Jakob Hartbrich, Technische Universität Ilmenau; Christian Schneiderwind, Technische Universität Ilmenau; Chenyao Diao, Technische Universität Ilmenau; Christoph Gerhardt, Technische Universität Ilmenau; Tatiana Surdu, Technische Universität Ilmenau; Florian Weidner, Lancaster University; Wolfgang Broll, Ilmenau University of Technology; Stephan Werner, Technische Universität Ilmenau; Alexander Raake, Technische Universität Ilmenau
The interplay of virtual humans' rendering style, movements, and associated sounds in real-world interactions, particularly in Augmented Reality, is yet to be explored. In this paper, we investigate the influence of three distinct movement patterns (circle, diagonal, and standing), two rendering styles (realistic and cartoon), and two types of audio (spatial audio and non-spatial audio) on emotional responses, social presence, appearance and behavior plausibility, audiovisual coherence and auditory plausibility in a study (N=36) where participants observed an agent reciting a short story. Our results indicate movement and rendering style influence the collected measures and point towards a minor effect of audio rendering.
Animatable Virtual Humans: Learning pose-dependent human representations in UV space for interactive performance synthesis (Journal: P1989)
Wieland Morgenstern, Fraunhofer HHI; Milena Bagdasarian, Fraunhofer HHI; Anna Hilsmann, Fraunhofer HHI; Peter Eisert, Fraunhofer HHI
We propose a novel representation of virtual humans for highly realistic real-time animation and rendering in 3D applications. We learn pose dependent appearance and geometry from highly accurate dynamic mesh sequences obtained from state-of-the-art multiview-video reconstruction. We learn the difference between the observed geometry and the fitted SMPL model, encoding both appearance and geometry in the consistent UV space of the SMPL model. This approach not only ensures a high level of realism but also facilitates streamlined processing and rendering of virtual humans in real-time scenarios.
Session: Locomotion and Navigation (TH4J)
Date & Time: Thursday, 21 March 2024, 13:30-15:00 (Orlando, Florida USA UTC-4)
Room: Fantasia Ballroom J
Session Chair: Jerald Thomas
A Novel Approach for Virtual Locomotion Gesture Classification: Self-Teaching Vision Transformer for a Carpet-Type Tactile Sensor (Conference: P1135)
Sung-Ha Lee, Gwangju Institute of Science and Technology; Ho-Taek Joo, (GIST) Gwangju Institute of Science and Technology; Insik Chung, School of Integrated Technology; Donghyeok Park, None; Yunho Choi, Gwang-ju institute of science and technology; KyungJoong Kim, GIST
Locomotion gesture classification in virtual reality is the process of analyzing and identifying specific user movements in the real world to navigate virtual environments. In this paper, we utilize a high-resolution carpet-type tactile sensor as a foot action recognition interface. This interface can capture the user's foot pressure data in detail to distinguish similar actions. To efficiently process the captured user's detailed foot tactile data and classify nuanced actions, we propose a novel self-teaching vision transformer (STViT) model integrating elements of the shifted window vision transformer (SwinViT) and data-efficient image transformer (DeiT). However, unlike DeiT, our model uses itself from N-steps prior as the teacher model.
Try This for Size: Multi-Scale Teleportation in Immersive Virtual Reality (Journal: P1016)
Tim Weissker, RWTH Aachen University; Matthis Franzgrote, RWTH Aachen University; Torsten Wolfgang Kuhlen, RWTH Aachen University
We present three novel teleportation-based techniques that enable users to adjust their own scale while traveling through virtual environments. Our approaches build on the extension of known teleportation workflows and suggest specifying scale adjustments either simultaneously with, as a connected second step after, or separately from the user's new horizontal position. The results of a two-part user study with 30 participants indicate that the simultaneous and connected specification paradigms are both suitable candidates for effective and comfortable multi-scale teleportation with nuanced individual benefits. Scale specification as a separate mode, on the other hand, was considered less beneficial.
MagicMap: Enhancing Indoor Navigation Experience in VR Museums (Conference: P1551)
Xueqi Wang, Xi'an Jiaotong-Liverpool University; Yue Li, Xi'an Jiaotong-Liverpool University; Hai-Ning Liang, Xi'an Jiaotong-Liverpool University
We present MagicMap, a novel navigation system for VR museums. It features a museum WiM attached to a 2D mini-map, supporting scalable map navigation and personal meaning-making through the annotations of museum artifacts and the recording of visiting traces. Translating the curatorial principles of museum visiting into a hierarchical menu layout, MagicMap demonstrates the ability to support prolonged engagement in VR museums and reduces users’ perceived workload in wayfinding. Our findings have implications for the future design of navigation systems in VR museums and complex indoor environments.
Locomotion Techniques for Dynamic Environments: Effects on Spatial Knowledge and User Experiences (Journal: P1870)
Hyunjeong Kim, Yonsei University; Sang-Bin Jeon, Yonsei University; In-Kwon Lee, Yonsei University
Various locomotion techniques provides different experiences and performances to users in virtual environments (VE). In this study, we compare the effects of different locomotion techniques (joystick, teleportation, and redirected walking (RDW)) on the user's spatial knowledge and experience, depending on whether the virtual objects are moving or not. The results showed that the differences in spatial knowledge and user experience provided by different locomotion techniques can vary depending on whether the environment is static or dynamic. Our results also showed that for a given VE, there are different locomotion techniques that induce fewer collisions, or reduce the time it takes the user to perform a given task.
Redirection Strategy Switching: Selective Redirection Controller for Dynamic Environment Adaptation (Journal: P1873)
Ho Jung Lee, Yonsei University; Sang-Bin Jeon, Yonsei University; Yong-Hun Cho, Korea University; In-Kwon Lee, Yonsei University
Selective Redirection Controller (SRC) is a novel approach to Redirected Walking (RDW) that dynamically switches between four redirection controllers (S2C, TAPF, ARC, SRL) based on the user's environment. Unlike traditional methods, SRC, trained by reinforcement learning, allows real-time controller switching to optimize the RDW experience. It's been evaluated through simulations and user studies, demonstrating a significant reduction in resets compared to conventional controllers. SRC's decision making is analysed using heat map visualization, allowing it to effectively exploit the advantages of each strategy. The result is a more immersive and seamless RDW experience that showcases SRC's innovative, contextual design.
Spatial Contraction Based on Velocity Variation for Natural Walking in Virtual Reality (Journal: P2112)
Sen-Zhe Xu, Tsinghua University; Kui Huang, Tsinghua University; Cheng-Wei Fan, Tsinghua University; Song-Hai Zhang, Tsinghua University
Virtual Reality (VR) offers an immersive 3D digital environment, but enabling natural walking sensations without the constraints of physical space remains a technological challenge. This paper introduces "Spatial Contraction (SC)", an innovative VR locomotion method inspired by the phenomenon of Lorentz contraction in Special Relativity. Similar to the Lorentz contraction, our SC contracts the virtual space along the user's velocity direction in response to velocity variation. The virtual space contracts more when the user's speed is high, whereas minimal or no contraction happens at low speeds. We provide a virtual space transformation method for spatial contraction and optimize the user experience in smoothness and stability. Through SC, VR users can effectively traverse a longer virtual distance with a shorter physical walking. Different from locomotion gains, there is no inconsistency between the user's proprioception and visual perception in SC. SC is a general locomotion method that has no special requirements for VR scenes. The experimental results of live user studies in various virtual scenarios demonstrate that SC has a significant effect in reducing both the number of resets and the physical walking distance users need to cover. Furthermore, experiments have also demonstrated that SC has the potential for integration with the translation gain.
Conference Sponsors
Doctoral Consortium Sponsors
Supporters
Code of Conduct
©IEEE VR Conference 2024, Sponsored by the IEEE Computer Society and the Visualization and Graphics Technical Committee