IEEE VR 2022 Conference Awards
| Journal Papers |
|---|
| TVCG - Best Journal Papers |
| TVCG - Honorable Mentions |
| TVCG - Best Journal - Nominees |
| Conference Papers |
|---|
| Best Conference Papers |
| Honorable Mentions |
| Nominees |
| Posters |
|---|
| Best Posters |
| Honorable Mentions |
| Nominees |
| Demos |
|---|
| Best Demo |
| Honorable Mentions |
| 3DUI Contest |
|---|
| Best 3DUI Contest Entry |
| Best 3DUI Contest Entry Nominees |
| Doctoral Consortium |
|---|
| Best Doctoral Consortium Student Award |
| Ready Player 22 |
|---|
| Winner |
TVCG - Best Journal Papers
VirtualCube: An Immersive 3D Video Communication System
Yizhong Zhang, Jiaolong Yang, Zhen Liu, Ruicheng Wang, Guojun Chen, Xin Tong, Baining Guo
The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies. The physical setup of VirtualCube is a standardized cubicle installed with off-the-shelf hardware including 3 TV displays and 6 RGBD cameras. With high-quality 3D capturing and rendering algorithm, the system teleports the remote participants into a virtual meeting room to achieve immersive in-person meeting experience with correct eye gaze. A set of VirtualCubes can be easily assembled into a V-Cube Assembly to model different video communication and shared workspace scenarios, as if all participants were in the same room.
Video See-Through Mixed Reality with Focus Cues
Christoph Ebner, Shohei Mori, Peter Mohr, Yifan (Evan) Peng, Dieter Schmalstieg, Gordon Wetzstein, Denis Kalkofen
We introduce the first approach to video see-through mixed reality with support for focus cues. By combining the flexibility to adjust the focus distance found in varifocal designs with the robustness to eye-tracking error of multifocal designs, our novel display architecture delivers focus cues over large workspaces. In particular, we introduce gaze-contingent layered displays and mixed reality focal stacks, an efficient representation of mixed reality content that lends itself to fast processing for driving layered displays in real time. We evaluate this approach by building an end-to-end pipeline for capture, render, and display of focus cues in video see-through displays.
ScanGAN360: A Generative Model of Realistic Scanpaths for 360° Images
Daniel Martin, Ana Serrano, Alexander William Bergman, Gordon Wetzstein, Belen Masia
We present ScanGAN360, a new generative adversarial approach to address this problem. We propose a novel loss function based on dynamic time warping and tailor our network to the specifics of 360° images. The quality of our generated scanpaths outperforms competing approaches by a large margin, and is almost on par with the human baseline. ScanGAN360 allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior, facilitating experimentation, and aiding novel applications in virtual reality and beyond.
TVCG - Honorable Mentions
Omnidirectional Galvanic Vestibular Stimulation in Virtual Reality
Colin Groth, Jan-Philipp Tauscher, Nikkel Heesen, Max Hattenbach, Susana Castillo, Marcus Magnor
Cybersickness often taints virtual experiences. Its source can be associated to the perceptual mismatch happening when our eyes tell us we are moving while we are, in fact, at rest. We reconcile the signals from our senses by using omnidirectional galvanic vestibular stimulation (GVS), stimulating the vestibular canals behind our ears with low-current electrical signals specifically attuned to the visually displayed camera motion. We describe how to calibrate and generate the appropriate GVS signals in real-time for pre-recorded omnidirectional videos exhibiting ego-motion in all three spatial directions, and prove that it significantly reduces discomfort for cybersickness-susceptible VR users.
Online Projector Deblurring Using a Convolutional Neural Network
Yuta Kageyama, Daisuke Iwai, Kosuke Sato
Projector deblurring is an important technology for dynamic projection mapping (PM), where the distance between a projector and a projection surface changes in time. However, conventional deblurring techniques do not support dynamic PM because they need to project calibration patterns to estimate the amount of defocus blur each time the surface moves. We present a deep neural network that can compensate for defocus blur in dynamic PM without projecting calibration patterns. We also propose a pseudo-projection technique for synthesizing physically plausible training data. Both simulation and physical PM experiments showed that our technique alleviated the defocus blur in dynamic PM.
TVCG - Best Journal Paper Nominees
Breaking Plausibility Without Breaking Presence - Evidence For The Multi-Layer Nature Of Plausibility
Larissa Brübach*, Franziska Westermeier*, Carolin Wienrich, Marc Erich Latoschik (* = equal contribution)
Recently, a novel theoretical model introduced coherence and plausibility as the essential conditions of XR experiences. Plausibility results from multi-layer (cognitive, perceptual, and sensational) coherence activation. We utilized breaks in plausibility (analogous to breaks in presence) by introducing incoherence on the perceptual and cognitive layer. A simulation of gravity-defying objects, i.e., the perceptual manipulation, broke plausibility, however, not presence. Simultaneously, the cognitive manipulation, presented as storyline framing, was too weak to counteract the strong bottom-up inconsistencies. Both results confirm the predictions of the novel model, incorporating well-known top-down and bottom-up rivalries and a theorized increased independence between plausibility and presence.
Comparing Direct and Indirect Methods of Audio Quality Evaluation in Virtual Reality Scenes of Varying Complexity
Thomas Robotham, Olli S. Rummukainen, Miriam Kurz, Marie Eckert, Emanuël A. P. Habets
This study uses four subjective audio quality evaluation methods (viz. multiple-stimulus with and without reference for direct scaling, and rank-order elimination and pairwise comparison for indirect scaling) to investigate the contributing factors present in multi-modal 6-DoF VR on quality ratings of real-time audio rendering. Five scenes were designed for evaluation with various amounts of user interactivity and complexity. Our results show rank-order elimination proved to be the fastest method, required the least amount of repetitive motion, and yielded the highest discrimination between spatial conditions. Ratings across scenes indicate complex scenes and interactive aspects of 6-DoF VR can impede quality judgments.
Omnidirectional Galvanic Vestibular Stimulation in Virtual Reality
Colin Groth, Jan-Philipp Tauscher, Nikkel Heesen, Max Hattenbach, Susana Castillo, Marcus Magnor
Cybersickness often taints virtual experiences. Its source can be associated to the perceptual mismatch happening when our eyes tell us we are moving while we are, in fact, at rest. We reconcile the signals from our senses by using omnidirectional galvanic vestibular stimulation (GVS), stimulating the vestibular canals behind our ears with low-current electrical signals specifically attuned to the visually displayed camera motion. We describe how to calibrate and generate the appropriate GVS signals in real-time for pre-recorded omnidirectional videos exhibiting ego-motion in all three spatial directions, and prove that it significantly reduces discomfort for cybersickness-susceptible VR users.
Studying the Effects of Congruence of Auditory and Visual Stimuli on Virtual Reality Experiences
Hayeon Kim, In-Kwon Lee
This paper explores how the congruence between auditory and visual (AV) stimuli, which are the sensory stimuli typically provided by VR devices. We defined the types of (in)congruence between AV stimuli, and then designed 12 virtual spaces with different degrees of congruence between AV stimuli with evaluating user experience changes. We observed the following key findings: 1) there is a limit to the degree of temporal or spatial incongruence that can be tolerated; 2) users are tolerant of semantic incongruence; 3) a simulation that considers synesthetic congruence contributes to the user's sense of immersion and presence.
Online Projector Deblurring Using a Convolutional Neural Network
Yuta Kageyama, Daisuke Iwai, Kosuke Sato
Projector deblurring is an important technology for dynamic projection mapping (PM), where the distance between a projector and a projection surface changes in time. However, conventional deblurring techniques do not support dynamic PM because they need to project calibration patterns to estimate the amount of defocus blur each time the surface moves. We present a deep neural network that can compensate for defocus blur in dynamic PM without projecting calibration patterns. We also propose a pseudo-projection technique for synthesizing physically plausible training data. Both simulation and physical PM experiments showed that our technique alleviated the defocus blur in dynamic PM.
The One-Man-Crowd: Single User Generation of Crowd Motions Using Virtual Reality
Tairan Yin, Ludovic Hoyet, Marc Christie, Marie-Paule R. Cani, Julien Pettré
Crowd motion data is fundamental for understanding and simulating crowd behaviours. Such data is usually collected through controlled experiments and is scarce due to difficulties involved in its gathering. In this work, we propose a novel Virtual Reality based approach for the acquisition of crowd motion data, which immerses a single user in virtual scenarios to act each crowd member. We validate our approach by replicating three real experiments, and compare the results. Using our approach, realistic collective behaviours can naturally emerge, even though with lower behavioural variety. These results provide valuable insights to virtual crowd experiences, and reveal key directions for further improvements.
ScanGAN360: A Generative Model of Realistic Scanpaths for 360° Images
Daniel Martin, Ana Serrano, Alexander William Bergman, Gordon Wetzstein, Belen Masia
We present ScanGAN360, a new generative adversarial approach to address this problem. We propose a novel loss function based on dynamic time warping and tailor our network to the specifics of 360° images. The quality of our generated scanpaths outperforms competing approaches by a large margin, and is almost on par with the human baseline. ScanGAN360 allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior, facilitating experimentation, and aiding novel applications in virtual reality and beyond.
Mood-Driven Colorization of Virtual Indoor Scenes
Michael S Solah, Haikun Huang, Jiachuan Sheng, Tian Feng, Marc Pomplun, Lap-Fai Yu
A challenging task in virtual scene design for Virtual Reality (VR) is invoking particular moods in viewers. The subjective nature of moods brings uncertainty to this purpose. We propose a novel approach for automatic color adjustment of textures for objects in virtual indoor scenes, enabling them to match target moods. A dataset of 25,000 indoor environment images was used to train a classifier with features extracted via deep learning. We use an optimization process that colorizes virtual scenes automatically according to the target mood. Our approach was tested on four indoor scenes used in user studies with a VR headset.
Video See-Through Mixed Reality with Focus Cues
Christoph Ebner, Shohei Mori, Peter Mohr, Yifan (Evan) Peng, Dieter Schmalstieg, Gordon Wetzstein, Denis Kalkofen
We introduce the first approach to video see-through mixed reality with support for focus cues. By combining the flexibility to adjust the focus distance found in varifocal designs with the robustness to eye-tracking error of multifocal designs, our novel display architecture delivers focus cues over large workspaces. In particular, we introduce gaze-contingent layered displays and mixed reality focal stacks, an efficient representation of mixed reality content that lends itself to fast processing for driving layered displays in real time. We evaluate this approach by building an end-to-end pipeline for capture, render, and display of focus cues in video see-through displays.
VirtualCube: An Immersive 3D Video Communication System
Yizhong Zhang, Jiaolong Yang, Zhen Liu, Ruicheng Wang, Guojun Chen, Xin Tong, Baining Guo
The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies. The physical setup of VirtualCube is a standardized cubicle installed with off-the-shelf hardware including 3 TV displays and 6 RGBD cameras. With high-quality 3D capturing and rendering algorithm, the system teleports the remote participants into a virtual meeting room to achieve immersive in-person meeting experience with correct eye gaze. A set of VirtualCubes can be easily assembled into a V-Cube Assembly to model different video communication and shared workspace scenarios, as if all participants were in the same room.
Best Conference Papers
The Chaotic Behavior of Redirection - Revisiting Simulations in Redirected Walking
Christian Hirt, Yves Kompis, Christian Holz, Andreas Kunz
Redirected Walking is a common technique to allow real walking for exploring large virtual environments in constrained physical spaces. Many existing approaches were evaluated in simulation only, and researchers argued that the findings would translate to real scenarios to motivate the effectiveness of their algorithms. In this paper, we argue that simulation-based evaluations require critical reflection. We demonstrate simulations that show the chaotic process fundamental to RDW, in which altering the initial user's position by mere millimeters can drastically change the resulting steering behavior. This insight suggests that redirection is more sensitive to underlying data than previously assumed.
Investigating how speech and motion realism influence the perceived personality of virtual characters and agents.
Sean Thomas, Ylva Ferstl, Rachel McDonnell, Cathy Ennis
The portrayed personality of virtual characters and agents influences how we perceive and engage with digital applications. Understanding the influence of speech and animation allows us to design more personalized and engaging characters. Using performance capture data from multiple datasets, we contrast performance-driven characters to those portrayed by generated gestures and synthesized speech, analysing how the features of each influence personality according to the Big Five personality traits. Our results highlight motion as dominant for portraying extraversion and speech for communicating agreeableness and emotional stability, supporting the development of virtual characters, social agents and 3DUI agents with targeted personalities.
Body Warping Versus Change Blindness Remapping: A Comparison of Two Approaches to Repurposing Haptic Proxies for Virtual Reality
Cristian Patras, Mantas Cibulskis, Niels Christian Nilsson
This paper details a study comparing two techniques for repurposing haptic proxies; namely haptic retargeting based on body warping and change blindness remapping. Participants performed a simple button-pressing task, and 24 virtual buttons were mapped onto four haptic proxies with varying degrees of misalignment. Body warping and change blindness remapping were used to realign the real and virtual buttons, and the results indicate that users failed to reliably detect realignment of up to 7.9 cm for body warping and up to 9.7 cm for change blindness remapping. Moreover, change blindness remapping yielded significantly higher self-reported agency, and marginally higher ownership.
Conference Papers - Honorable Mentions
Combining Real-World Constraints on User Behavior with Deep Neural Networks for Virtual Reality (VR) Biometrics
Robert Miller, Natasha Kholgade Banerjee, Sean Banerjee
Deep networks demonstrate enormous potential for VR security using behavioral biometrics. Existing datasets are small, and make automated learning of real-world behavior features using deep networks challenging. We incorporate real-world constraints such as spatial relationships between devices in the form of displacement vectors and trajectory smoothing in input data for deep networks performing behavior-based identification and authentication. We evaluate our approach against baseline methods that use raw data directly and that perform global normalization. Using displacement vectors, we show higher success over baseline methods in 36 out of 42 cases of varying user sets, VR systems, and sessions.
RedirectedDoors: Redirection While Opening Doors in Virtual Reality
Yukai Hoshikawa, Kazuyuki Fujita, Kazuki Takashima, Morten Fjeld, Yoshifumi Kitamura
We propose RedirectedDoors, a novel technique for redirection in VR focused on door-opening behavior. This technique manipulates the user's walking direction by rotating the entire virtual environment at a certain angular ratio of the door being opened, while the virtual door's position is kept unmanipulated to ensure door-opening realism. Results of a user study using two types of door-opening interfaces (with and without a passive haptic prop) revealed that the estimated detection thresholds generally showed a higher space efficiency of redirection. Following the results, we derived usage guidelines for our technique that provide lower noticeability and higher acceptability.
Real-Time Gaze Tracking with Event-Driven Eye Segmentation
Yu Feng, Nathan Goulding-Hotta, Asif Khan, Hans Reyserhove, Yuhao Zhu
Gaze tracking is an essential component in X-Reality. Modern gaze tracking algorithms are heavyweight; they operate at most 5 Hz on mobile processors despite that cameras can operate at a real-time rate (> 30 Hz). This paper presents a real-time eye tracking algorithm that can operate at 30 Hz on a mobile processor, achieves 0.1°-0.5° gaze accuracies, all the while requiring one to two orders of magnitude smaller parameters than state-of-the-art eye tracking algorithms. The key is an Auto ROI mode, which continuously predicts and processes only the Regions of Interest (ROIs) of near-eye images. In particular, we discuss how to accurately predict ROI by emulating an event camera without requiring special hardware support.
ENI: Quantifying Environment Compatibility for Natural Walking in Virtual Reality
Niall L. Williams, Aniket Bera, Dinesh Manocha
We present a metric to measure the similarity between physical and virtual environments for natural walking in VR. We use geometric techniques based on visibility polygons to compute the Environment Navigation Incompatibility (ENI) metric to measure the complexity VR locomotion. ENI is useful for highlighting regions of incompatibility and guiding the design of the virtual environments to make them more compatible. User studies and simulations show that ENI identifies environments where users are able to walk larger distances before colliding with objects. ENI is the first general metric that automatically quantifies environment navigability for VR locomotion. Project page: gamma.umd.edu/eni
Conference Papers - Nominees
Body Warping Versus Change Blindness Remapping: A Comparison of Two Approaches to Repurposing Haptic Proxies for Virtual Reality
Cristian Patras, Mantas Cibulskis, Niels Christian Nilsson
This paper details a study comparing two techniques for repurposing haptic proxies; namely haptic retargeting based on body warping and change blindness remapping. Participants performed a simple button-pressing task, and 24 virtual buttons were mapped onto four haptic proxies with varying degrees of misalignment. Body warping and change blindness remapping were used to realign the real and virtual buttons, and the results indicate that users failed to reliably detect realignment of up to 7.9 cm for body warping and up to 9.7 cm for change blindness remapping. Moreover, change blindness remapping yielded significantly higher self-reported agency, and marginally higher ownership.
The Chaotic Behavior of Redirection - Revisiting Simulations in Redirected Walking
Christian Hirt, Yves Kompis, Christian Holz, Andreas Kunz
Redirected Walking is a common technique to allow real walking for exploring large virtual environments in constrained physical spaces. Many existing approaches were evaluated in simulation only, and researchers argued that the findings would translate to real scenarios to motivate the effectiveness of their algorithms. In this paper, we argue that simulation-based evaluations require critical reflection. We demonstrate simulations that show the chaotic process fundamental to RDW, in which altering the initial user's position by mere millimeters can drastically change the resulting steering behavior. This insight suggests that redirection is more sensitive to underlying data than previously assumed.
ENI: Quantifying Environment Compatibility for Natural Walking in Virtual Reality
Niall L. Williams, Aniket Bera, Dinesh Manocha
We present a metric to measure the similarity between physical and virtual environments for natural walking in VR. We use geometric techniques based on visibility polygons to compute the Environment Navigation Incompatibility (ENI) metric to measure the complexity VR locomotion. ENI is useful for highlighting regions of incompatibility and guiding the design of the virtual environments to make them more compatible. User studies and simulations show that ENI identifies environments where users are able to walk larger distances before colliding with objects. ENI is the first general metric that automatically quantifies environment navigability for VR locomotion. Project page: gamma.umd.edu/eni
Investigating how speech and motion realism influence the perceived personality of virtual characters and agents.
Sean Thomas, Ylva Ferstl, Rachel McDonnell, Cathy Ennis
The portrayed personality of virtual characters and agents influences how we perceive and engage with digital applications. Understanding the influence of speech and animation allows us to design more personalized and engaging characters. Using performance capture data from multiple datasets, we contrast performance-driven characters to those portrayed by generated gestures and synthesized speech, analysing how the features of each influence personality according to the Big Five personality traits. Our results highlight motion as dominant for portraying extraversion and speech for communicating agreeableness and emotional stability, supporting the development of virtual characters, social agents and 3DUI agents with targeted personalities.
Real-Time Gaze Tracking with Event-Driven Eye Segmentation
Yu Feng, Nathan Goulding-Hotta, Asif Khan, Hans Reyserhove, Yuhao Zhu
Gaze tracking is an essential component in X-Reality. Modern gaze tracking algorithms are heavyweight; they operate at most 5 Hz on mobile processors despite that cameras can operate at a real-time rate (> 30 Hz). This paper presents a real-time eye tracking algorithm that can operate at 30 Hz on a mobile processor, achieves 0.1°-0.5° gaze accuracies, all the while requiring one to two orders of magnitude smaller parameters than state-of-the-art eye tracking algorithms. The key is an Auto ROI mode, which continuously predicts and processes only the Regions of Interest (ROIs) of near-eye images. In particular, we discuss how to accurately predict ROI by emulating an event camera without requiring special hardware support.
RedirectedDoors: Redirection While Opening Doors in Virtual Reality
Yukai Hoshikawa, Kazuyuki Fujita, Kazuki Takashima, Morten Fjeld, Yoshifumi Kitamura
We propose RedirectedDoors, a novel technique for redirection in VR focused on door-opening behavior. This technique manipulates the user's walking direction by rotating the entire virtual environment at a certain angular ratio of the door being opened, while the virtual door's position is kept unmanipulated to ensure door-opening realism. Results of a user study using two types of door-opening interfaces (with and without a passive haptic prop) revealed that the estimated detection thresholds generally showed a higher space efficiency of redirection. Following the results, we derived usage guidelines for our technique that provide lower noticeability and higher acceptability.
Supporting Jury Understanding of Expert Evidence in a Virtual Environment
Carolin Reichherzer, Andrew Cunningham, Jason Barr, Tracey Coleman, Kurt McManus, Dion Sheppard, Scott Coussens, Mark Kohler, Mark Billinghurst, Bruce H Thomas
This work investigates the use of Virtual Reality (VR) to present forensic evidence to the jury in a courtroom trial. We performed a between-participant study comparing comprehension of an expert statement in VR to the traditional courtroom presentation of still images. We measured understanding of the expert domain, mental effort and content recall and found that VR significantly improves the understanding of spatial information and knowledge acquisition. We also identify different patterns of user behaviour depending on the display method. We conclude with suggestions on how to best adapt evidence presentation to VR.
Tangiball: Foot-Enabled Embodied Tangible Interaction with a Ball in Virtual Reality
Lila Bozgeyikli, Evren Bozgeyikli
Interaction with tangible user interfaces in virtual reality (VR) is known to offer several benefits. In this study, we explored foot-enabled embodied interaction in VR through a room-scale tangible soccer game (Tangiball). Users interacted with a physical ball with their feet in real time by seeing its virtual counterpart inside a VR head mounted display. A user study was performed with 40 participants, in which Tangiball was compared with the control condition of foot-enabled embodied interaction with a purely virtual ball. The results revealed that tangible interaction improved user performance and presence significantly, while no difference in terms of motion sickness was detected between the tangible and virtual versions.
TimeTables: Embodied Exploration of Immersive Spatio-Temporal Data
Yidan Zhang, Barrett Ens, Kadek Ananta Satriadi, Arnaud Prouzeau, Sarah Goodwin
We propose TimeTables, a novel prototype system that aims to support data exploration, using embodiment with space-time cubes in virtual reality. TimeTables uses multiple space-time cubes on virtual tabletops, which users can manipulate by extracting time layers to create new tabletop views. The system presents information at different time scales by stretching layers to drill down in time. Users can also jump into tabletops to inspect data from an egocentric perspective. From our use case scenario of energy consumption displayed on a university campus, we believe the system has a high potential in supporting spatio-temporal data exploration and analysis.
Virtual Reality Observations: Using Virtual Reality to Augment Lab-Based Shoulder Surfing Research
Florian Mathis, Joseph O'Hagan, Mohamed Khamis, Kami Vaniea
We exploit VR's unique characteristics and study the use of non-immersive/immersive VR recordings for real-world shoulder surfing research. We demonstrate the strengths of VR-based shoulder surfing research by exploring three different authentication scenarios: automated-teller-machine (ATM), smartphone PIN, and smartphone pattern authentication. Our results show that applying VR for shoulder surfing research is advantageous in many ways compared to currently used approaches (e.g., lab-based 2D video recordings). We discuss the strengths and weaknesses of using VR for shoulder surfing research and conclude with four recommendations to help researchers decide when (and when not) to employ VR for shoulder surfing research.
Virtual Workspace Positioning Techniques during Teleportation for Co-located Collaboration in Virtual Reality using HMDs
Yiran Zhang, Huyen Nguyen, Nicolas Ladeveze, Cedric Fleury, Patrick Bourdot
Many co-located collaborative Virtual Reality applications rely on a one-to-one mapping of users' relative positions in real and virtual environments. However, the users' individual virtual navigation may break this spatial configuration. This work enables the recovery of spatial consistency after individual navigation. We provide a virtual representation of the users' shared physical workspace and developed two techniques to position it. The first technique enables a single user to control the virtual workspace location, while the second allows concurrent manipulation. Experimental results show that allowing two users to co-manipulate the virtual workspace significantly reduces negotiation time.
A Keylogging Inference Attack on Air-Tapping Keyboards in Virtual Environments
ülkü Meteriz-Yildiran, Necip Fazil Yildiran, Amro Awad, David Mohaisen
Enabling users to push the physical world's limits, AR/VR opened a new chapter in perception. Novel immersive experiences resulted in novel interaction methods, which came with unprecedented security and privacy risks. This paper presents a keylogging attack to infer inputs typed with in-air tapping keyboards. We exploit the observation that hands follow specific patterns when typing in-air and build a 5-stage pipeline. Through various experiments, we showed that our attack achieves 40% - 89% accuracy. Finally, we discuss countermeasures, while the results presented provide a cautionary tale of the security and privacy risk of the immersive mobile technology.
Combining Real-World Constraints on User Behavior with Deep Neural Networks for Virtual Reality (VR) Biometrics
Robert Miller, Natasha Kholgade Banerjee, Sean Banerjee
Deep networks demonstrate enormous potential for VR security using behavioral biometrics. Existing datasets are small, and make automated learning of real-world behavior features using deep networks challenging. We incorporate real-world constraints such as spatial relationships between devices in the form of displacement vectors and trajectory smoothing in input data for deep networks performing behavior-based identification and authentication. We evaluate our approach against baseline methods that use raw data directly and that perform global normalization. Using displacement vectors, we show higher success over baseline methods in 36 out of 42 cases of varying user sets, VR systems, and sessions.
Best Poster
Studying the User Adaptability to Hyperbolic Spaces and Delay Time Scenarios
Ana Rita Rebelo: NOVA LINCS, NOVA School of Science and Technology; Rui Nóbrega: Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa; Fernando Birra: Nova School of Science and Technology
To create immersive virtual experiences, it is crucial to understand how users perceive Virtual Environments (VEs) and which interaction techniques are most appropriate for their tasks. We created a tangible VE - the VR Lab - where it is possible to study space and time conditions to analyse the user's adaptability to different forms of interaction. As a case study, we restricted the scope of the investigation to two morphing scenarios. The space morphing scenario compares the adaptability of users to Euclidean versus hyperbolic spaces. The time morphing scenario aims to establish from which values the visual delay affects performance.
On the Effectiveness of Conveying BIM Metadata in VR Design Reviews for Healthcare Architecture
Emma Buchanan: University of Canterbury; Giuseppe Loporcaro: University of Canterbury; Stephan Lukosch: University of Canterbury
This research seeks to assess whether Virtual Reality (VR) can be used to convey Building Information Modelling (BIM) metadata alongside geometric and spatial data in a virtual environment, and by doing so, determine if this increases the understanding of the design by the stakeholder. A user study assessed participants performance and preference for conducting design reviews in VR or using a traditional design review system of PDF drawings and a 3D model. Early results indicate VR was preferred with fewer errors made during assessment and a higher System Usability Scale SUS score.
Posters - Honorable Mentions
Jamming in MR: Towards Real-Time Music Collaboration in Mixed Reality
Ruben Schlagowski: University of Augsburg; Kunal Gupta: The University of Auckland; Silvan Mertes: University of Augsburg; Mark Billinghurst: University of Auckland; Susanne Metzner: University of Augsburg; Elisabeth André: University of Augsburg
Recent pandemic-related contact restrictions have made it difficult for musicians to meet in person to make music. As a result, there has been an increased demand for applications that enable remote and real-time music collaboration. One desirable goal here is to give musicians a sense of social presence, to make them feel that they are "on site" with their musical partners. We conducted a focus group study to investigate the impact of remote jamming on users' affect. Further, we gathered user requirements for a Mixed Reality system that enables real-time jamming and developed a prototype based on these findings.
Movement Augmentation in Virtual Reality: Impact on Sense of Agency Measured by Subjective Responses and Electroencephalography
Liu Wang: Xi'an Jiaotong-Liverpool University; Mengjie Huang: Xi'an Jiaotong-Liverpool University; Chengxuan Qin: Xi'an Jiaotong-Liverpool University; Yiqi Wang: University College London; Rui Yang: Xi'an Jiaotong-Liverpool University
Virtual movement augmentation, which refers to the visual amplification of remapped movement, shows potential to be applied in motion-related virtual reality programs. Sense of agency (SoA), which measures the user's feeling of control in their action, has not been fully investigated for augmented movement. This study investigated the effect of augmented movement at three different levels (baseline, medium, and high) on users' SoA using both subjective responses and electroencephalography (EEG). Results show that SoA can be boosted slightly at medium augmentation level but drops at high level. The augmented virtual movement only helps to enhance SoA to a certain extent.
Best Poster - Nominees
Augmenting VR Ski Training using Time Distortion
Takashi Matsumoto: Tokyo Institute of Technology; Erwin Wu: Tokyo Institute of Technology; Hideki Koike: Tokyo Institute of Technology
Virtual reality-based sports simulators are widely developed, which makes training in a virtual environment possible. On the other hand, methods using temporal features are also introduced to realize an adaptive training. In this paper, we study the effect of time distortion on alpine ski training to find out how modifying the temporal space can affect sports training. Experiments are conducted to investigate how a fast/slow and a static/dynamic time distortion-based training, respectively, can impact the performance of users.
Studying the User Adaptability to Hyperbolic Spaces and Delay Time Scenarios
Ana Rita Rebelo: NOVA LINCS, NOVA School of Science and Technology; Rui Nóbrega: Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa; Fernando Birra: Nova School of Science and Technology
To create immersive virtual experiences, it is crucial to understand how users perceive Virtual Environments (VEs) and which interaction techniques are most appropriate for their tasks. We created a tangible VE - the VR Lab - where it is possible to study space and time conditions to analyse the user's adaptability to different forms of interaction. As a case study, we restricted the scope of the investigation to two morphing scenarios. The space morphing scenario compares the adaptability of users to Euclidean versus hyperbolic spaces. The time morphing scenario aims to establish from which values the visual delay affects performance.
High-speed Gaze-oriented Projection by Cross-ratio-based Eye Tracking with Dual Infrared Imaging
Ayumi Matsumoto: The University of Tokyo; Tomohiro Sueishi: The University of Tokyo; Masatoshi Ishikawa: The University of Tokyo
While gaze-oriented projection can be high-resolution and wide-area display, conventional methods have difficulties in handling quick human eye movements. In this paper, we propose a high-speed gaze-oriented projection system using a synchronized high-speed tracking projector and cross-ratio-based eye tracking. The tracking projector with a high-speed projector and rotational mirrors enables temporal geometric consistency of the projection. The eye tracking uses high-speed cameras and infrared lightings of different wavelengths, and can achieve fast and almost calibration-free due to the cross-ratio algorithm. We have experimentally validated the eye tracking speed and accuracy, system latency, and demonstrated gaze-oriented projection.
Feasibility of Training Elite Athletes for Improving their Mental Imagery Ability Using Virtual Reality
Yuanjie Wu: University of Canterbury; Stephan Lukosch: University of Canterbury; Heide Lukosch: University of Canterbury; Robert W. Lindeman: University of Canterbury; Ryan Douglas McKee: University of Canterbury; Shunsuke Fukuden: HIT Lab NZ, University of Canterbury; Cameron Ross: Snow Sports NZ; Dave Collins: Snow Sports NZ
The goal of imagery training for athletes is to create realistic images in their minds and to familiarize them with certain procedures, environments, and other aspects related to competition. Traditional imagery training methods use still images or videos, and athletes study the pictures or watch the videos in order to mentally rehearse. However, factors such as distractions and low realism can affect the training quality. In this paper, we present a VR solution and a study that explores our hypotheses that 1) high-fidelity VR systems improve mental imagery skills, and that 2) the presence of elements such as an audience or photographers in the VR environment result in better mental imagery skill improvement.
AIR-range: Arranging optical systems to present mid-AIR images with continuous luminance on and above a tabletop
Tomoyo Kikuchi: The University of Tokyo; Yuchi Yahagi: The University of Tokyo; Shogo Fukushima: The University of Tokyo; Saki Sakaguchi: Tokyo Metropolitan University; Takeshi Naemura: The University of Tokyo
We propose "AIR-range"- a system that seamlessly connects mid-air images from the surface of a table to mid-air space. This system can display tall mid-air images in the three-dimensional (3D) space beyond the screen. AIR-range is implemented using a symmetrical mirror structure that displays a large image by integrating multiple imaging paths. The mirror arrangement in previous research had a problem in that the luminance was discontinuous. In this study, we theorize the relationship between the parameters of optical elements and the appearance of mid-air images and optimize an optical system to minimize the difference in luminance between image paths.
Holding Hands for Short-Term Group Navigation in Social Virtual Reality
Tim Weissker: Bauhaus-Universitaet Weimar; Pauline Bimberg: Bauhaus-Universitaet Weimar; Ankith Kodanda: Bauhaus-Universitaet Weimar; Bernd Froehlich: Bauhaus-Universität Weimar
Prior research has shown that social interactions in VR benefit from techniques for group navigation that bring multiple users to a common destination together. In this work, we propose the metaphor of holding onto another user's virtual hand for the ad-hoc formation of a navigational group and report on the positive results of an initial usability study in an exploratory two-user scenario.
Automatic 3D Avatar Generation from a Single RBG Frontal Image
Alejandro Beacco: Universitat de Barcelona; Jaime Gallego: University of Barcelona; Mel Slater: University of Barcelona
We present a complete automatic system to obtain a realistic 3D avatar reconstruction of a person using only a frontal RGB image. Our proposed workflow first determines the pose, shape and semantic information from the input image. All this information is processed to create the skeleton and the 3D skinned textured mesh that conform the final avatar. We use a specific head reconstruction method to correctly match our final mesh to a realistic avatar. Our pipeline focuses on three main aspects: automation of the process, identification of the person, and usability of the avatar.
On the Effectiveness of Conveying BIM Metadata in VR Design Reviews for Healthcare Architecture
Emma Buchanan: University of Canterbury; Giuseppe Loporcaro: University of Canterbury; Stephan Lukosch: University of Canterbury
This research seeks to assess whether Virtual Reality (VR) can be used to convey Building Information Modelling (BIM) metadata alongside geometric and spatial data in a virtual environment, and by doing so, determine if this increases the understanding of the design by the stakeholder. A user study assessed participants performance and preference for conducting design reviews in VR or using a traditional design review system of PDF drawings and a 3D model. Early results indicate VR was preferred with fewer errors made during assessment and a higher System Usability Scale SUS score.
Movement Augmentation in Virtual Reality: Impact on Sense of Agency Measured by Subjective Responses and Electroencephalography
Liu Wang: Xi'an Jiaotong-Liverpool University; Mengjie Huang: Xi'an Jiaotong-Liverpool University; Chengxuan Qin: Xi'an Jiaotong-Liverpool University; Yiqi Wang: University College London; Rui Yang: Xi'an Jiaotong-Liverpool University
Virtual movement augmentation, which refers to the visual amplification of remapped movement, shows potential to be applied in motion-related virtual reality programs. Sense of agency (SoA), which measures the user's feeling of control in their action, has not been fully investigated for augmented movement. This study investigated the effect of augmented movement at three different levels (baseline, medium, and high) on users' SoA using both subjective responses and electroencephalography (EEG). Results show that SoA can be boosted slightly at medium augmentation level but drops at high level. The augmented virtual movement only helps to enhance SoA to a certain extent.
Jamming in MR: Towards Real-Time Music Collaboration in Mixed Reality
Ruben Schlagowski: University of Augsburg; Kunal Gupta: The University of Auckland; Silvan Mertes: University of Augsburg; Mark Billinghurst: University of Auckland; Susanne Metzner: University of Augsburg; Elisabeth André: University of Augsburg
Recent pandemic-related contact restrictions have made it difficult for musicians to meet in person to make music. As a result, there has been an increased demand for applications that enable remote and real-time music collaboration. One desirable goal here is to give musicians a sense of social presence, to make them feel that they are "on site" with their musical partners. We conducted a focus group study to investigate the impact of remote jamming on users' affect. Further, we gathered user requirements for a Mixed Reality system that enables real-time jamming and developed a prototype based on these findings.
HoloCMDS: Investigating Around Field of View Glanceable Commands Selection in AR-HMDs
Rajkumar Darbar: INRIA Bordeaux; Arnaud Prouzeau: Inria; Martin HACHET: Inria
Augmented reality merges the real and virtual worlds seamlessly in real-time. However, we need contextual menus to manipulate virtual objects rendered in our physical space. Unfortunately, designing a menu for augmented reality head-mounted displays (AR-HMDs) is challenging because of their limited display field of view (FOV). In this paper, we propose HoloCMDS to support quick access of contextual commands in AR-HMDs and conduct an initial experiment to get users' feedback about this technique.
Best Demo
Pixel Processor Arrays For Low Latency Gaze Estimation
Laurie Bose, Piotr Dudek, Stephen Carey, Jianing Chen
We demonstrate gaze tracking at over 10,000 Hz, with processing latency below 0.1 ms via use of a Pixel Processor Array (PPA) vision sensor. The PPA allows visual data to be processed efficiently at the point of light capture. By extracting features used for gaze tracking upon the PPA, we reduce data transfer from sensor to processing from entire images to a hand-full of contextual bytes, saving significant power, time and allowing for frame-rates far exceeding traditional camera sensors.
Demos - Honorable Mentions
Mid-air Haptic Texture Exploration in VR
Orestis Georgiou, Jonatan Martinez, Abdenaceur Abdouni, Adam Harwood
Mid-air haptic feedback has traditionally been used to enhance gesture input interactions. Here we present a VR research demo that expands such interactions to include for active haptic exploration. In the demo, the user can explore a virtual object using both hands and feel its intrinsic properties such as consistency and texture. The paper describes the physical apparatus used, the haptic rendering techniques leveraged, and the demo's relevance to applications such as VR shopping.
Demonstrating Immersive Gesture Exploration with GestureExplorer
Ang Li, Jiazhou Liu, Maxime Cordeil, Barrett Ens
We demonstrate GestureExplorer, which features versatile immersive visualisations to grant the user free control over their perspective, allowing them to gain a better understanding of gestures. It provides multiple data visualisation views, and interactive features to support analysis and exploration of gesture data sets. This demonstration shows the potential of GestureExplorer for providing a useful and engaging experience for exploring gesture data.
3DUI Contest - Best 3DUI Contest Entry
Clean the Ocean: An Immersive VR Experience Proposing New Modifications to Go-Go and WiM Techniques
Lee Lisle: Virginia Tech; Feiyu Lu: Virginia Tech; Shakiba Davari: Virginia Tech; Ibrahim Asadullah Tahmid: Virginia Tech; Alexander Giovannelli: Virginia Tech; Cory Ilo: Virginia Tech; Leonardo Pavanatto: Virginia Tech; Lei Zhang: Virginia Tech; Luke Schlueter: Virginia Tech; Doug Bowman: Virginia Tech
In this paper we present our solution to the 2022 3DUI Contest challenge. We aim to provide an immersive VR experience to increase player's awareness of trash pollution in the ocean while improving the current interaction techniques in virtual environments. To achieve these objectives, we adapted two classic interaction techniques, Go-Go and World in Miniature (WiM), to provide an engaging minigame in which the user collects the trash in the ocean. To improve precision, occlusion, and object retrieval issues in these traditional techniques, we present both ReX Go-Go and Rabbit-Out-of-the-Hat WiM.
3DUI Contest - Nominees
The Rubber Slider Metaphor: Visualisation of Temporal and Geolocated Data
Antonin Cheymol: Univ Rennes, INSA Rennes, Inria, CNRS, IRISA; Gwendal Fouché: Univ Rennes, Inria, INSA Rennes, CNRS, IRISA; Lysa Gramoli: Univ Rennes, INSA Rennes, Inria, CNRS, IRISA; Yutaro Hirao: The University of Tokyo; Emilie Hummel: Univ Rennes, INSA Rennes, Inria, CNRS, IRISA; Maé Mavromatis: Univ Rennes, INSA Rennes, Inria, CNRS, IRISA; Yann Moullec: Univ Rennes, Inria, CNRS, IRISA; Ferran Argelaguet Sanz: Inria, Univ Rennes, CNRS, IRISA; Florian Nouviale: Univ Rennes, INSA Rennes, Inria, CNRS, IRISA
In the context of the IEEE VR 2022 3DUI Contest entitled "Arts, Science, Information, and Knowledge - Visualized and Interacted", this paper presents a VR application to highlight the usage of the rubber slider metaphor. The rubber slider is an augmentation of usual 2D slider controls where users can bend the slider axis to control an additional degree of freedom value in the application. This demonstration immerses users in a Virtual Environment where they will be able to explore a database of art pieces geolocated on a 3D earth model and their corresponding art movements displayed on a timeline interface.
Clean the Ocean: An Immersive VR Experience Proposing New Modifications to Go-Go and WiM Techniques
Lee Lisle: Virginia Tech; Feiyu Lu: Virginia Tech; Shakiba Davari: Virginia Tech; Ibrahim Asadullah Tahmid: Virginia Tech; Alexander Giovannelli: Virginia Tech; Cory Ilo: Virginia Tech; Leonardo Pavanatto: Virginia Tech; Lei Zhang: Virginia Tech; Luke Schlueter: Virginia Tech; Doug Bowman: Virginia Tech
In this paper we present our solution to the 2022 3DUI Contest challenge. We aim to provide an immersive VR experience to increase player's awareness of trash pollution in the ocean while improving the current interaction techniques in virtual environments. To achieve these objectives, we adapted two classic interaction techniques, Go-Go and World in Miniature (WiM), to provide an engaging minigame in which the user collects the trash in the ocean. To improve precision, occlusion, and object retrieval issues in these traditional techniques, we present both ReX Go-Go and Rabbit-Out-of-the-Hat WiM.
Heart-In-Hand, a swapping point of view for immersive navigation in medical cardiology
Carlos Javier Latorre Rojas: Universidad Militar Nueva Granada; Alexander Rozo-Torres: Universidad Militar Nueva Granada; Laura Cortes-Rico: Universidad Militar Nueva Granada; Wilson J. Sarmiento: Universidad Militar Nueva Granada
This work shows an interaction technique that allows the user to swap egocentric-exocentric views while navigating inside the human heart. This proposal furnishes the user with a natural interaction supported by a tangible heart that the user may manipulate in an exocentric view and a set of manual gestures that provide locomotion actions in an egocentric view. Two classic 3D interaction techniques inspire this work: World In Miniature and Voodoo Dolls. The preliminary tests showed that users take advantage of the first-person navigation and use the third-person view to get a global visual point without missing the anatomic location.
Doctoral Consortium - Best Presentation
Designing Immersive Tools for Supporting Cognition in Remote Scientific Collaboration
Monsurat Olaosebikan
Tufts University
Ready Player 22 - Winner
Xiaodan Hu
Nara Institute of Science and Technology, Ikoma, Japan
Conference Sponsors
Supporters
Doctoral Consortium Sponsors
