3DUI Contest 2019

The 10th annual IEEE 3DUI Contest focuses on the development of 3D User Interfaces (3DUIs) using Passive Haptics feedback in Mixed Reality Environments. Material properties of real objects are enhanced with virtual behaviors while used for interaction in a mixed reality scenario. The contest was open to anyone interested in 3DUIs, from researchers to students, enthusiasts, and professionals. The purpose of the contest is to stimulate innovative and creative solutions to challenging 3DUI problems.

The following submissions will be presented March 25 and 26. All videos can be seen at here

(Team VitaZ) Gamified Mixed Reality Multisensorial Interactions

Team Members: D. Bitzas, S. Zouras, A. Chrysanthakopoulou, D. Laskos, K. Kalatzis, M. Pavlou, I. Balasi and K. Moustakas

Electrical and Computer Engineering Department, University of Patras, Rion-Patras, Greece.

Abstract: This paper presents multiple Mixed Reality 3D interaction, manipulation and simulation techniques in the context of the 2019 3DUI contest of the IEEE VR conference. The proposed schemes provide smart, seamless transition from the real to the virtual world and demonstrate passive haptics, mid-air haptics, object manipulation and abstract entities (time) manipulation. All techniques are integrated in the context of a mixed reality escape-room or treasure-hunt game, where information from both the real and the virtual world is necessary to solve the puzzle. The paper concludes with a discussion on the extensibility and translational application of the approaches in practical problem solving.

(Team VR@POLITO) Building Reconfigurable Passive Haptic Interfaces On Demand Using Off-the-shelf Construction Bricks

Team Members: Davide Calandra, F. Gabriele Prattico, Alberto Cannavo, Luca Micelli, Fabrizio Lamberti

Politecnico di Torino, Dipartimento di Automatica e Informatica, Corso Duca degli Abruzzi, 24, 10129 Torino, Italy.

Abstract: Although passive active interfaces have been shown to be capable to enhance the user’s sense of presence in Mixed Reality experiences, their use is still constrained by the need to rely on exact replicas of virtual objects or on custom-made devices mimicking the original ones. Unfortunately, the former are not flexible enough in terms of reconfigurability, whereas the latter may be difficult to reproduce. To tackle these issues, this paper explores the possibility to build passive haptic interfaces using off-the-shelf toy construction bricks. Bricks can be assembled to provide the intended feedback in more than one task. Moreover, they may be reassembled in another application to mimic completely new objects and support totally different tasks.

(Team Caverna Digital) Batmen X The Puzzler - escaping AR’s drawbacks with augmented virtuality and low cost sensors

Team Members: André Montes Rodrigues, Mário Nagamura, Luis Gustavo Freire da Costa, Rodrigo Rodrigues Gesuatto Faria, Pier Luigi Nakai Ricchetti, Eric Nozomi Tatsuta, Roseli de Deus Lopes, Marcelo Knorich Zuffo

Interdisciplinary Center in Interactive Technologies - PSI - Escola Politécnica - Universidade de São Paulo

Abstract: We present an augmented virtuality system using low-cost or outdated devices as sensors, attached to tangible objects. Five interactions were implemented, featuring rotation, pressing, pulling, pushing and insertion movements, including passive mechanical resistance. Interactions were tied together by a puzzling narrative. Concerning storytelling, in contrast to AR, pure virtual reality offers unlimited visual possibilities and higher flexibility. Augmenting VR with physical objects allows ignoring most details of the physical environment and exploring ‘magical’ interactions. Precise registration was accomplished using wireless tracking system. The main challenge was the absent physical counterparts of the scenario, which was tackled by limiting user’s reach to available interactions. Despite Leap Motion’s erratic hand tracking, mouse and tablet sensors were precise enough and the augmented environment allowed a high sense of presence.

(Team Crystal Palace) Merging Virtual Objects and Physical Hand-held Tools

*Team Members: Toshiro Kashiwagi, Kaoru Sumi, Sidney Fels, Qian Zhou, Fan Wu

Qian Zhou, Fan Wu. University of British Columbia & Kaoru Sumi. Future University Hakodate

Abstract: We developed a mixed-reality approach, called Crystal Palace, which provides prop-based passive haptic feedback for the 3DUI contest. In this system, we propose an interface of controlling virtual object with physical tools such as scissors, a spray and a screwdriver as game controllers. Our prop-based approach provides natural visual affordance and passive haptics by associating physical tools with various 3D tasks. Using 360 degrees spherical display, players will be able to use common real-world tools in an intuitive way to interact with 3D objects in a virtual escape room.

(Team Hybrid) Pyramid Escape: Design of Novel Passive Haptics Interactions for an Immersive and Modular Scenario

Team Members: Hugo Brument, Rebecca Fribourg, Gerard Gallagher, Thomas Howard, Flavien Lecuyer, Tiffany Luong, Victor Mercado, Etienne Peillard, Xavier de Tinguy, and Maud Marchal

Univ. Rennes, Inria, INSA, CNRS, IRISA, Ecole Centrale de Nantes, UMR AAU-CRENAU, IRT b<>com, France

Abstract: In this paper, we present the design of ten different 3D user interactions using passive haptics and embedded in an escape game scenario in which users have to escape from a pyramid in a limited time. Our solution is innovative by its modularity, allowing interactions with virtual objects using tangible props manipulated either directly using the hands and feet or indirectly through a single prop held in the hand, in order to perform several interactions with the virtual environment (VE). We also propose a navigation technique based on the “impossible spaces” design, allowing users to naturally walk through several overlapping rooms of the VE. All together, our different interaction techniques allow the users to solve several enigmas built into a challenging scenario inside a pyramid.

(Team Fighting Gobblers) Save the Space Elevator: An Escape Room Scenario Involving Passive Haptics in Mixed Reality

Team Members: Shakiba Davari, Yuan Li, Lee Lisle, Feiyu Lu, Lei Zhang, Leslie Blustein, Xueting Feng, Brianna Gabaldon, Marc Kwiatkowski, and Doug A. Bowman

Center for Human-Computer Interaction, Department of Computer Science, Virginia Tech

Abstract: This paper presents our solution to the 2019 3DUI Contest challenge, which focuses on passive haptics. We aimed to provide a compelling user experience in virtual reality while overcoming the limitations of physical space and current tracking devices. To meet these goals, we designed a time travel scenario that incorporates several novel features. A time machine increases efficiency in the use of physical space. A passive haptic camera prop provides a help system that is integrated into the storyline. Finally, the concept of a “temporal stabilizer” provides a plausible way to reuse a single tracking device to track multiple passive haptic props.

(Team VRAC) A Collaborative Virtual Reality Escape Room with Passive Haptics

Team Members: Austin Hanus, Mindy Hoover, Alex Lim, Jack Miller

Iowa State University

Abstract: Escape rooms have recently become a popular way to socialize and problem solve in an immersive environment. However, it can be difficult and expensive to create elaborate escape rooms with realistic props. Virtual reality (VR) technology allows developers to create and customize escape rooms more easily. However, to truly make the VR escape room immersive, physical and cooperative interactions are necessary. In this paper, the authors propose and demonstrate a two-player VR escape room developed for the HTC Vive. Physical interactions were made possible by using passive haptics in the form of simple props tracked using HTC Vive trackers and controllers. Additionally, the HTC Vives were networked, and players hands were tracked using the Leap Motion to provide head and hand position cues to teammates.

(Team NextGen Interactions) Simulating a Futuristic Fire Pump Panel in Virtual Reality

Team Members: Mohamed Suhail, Scott Gainer, Jason Haskins, Blake Boyd, Charles Laird, Will Huse, Suraj Eadara, Jason Jerald

NextGen Interactions

Abstract: Based on input and feedback from firefighters, we built a futuristic pump panel simulator using consumer VR hardware, basic electronics components, and simple physical props. The simulator mixes the physical and the virtual together in order to demonstrate how passive haptics can be used to enhance the user experience. This prototype will be used to explore how VR might be used with future first-responder training.

(Team Weekend Warriors) Virtual Reality Training with Passive Haptic Feedback for CryoEM Sample Preparation

Team Members: Jiahui Dong, Jun Zhang, Xiao Ma, Pengyu Ren, Zhenyu Cheryl Qian, Yingjie Victor Chen

Purdue University

Abstract: We present an immersive virtual reality training system cryoVR with passive haptic feedback for training biological scientists preparing bio-sample for Cryo-Electron Microscopy (CryoEM). CryoEM requires the user to conduct careful operations on expensive delicate equipment. To minimize the risk and interruption of crucial research work, we tried to mimic the real lab using VR to let trainer practice in a virtual environment. We used 3D printed objects to provide passive haptic feedback in order to achieve a more realistic training experience. Participants are able to interact with the equipment in the virtual environment by moving or touching physical models. We developed all the necessary operations with haptic feedback, including moving, clicking, pouring, rotating, polling and pushing. By following instructions provided by our virtual reality simulator and interacting with physical objects, trainees will learn how to operate CryoEM with low cost and risk. By developing our training system, we explore the benefits, limitations and precautions of embedding haptic feedback to scientific VR training.