To Live, or to ReLive?
Bridging Virtual and Desktop Analysis for Mixed Reality Studies
Imagine being able to step into a virtual environment to view the data from your user studies in person while at the same time having access to powerful desktop tools for data analysis. This was exactly our goal with the ReLive framework, which colleagues from University of Konstanz and University of Maryland and I presented at the CHI 2022 conference in New Orleans, LA, USA. ReLive is a mixed-immersion visual analytics tool that combines the immersive experience of virtual reality (VR) with the analytical power of standard desktop environments. The tool remains relevant today as the field of mixed reality (MR) continues to expand.
In MR, user studies and field evaluations are essential yet challenging. Such studies generate vast amounts of spatial, temporal, and interaction data. Effectively analyzing this data typically requires both ex-situ desktop and in-situ immersive analysis: in other words, the ability to both step into the actual 3D space itself, as well as in a 2D analytics workspace. ReLive addresses this by seamlessly integrating in-situ and ex-situ analysis, thus bridging the gap between immersive analytics and desktop-based visual analytics. We wanted researchers to not only immerse themselves in their data but also to step back and see the bigger picture. Consequently, ReLive allows users to “relive” their studies in VR, providing a rich, contextually immersive environment. At the same time, it offers a desktop interface where users can perform detailed analysis and visualizations.
As MR technologies evolve, the complexity and volume of data grow. ReLive’s approach to combining VR immersion with desktop analytics supports holistic data exploration, making it easier to derive insights that might be missed when using separate tools. Consider a user study where participants interact with virtual furniture in a simulated room. In VR, you can walk through the room, see the participants’ movements, and understand the spatial relationships between objects. Simultaneously, on the desktop, you can analyze aggregated data, create visualizations, and compare metrics across different sessions.
During our evaluations of the ReLive tool, we found that being able to seamlessly switch between immersive and non-immersive views helped identify discrepancies in user behavior. In one instance, a VR session revealed that a participant’s view was obstructed by piece of virtual furniture, affecting their performance. This insight was corroborated with quantitative data on the desktop, which showed increased task completion times for that session.
ReLive exemplifies how integrating VR with traditional analytics can transform MR study analysis. By enabling a seamless transition between immersive and non-immersive environments, it provides a rich and nuanced understanding of user behavior and study outcomes. As MR technologies continue to advance, tools like ReLive will remain indispensable, helping researchers and designers uncover deeper insights and make more informed decisions.
Citation
- Hubenschmid, S., Wieland, J., Fink, D. I., Batch, A., Zagermann, J., Elmqvist, N., & Reiterer, H. (2022). ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies. In Proceedings of the ACM Conference on Human Factors in Computing Systems, 21 pages. https://doi.org/10.1145/3491102.3517550
