The Reality of the Situation

Surveying the power of situated analytics

Sensemaking in-situ where the user is exploring visualizations anchored in their real-world position and shown as 2D planes facing the user.

Imagine walking through a vineyard and seeing not just the rows of grapes, but also a visualization of soil moisture, grape ripeness, and historical yield data, all overlaid onto the physical vines through augmented reality (AR) glasses. This is the promise of situated analytics (SA), where data relevant to a physical location is visualized directly in that location, making sensemaking more intuitive and impactful. Situated analytics combines the physical world with data analytics, leveraging AR to present information that is contextually anchored to the user’s environment. This approach enhances our ability to understand complex data by situating it within the space where it is most relevant.

In a recent survey paper published in IEEE TVCG titled “The Reality of the Situation: A Survey of Situated Analytics”, my colleagues and I at University of Maryland, Bangor University, and Aarhus University are attempting to bring order to this exciting new research area.

Humans are fundamentally embodied beings. Our physical actions, surroundings, and the objects we interact with play a crucial role in our cognition. We remember things better when we physically act them out, and spatial metaphors like “feeling up” or “on top of this” are ingrained in our language. Such embodied experiences suggest that embedding data directly into our physical context could significantly enhance our understanding and decision-making processes.

Various forms of data analytics and visualizations classified based on their use of data (Data), visualization (Vis), computing platform (Platform), user’s physical location (Loc), and integration of the analysis process (AP).

Recent advances in AR technology have made it possible to seamlessly overlay computer-generated imagery onto the real world. This development has brought situated visualization, where data is presented in its physical context, into the mainstream. While AR has been a research topic for decades, it is only recently that it has become accessible enough to be used for complex data analysis by the average person. But what is it that makes the use of such situated visualizations a situated analytics workflow? In our survey, we use the integration of an analysis process as indicator of situated analytics: activities such as viewing and exploring data, schematizing insights, and reporting on analysis outcomes.

By classifying and clustering 47 situated analytics papers from the literature, we identified four main patterns in situated analytics systems: Simulators, Assistants, Planners, and Scanners.

Ensemble combination of multiple cluster analyses performed on our classification.

Simulators use AR to present data as if it were a physical object in the real world. For example, Riverwalk (Cavallo et al. 2016) overlays historical images onto matching views of a city, creating an immersive experience that combines past and present. Similarly, AR can simulate underground infrastructure, allowing urban planners to see pipes and cables beneath the streets.

Riverwalk is novel app-based AR experience that superimposes historical imagery onto matching views in downtown Chicago, Illinois on the bank of the Chicago River.

Assistants provide location-specific information to aid decision-making. For instance, FieldView (Whitlock et al. 2020) uses AR to help field analysts by overlaying various types of data directly onto the landscape. This immediate, contextual information can significantly improve the efficiency and accuracy of fieldwork.

FieldView is an data collection and visualization system that uses mobile overviews and situated AR visualizations to communicate real-time data.

Planners are designed for spatial planning and optimization. An example is a construction site tool (Behzadan & Kamat 2005) that allows users to visualize and arrange construction elements in 3D, helping to identify the optimal locations for equipment and materials before actual construction begins.

Supporting construction sites with situated analytics.

Scanners enhance real-world objects with additional information. For instance, an AR shopping assistant (Ahn et al. 2015) can display nutritional information and budget tracking for products viewed through the device. This immediate access to relevant data helps users make informed decisions on the spot.

From left: AR mobile shopping app (left), health information, product info, and standard non-AR shopping list.

Situated analytics has the potential to revolutionize various fields by making data more accessible and relevant. From medical surgery, where AR can provide real-time anatomical data, to agriculture, where farmers can see environmental data overlaid on their fields, the applications are vast and varied. As AR technology continues to evolve and become more widespread, the use of situated analytics will likely expand. Future developments could see even more sophisticated integrations, such as real-time data analysis and machine learning to provide predictive insights directly in the user’s environment. Our survey provides a starting point for organizing this nascent research field. We look forward to seeing how the taxonomy and the four archetypical SA systems will help structure future work in this exciting topic!

Citation

• Sungbok Shin, Andrea Batch, Peter W. S. Butcher, Panagiotis D. Ritsos, and Niklas Elmqvist. “The Reality of the Situation: A Survey of Situated Analytics.” IEEE Transactions on Visualization and Computer Graphics, 2024.

References

• J. Ahn, J. Williamson, M. Gartrell, R. Han, Q. Lv, and S. Mishra. Supporting healthy grocery shopping via mobile augmented reality. ACM Transactions on Multimedia Computing, Communicatio, and Applications, 12(1s), 2015. doi: 10.1145/2808207
• A. Behzadan and V. Kamat. Visualization of construction graphics in outdoor augmented reality. In Proceedings of the Winter Simulation Conference. IEEE Computer Society, Los Alamitos, CA, USA, 2005. doi: 10.1109/WSC.2005.1574469
• M. Cavallo, G. A. Rhodes, and A. G. Forbes. Riverwalk: Incorporating historical photographs in public outdoor augmented reality experiences. In Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 160–165. IEEE, Piscataway, NJ, USA, 2016. doi: 10.1109/ISMAR-Adjunct.2016.0068
• M. Whitlock, K. Wu, and D. Szafir. Designing for mobile and immersive visual analytics in the field. IEEE Transactions on Visualization and Computer Graphics, 26(1):503–513, 2020. doi: 10.1109/TVCG.2019.2934282