Lindsey Gordon is a first-year astrophysics Ph.D. student at University of Minnesota Twin Cities. Her past research has ranged from exoplanets to quasars to particle accelerators but she’s currently working on computational studies of the ICM and AGN jets. If she had any free time she’d spend it knitting or reading a fantasy novel thick enough to be a door stop.
The translation of the three dimensional universe into two dimensional images has always created issues with preserving the size, scale, and positions of objects (see Fluke 2018). This distortion from flattening 3D geometry poses problems across astronomy, but particularly for science education which suffers when learners are presented with inaccurate data visualizations. Extended reality programs and head mounted devices (HMDs) may present a new solution to this problem.
Figure 1: Reality and Virtual environments are often presented on a spectrum ranging from the fully real on the far right to the full digital on the far left. While all enhancements to reality are generally referred to as ‘extended’ reality, most people consider any programs with a strong immersive nature to be part of ‘virtual’ reality. [image credit: Northwestern University]
Extended reality is a spectrum of experiences that fall somewhere between a fully real and fully virtual environment, as shown in Figure 1. Depending on the level of immersion, these enhancements may be referred to as mixed reality, augmented reality, or virtual reality (VR), with VR being a common catch-all buzzword for any kind of reality-digital mixing that has a strong immersive component. HMDs were once the thing of science fiction but are slowly coming into common use, with commercial offerings from many major tech companies. HMDs offer users a full visual immersion through a headset component, and are usually augmented with a hand held device(s) to control the user experience (see Figure 2 for an example).
Figure 2: An example HMD setup, with a visor and built-in earbuds and two hand held controllers. [image credit: HTC]
Planetariums, Headsets, and Immersion
Extended reality is already in play in astronomy education through planetarium shows. Modern planetariums share many characteristics with VR, particularly their preservation of mapping in three dimensional space and their immersive spherical framing. There are many techniques for how to develop immersive experiences – Stephanie Rigg’s The End of Storytelling is a great resource if you’re looking for more (it’s also a beautiful book). The planetarium and headset experiences have many similarities, including using screens that take up the entire view of the audience, creating programs that emphasize audio/visual cues over text, and developing a ritual to the space. In the same way that you enter a planetarium and experience pre-show cues like lights dimming and announcements, in a headset you have a ritual of putting on gear and watching a loading screen. Creating a smooth transition between full reality and a mixed reality experience is a key piece to the immersive experience for the audience. The agency of the audience to explore is often touted as one of the benefits of a VR experience; users make their own decisions to create a self-directed experience that may be more impactful on their education. Sophisticated planetariums do offer live shows that allow a measure of agency to the viewers to interact with the presenter, ask questions, and guide the visual and aural experience of the show. Education is a goal for planetariums, but often the goal of science museums is to be the impetus for future exploration by guests; they teach viewers that fun can be had in a scientific setting to encourage continued curiosity.
Figure 3: The Boston Museum of Science’s planetarium. Note the large screen & chair placement and angles, with the seats tilted back to immerse guests as much as possible by making the screen most of if not their entire view. [image credit: Boston Museum of Science]
The problem with planetariums is that they are large, expensive to build, and (generally) immobile. Professional observatories and telescopes are also generally stationary and are equally limited by their location, weather, and cost. An HMD is comparatively cheap–while current models go for several hundred dollars, they can be used by many guests over years of outreach as a long-term investment with a lower cost per use than planetarium tickets. HMDs are also portable, requiring only a computer to connect to, and accessible, with very few physical requirements for use. Like a planetarium, an HMD-based VR program can preserve three dimensional spatial relationships, visualize large bodies of data, and create a user-directed immersive astronomy experience. Unlike a real observatory, they have no dependencies on the weather, the local light pollution, or the physical location on the Earth. HMDs are more commercially accessible than ever before, but are not yet a household staple. This provides the benefit of being something new and interesting for guests to experience; there is a ‘wow factor’ to their inclusion in educational events. Guests have been known to line up for over an hour to get to try very short HMD-based astronomy experiences.
On the other hand, their use requires funding to acquire the device(s), as well as specialized software and training for the educators offering HMD programming. There is also a question of the isolatory nature of headset use. While the immersion might be the same or better in an HMD, the communal experience of going to a planetarium show is considered to be psychologically important to the experience. The feeling of community is well acknowledged to be difficult to recreate in VR; virtual group gatherings are not the same as all being in the same place. Communal VR experiences would also require observatories and classrooms to purchase enough headsets for all users to be within the experience at once, incurring heavy costs. However, this communal effect may not be necessary, as the use of the HMD itself as an emergent technology may incite a similar excitement towards the subject matter being presented.
Figure 4: Desktop view of WWT, focused on a view of Earth. The desktop version of WWT still suffers from the 3D to 2D translation, but is a free and accessible way to explore real astronomical data. [image credit: WWT]
VR Programs Available to the Public
Despite the difficulties, there has been a boom in software development for astronomy education in VR. The most significant piece of software released to date is Microsoft’s WorldWide Telescope (WWT) platform, which was later taken up by the American Astronomical Society (AAS) (WWT paper here, WWT online version here). WWT presents a spatially accurate view of all publicly available astronomical catalog data and allows users to explore everything from our solar system to deep space in a browser, on a desktop (see Figure 4), or projected in 3D into a planetarium or HMD. WWT has been integrated successfully into a number of smaller planetariums, kindergarten through college level courses, and virtual reality based museum exhibits. Both WWT and classic planetarium shows utilize what’s called a “contextual narrative layer”, a VR user experience technique that uses storytelling, user-driven exploration, and built-in real information (for more on the specifics of this technique as implemented in WWT, see Wong 2008). Users can then follow recorded guided 3D tours or explore on their own, all while having access to the actual catalog data being used to generate their view. WWT is free and open-source and is designed to be customized for different user experiences, making it one of the most popular tools for astronomy education in VR.
Other virtual reality implementations for astronomy education include allSkyVR (paper; software), released in 2018, which was designed as a ready-to-use astronomy visualization tool for HMD-based VR as well as a basis for customizable development. Stellarium (paper; software) is a tool similar to WWT that is designed primarily for desktop use but also has extensions for planetarium projection use. They also have an augmented reality mobile application that uses the phone’s position data to show an overlay of what is in the sky at that time. MarsVR (paper; website) is a recent launch that uses real NASA Mars data to allow users to explore the surface of Mars and is explicitly designed for public education and HMD use. Scientists at the Astrophysical Big Bang Laboratory produced the VR experience SN2SNR (paper) to allow guests to explore simulations of the explosion of a supernova and its transition to a supernova remnant.
Virtual reality programs are slowly beginning to take root in astronomy outreach education. There are no perfect educational tools, but HMD-based VR programs have many benefits that observatories, museums, and classrooms are only beginning to leverage.
Edited by Macy Huston & Jenny Calahan
Featured image from Boston Museum of Science
