Can Immersive Storytelling with VR stand up to the Experience of Visiting Antarctica?
Lessons learnt from a unique setting and a fanatical audience that can guide immersive storytelling development more widely.
Working with the UK Antarctica Heritage Trust (UKAHT) I undertook research to look at their initial development of a Virtual Reality (VR) experience of Antarctica. On a personal level, I had never previously dreamed of visiting Antarctica, but as I came to learn through this project, it attracts a lot of fanatical people with a desire to do just that. These “fans” who became the research participants brought this project to life and highlighted insights that can be useful for those making immersive storytelling experiences more widely.
Often when conducting research, it can be a struggle to recruit participants, and in market research it is common to circumnavigate this by using a professional recruiter, and offering financial incentives. So, when UKAHT suggested they would be able to find volunteers by including a request in their newsletter I was sceptical. What followed was a deluge of hundreds of emails requesting to join the research as a volunteer and outlining their exact connection to the place in autobiographical form.
The participants were split into focus groups based on the following criteria:
- Had been on a cruise to Antarctica
- Had worked as part of the British Antartica Survey (BAS)
- Were a young person or teacher connected to the region through the UK National Curriculum
- Were interested in the region but had never visited
In this post I will outline how each of these different expert groups provided insight into the importance of one particular aspect of immersive storytelling design in relation to their unique connection to the context of the story. These are (1) details, (2) memory, (3) physical materials and movement, and (4) interaction.
BAS Workers & Details
The group of British Antartica Survey (BAS) workers were in their 60s, 70s and 80s having worked in Antarctica in their 20s or 30s. They talked about their time working on BAS as being formative, and an experience that had been unparalleled since. Things that stick in my mind are their descriptions of the isolation of working there but without a sense of loneliness. Because they were BAS workers pre-internet they described having a telegram allowance of 50 words a month in and out of the region. Such limited contact with the outside world and their descriptions of returning to it reminded me of description provided by astronauts readjusting after returning to Earth. Perhaps the unique nature of the experience had caused so many details to remain prominent in their minds even decades later in a way that casual tourism today may not.
The detailed memories of the BAS workers about their time spent in Antarctica became apparent in their critique of the VR experience (still a work in progress at the time). The BAS participants were less interested in the overarching narrative, and more in the minute details. For example, they stressed that the chance to ride a dog-pulled sled were very small due to the reliance on a particular quality of snow.
Figure 1 below shows an initial rendering of snow in the VR story:
The quality of the snow in this scene went unmentioned by the other participant groups. Yet, a photograph of a dog-pulled sled shows the uneven texture of the snow (Figure 2).
Similarly, for the part of the immersive story where a worker’s tent is blown away in a blizzard (Figure 3), the BAS team stressed that neither the level of visibility nor the loudness of the wind were yet accurately portrayed. Nor were the the tent poles tied correctly. This was again different to other participant groups that focused on on the narrative, the horror of the event, and what it would be like to be in this situation.
Bell et al (2018) describe how aspects in the physical world can “pop” the audience “out” of being immersed in a digital story. Here, the lack of realistic detail for an audience who had spent a long time in Antarctica served to “pop” them “out” of being immersed with the wider narrative. Understanding this is a good reminder of the need to closely consider the desired audience for immersive storytelling. In my MA teaching of immersive storytelling often the desire to tell a particular story is foregrounded and the focus on who will engage with it coming later, whereas both should be intertwined from the start.
The importance of detail in relation to physical materials and VR was one of the findings of the UK-Japan Location-based VR Network project that I led in 2019–2022.
In relation to this Main (2020) wrote:
The power of little details is perhaps most evident in the production design for 2001: A Space Odyssey. With science fiction, the skill of world-building is more critical. The film needs to create a convincing reality which has never existed, but is relatable enough for the audience to believe is could exist. High concept ideas are intertwined with everyday mundanities to create persuasive moments. You see it in a hundred different vignettes: zero gravity jogging, space-age airline food, familiar product placement, and zero-gravity toilet instructions. Each of these elements is a small, convincing detail which immerses the audience in the reality.
2001: A Space Odyssey was something we thought about at that time as an exhibition of Kubrick’s work was on display at the Design Museum in London.
In the exhibition a model and information about the building of the Centrifuge was displayed (Figure 4). Both served as a reminder that if detail is paid to some aspects of a story then the audience will suspend disbelief at other less-detailed parts.
In other words, a VR experience would not need to be rendered to the same level of detail throughout its duration, rather decisions should be made about the best parts of the story to embellish with detail. Further, these decisions should be made with the intended audience for best results, such as the UKAHT are doing by instigating this research.
Young People & Tactile Materials
Both young people and their educators talked of the success of bringing tactile materials into the study of Antarctica in formal education. In relation to this, young people were most interested in physical materials that connected them to everyday life in the continent, which was further motivated by a desire to compare these with their current lives. For example, what do people eat, where do they go to the toilet, how do they they sleep etc. Teachers and students alike confirmed that the most enjoyable way to experience this information was through physical materials.
They also like to drag the tyre I used to train to mimic the pulk.
(Female, Secondary school Teacher)
In Yamada-Rice (2021), I write that one of the key ways in which children want to experience VR is in relation to tactile means, even when virtual content does not map onto physical materials. I related this to earlier ideas by Garzotto (2014) in relation to children’s interaction with digital rather than virtual stories.
Creators’ of interactive stories, individually or in groups. In both cases, interactivity has the potential of increasing enjoyment, and fostering new forms of creativity, social activities and learning. (Garzotto, 2014, p. 5–6)
How to bring tactile materials into an virtual experience is discussed in more detail later.
Antarctic Travellers & Memories
Before starting the project I had assumed that a visit to Antartica would be a once in a lifetime event for those that had visited. However, all of the participants who had visited the region suggested a desire to return. Some had already been multiple times. Yet, Antartica needs to be protected from over tourism. I began wondering whether VR could ever be up to the challenge of curbing the desire to revisit. In relation to this, the research data showed how those who had already been to Antartica were interested in their memories of the place and wanted to recall and share those.
Many technologies that have gone before have sought to allow people to capture memories of a place. Sontag (1979) has written extensively about this in relation to photography. In turn photography has been used as a means of allowing people to share memories of places; postcards being one example. Photography has also been the basis of many iterations of creating 3D imagery to remember places, such as with ViewMasters and earlier stereoscopic imagery (Figure 5).
However, the data from participants who had been to Antarctica did not simply describe what they had seen, they expressed full-bodied memories that included how cold it was, the “fishy” smell of the penguins that was not quite fish but hard to put in to words. Kunzru’s (2013) work “Memory Palace”, opens with a description of how to embed a place to memory by using details and senses.
Storytelling with Virtual Reality still foregrounds the visual mode and often when other senses are added to the experience such as Subpacs that vibrate on a users back to simulate movement, or scents to illustrate a particular environment they “pop” (Bell, 2018) the user out of the virtual experience and back into the physical world rather than more deeply immerse them. However, as will be described next, interaction is one of the ways in which VR can successfully meet fan audience’s expectations.
Fandom, Expectation & Interaction
Fandom for the continent was evident both with those who had travelled there and those that wanted to (though some had decided they would not in order to protect the region). One of the findings indicated that most of the participants had become fanatical about Antarctica after an encounter with a book or a story in childhood. From here, they had become avid consumers of books, exhibitions, talks and documentaries trying to learn more about the region. Most participants expressed a desire for more detailed knowledge, and became excited when other participants shared materials they had not come across before.
When reflecting on the VR world, Fandom was expressed in relation to a desire to be able to interact with virtual objects. For example, when shown the inside of Stonington Hut in VR (at the time a work in progress) they were asked what they would want to interact with in the space (Figure 6).
As can be seen in the image of Stonington below (Figure 7), the living space was cluttered with an array of objects, and participants were keen to interact with them all. As was the case with the young participants outlined above, this connected to a desire to understand all aspects of the everyday lives of the people that work in Antarctica.
The participants expectations were high, in as far as they had already consumed a lot of material about Antarctica in multiple modes, and were thus connoisseurs about what made a good documentary and use of moving image, compared to how audio could be used as part of documentation or storytelling. When asked about possibilities for the VR experience to be location-based, i.e. to include materials beyond a headset several of the participants described excitement that the immersive experience might link with other materials and senses. The participants ideas were fairly literal, for example, could UKAHT lower the temperature of the physical environment to feel like Antarctica or add smells such as that of the fishy smelling penguins.
As described above the use of scents can cause to the VR user to disconnect from the virtual experience rather than immerse them more deeply. However, drawing once again on the findings from the UK-Japan location-based VR project it is possible to see how the sense of touch is a much easier way to deepen a user into a virtual narrative.
In a simplistic way, the use of a physical carpet in Hashilus’ Soloman’s Carpet VR experience deepens the users belief that they are riding on a magic floating carpet in the VR world. As is shown in Figure 8 the contact of the carpet via the feet seeks to remind the user that this is reality not an illusion.
In the work of Matsumoto et al (2017) physical materials are mapped to virtual content to create a specific cognitive illusion. More precisely when a virtual triangle-shaped table is mapped to a square physical table, the VR user feels the table to be a triangle even when it is not. This is a cognitive illusion based on the notion that most users with good vision are more used to receiving information through vision than touch.
This post has sought to show how even when the narrative of a virtual story has been decided upon, developers need to continue to work together with the intended audience to explore, through research, which parts of the narrative need to contain a high level of detail, how it links with memories, the possibilities of physical materials in relation to the virtual story, and the role of interaction.
References
Bell, A., Ensslin, A., Van Der Bom, I. and Smith, J. (2018). Immersion in digital fiction. International Journal of Literary Linguistics, 7(1).
Garzotto, F. (2014) Interactive storytelling for children: a survey. International Journal of Arts and Technology, Vol.7(1), p.5–16.
Kunzru, H. (2013) Memory Palace. London: V&A Publications
Main, A. (2019) Detailed World-building. Available online at: <https://ukjapanvr.wordpress.com/2019/08/06/detailed-world-building/> Accessed: [31/3/23].
Matsumoto, K., Hashimoto, T., Mizutani, J., Yonahara, H., Nagao,R., Narumi, T., & Hirose, M. (2017). Magic table: Deformable props issuing visuo haptic redirection. SIGGRAPH Asia 2017, Emerging Technologies, p. 1–2.
Sontag, S. (1979) On Photography. Penguin
Stanley Kubrick: the Exhibition (2019) The Design Museum, London https://designmuseum.org/exhibitions/stanley-kubrick-the-exhibition
Yamada-Rice, D. (2021) Children’s interactive storytelling in Virtual Reality. Multimodality & Society, Vol.1(1), p.48–67.
Yamada-Rice, D., Dare, E. Main, A., Potter, J., Ando, A. Miyoshi, K., Narumi, T., Beshani, S., Clark, A., Duszenko, I. Love, S., Nash, R., Rodrigues, D., Stearman, N. (2020) Location-Based Virtual Reality Experiences for Children: Japan-UK Knowledge Exchange Network: Final Project Report. Available online here