Stunning Virtual Reality Project Takes You to the Arctic — and Into Climate Change

Amid a flutter of snowflakes floating aloft on gentle winds, you hear the snorts of caribou and the crunch of their hooves on the frozen tundra. But you’re not watching the herd while shivering in the Arctic.

You’re in the Natural History Museum of Los Angeles County.

The museum has long offered visitors a window into the habitats of wild animals by recreating snowy landscapes, lush jungles and more through its dioramas. (Just ask the average Angeleno who trekked there on school field trips.) But now people can go beyond looking at the landscape. They can immerse themselves in environmental education through virtual reality, thanks to USC students.

A group of game design, computer science and journalism students relied on a high-tech 3D scanning tool and hours of research and design to create “Beyond the Diorama,” an interactive VR experience that transports you to an icy world.

It’s a project with purpose. “We wanted to create something that could help with climate change,” says Rong Deng, a master’s student in interactive media and game design at the USC School of Cinematic Arts and the project’s creative director. “For both the caribou and permafrost, there is a direct connection with climate change.”

The climate project reveals that melting permafrost could release up to 53 times as much carbon as sources of pollution like cars in a major city like Los Angeles.

As Deng explains, the trouble starts with bugs. Insects called warble flies surround the herd and lay their eggs in the caribou’s hair. When the larvae hatch, they burrow into the flesh, leaving the caribou wounded and at risk of infection and predators. Normally, the herd would seek refuge from the flies in colder areas during the summer. But climate change means they have fewer places to escape the pests.

Another major consequence: As the tundra’s permafrost thaws, it emits carbon, creating a feedback loop of more warming. The USC student-led interactive virtual reality climate project reveals that this melting process could release up to 53 times as much carbon as sources of pollution like cars in a major city like Los Angeles.

“The caribou turned out to be the perfect heartbreaking story,” says Robert Hernandez, professor of professional practice at the USC Annenberg School for Communication and Journalism. He has led similar immersive journalism projects with his students as part of an initiative called JOVRNALISM. “It has such good conflict and drama.”

How USC Students Turned a Museum Diorama into an Immersive Experience

The caribou project brought together students and other collaborators from across campus: USC Annenberg, the USC Dornsife College of Letters, Arts, and Sciences, the USC Viterbi School of Engineering, USC Games and the USC Wrigley Institute for Environmental Studies. Overseen by Hernandez and Vangelis Lympouridis, a lecturer in computer science at USC Viterbi and USC Games, the project received support from the USC Office of the Provost.

This was an opportunity for students to make something that has an impact and is not in the traditional realm of a video game or cinema.

Robert Hernandez

“This was an opportunity for students to make something that has an impact and is not in the traditional realm of a video game or cinema,” Hernandez says. “But the true secret was these students with different backgrounds coming together to work as equals.”

Lympouridis has seen a trend toward more virtual and augmented reality experiences in media and entertainment, not to mention other industries like education, so the cross-campus partnership felt like a natural fit. Leica had loaned him a high-end lidar scanner to use in his class on augmented, virtual and mixed reality. The powerful device uses lasers to create a 360-degree digital model of an environment.

And what better fit for this high-tech approach than digitizing the natural history museum’s famed dioramas?

“These dioramas were the virtual reality technology of the 1920s,” Lympouridis says. “Explorers were traveling to exotic, undiscovered places and then trying to recreate these environments to show others what it’s like to be in Africa or the Artic pole, what it’s like to experience the animals and their habitats there, and so on.”

During COVID-19, when all museums had closed, he says this project illustrated the potential global reach of museum exhibits via VR and immersive technologies.

Users can download the USC “Beyond the Diorama” content and watch it on a virtual reality headset, including interactive quiz questions about climate change and its impact on caribou. An augmented reality version lets viewers enter the diorama through their handheld device. (iPhone and iPad options are currently available.) The team also created a 360-degree video version so people can watch the experience play out, even if they don’t have a VR headset or smartphone.

Virtual Reality Environmental Education Project Carries a Critical Climate Change Warning

More than 15 students worked on the project over two semesters. As creative director, Deng still marvels at how the effort involved many different disciplines across the university.

“I worked with scientists and journalists and engineers — so many different perspectives,” she says. “The journalists were super professional. They did research and prepared questions and worked so hard on the script. The engineers said: ‘Tell me what to do.’ So I’d tell them, ‘We need snow here and the flies should move like this.’ It only took them a short time to create a vivid game experience.”

Along with capturing the diorama with the lidar scanner and translating the data into an immersive experience, the team also had to ensure they used scientifically accurate content. Students interviewed experts at the museum and the USC Wrigley Institute to learn more about the caribou’s plight. Once they gathered their material, they had to shape it into an emotional arc to keep viewers engaged.

We make the people think about how this is an environment we need to protect.

Rong Deng

“I hope users initially have curiosity and want to explore the environment,” Deng says. “After 45 seconds or so, we build into our story. These caribou are living in a terrible environment right now with these terrible flies that bite them. We make the people think about how this is an environment we need to protect. And we actually can do things to make this better, to stop the glaciers or permafrost from melting, like driving less or using public transportation.”

Lympouridis said the project offered a valuable opportunity for his students to work with the latest 3D scanning technology and learn to collaborate with people from different fields. “They got a chance to create a project that demonstrates their ability to understand the medium of augmented and virtual reality and engineer solutions that have purpose and impact,” he says. “This is a lever for students to excel in their careers and get good opportunities in the industry.”

Similarly, Hernandez sees immersive reality as the future of journalism and entertainment. And he doesn’t want to wait for media companies to determine the path forward.

“I want diverse voices — voices that are often left out — to proactively shape this world,” he says. “Projects like this position them to be leaders and pioneers in those spaces.”