Capturing life aboard the ISS in a four-part VR documentary series

Only 553 people have ever been to space and, even with our emerging space tourism industry, the chances of you or anyone you know actually doing the same in this lifetime are astronomically small. But rather than bring people up to the international space, the Emmy award-winning team at Montreal’s Felix and Paul Studios is instead bringing a little bit of the international space station down to the people in its four-part immersive, virtual reality video series, Space Explorers: The ISS Experience.

"We use this medium to take people to places they cannot go," Félix Lajeunesse, co-founder and creative director at F&P Studios, told CollectSpace in 2020. "For us, that doesn't necessarily mean creating a fully interactive experience where we ask audiences to press buttons and do other such things as in a game. We remain on the side of a cinematic experience, where what we really try to do is nurture a sense of presence for our audiences."

SE:ISS grew out of the existing 2016 Space Explorers: The Story Begins project between F&P Studios and NASA documenting the training of a new generation of astronauts here on Earth, Lajeunesse explained. During that two-part series’ production, “we started to socialize the idea that we wanted to do a second season, or a second phase for the franchise,” Lajeunesse said, “but this time we would want to film in space aboard the International Space Station.”

NASA, having already been training its astronauts through virtual reality and immersive technologies, so it was an immediate and enthusiastic partner in the new endeavor. Soon after, the team realized that Time Studios was also working towards capturing a live spacewalk in VR. “We said, ‘we are looking at creating this, you know, large scale media endeavor up there in space and it seems like we have interests that are aligned so let's try to work together,’” Lajeunesse continued.

“Getting this camera to space was the culmination of five years of exceedingly hard work. Only 228 humans have ever conducted a spacewalk,” Jonathan Woods, Executive Producer for the project said in a press statement. “It is one of the most thrilling yet perilous tasks an astronaut can undertake.”

The F&P team also set about crafting a set of cameras capable of the environmental rigors required for life aboard the ISS, partnering with Nanoracks to do so. “Our Space Camera, purpose-built to capture this historic event in fully-immersive 3D, brings us one step closer to our goal of taking billions of minds to space, and having them experience a spacewalk as if they were astronauts themselves,” Lajeunesse said in a statement last year.

The Space Camera starts its life out as a Z-Cam V1 Pro camera. It’s outfitted with nine 4K imaging sensors arrayed in a ring that allows them to capture 3D, 360-degree images at 8K resolution. Nanoracks then proceeds to harden the equipment to withstand “low atmosphere orbit, including vacuum, solar ultraviolet (UV) radiation, charged particle (ionizing) radiation, plasma, surface charging and arcing, temperature extremes [from -250° F to +250° F], thermal cycling, impacts from micrometeoroids and orbital debris (MMOD), and environment‐induced contamination,” according to the release.

“One of the things that happens when you film in space,” Lajeunesse noted, “is that the heat from the camera is going to dissipate in a different way because there is no gravity. So we needed to change the thermal management of the camera so that heat could dissipate.”

The team also had to contend with cramped filming conditions aboard the ISS as well as the realization that it needed the “ability to remote control this camera from a distance, because, obviously, the cameras in space but as a production team, we are on Earth,“ Lajeunesse quipped. So, in order to capture quality stereoscopic images within the cozy confines of the ISS, they had to “think of a calibration slit scan system that allows the camera to basically capture by rotating the head.” he continued. “All points of light inside of a scene prior to filming the actual scene, and that gives us an extraordinary amount of visual data for the recreation of the environment and post production.”

The external camera F&P plans to use later this summer to capture the spacewalk has been hardened even further, to account for the five days it will have to remain exposed to the horrors of open space during filming, Stéphane Rituit, co-founder and CEO of Felix and Paul Studios, told Engadget. Based on the same V1 Pro as the interior camera, the spacewalk camera will “have to take the vacuum, the huge difference of temperature… and the fact that we're not inside so we're not in a pressurized environment,” he explained.

Let’s be honest, at this point JJ Abrams is the only one of us left who still thinks that lens flare is cool. But when attempting to capture video in space (where there is no atmosphere to obscure harsh glares) while traveling at 17,100 miles per hour (that’s 16 sunrises and sunsets per day) while using a 360 camera (which, by definition, is always at least partially staring directly at the sun) lens flare becomes a serious production quality issue. “So what we actually did is we designed the lenses with a very special coating to alleviate and diminish the effect of sun rays and the associated lens flares,” Lajeunesse crowed.

Radiation was a pervasive and ubiquitous issue for the team as well. “When we began production in early 201, we spoke to different people at NASA who told us, ‘you know, any electronics that you send to the space station — any camera equipment that is electronic — will eventually degrade because of radiation,” Lajeunesse noted. “The amount of radiation that you will get inside or outside the space station on your electronics is not fundamentally different.”

For traditional cameras this can be a major problem, especially when errant alpha particles go blacking out pixels at random throughout the imager. To get around this issue, the camera design team incorporated a high degree of intentional overlap between the nine 4K imagers so that even as the capabilities of each individual degrade over time, the entire rig can continue to capture content.

In fact, these cameras can collect up to 15 hours of 3D content at a time. That generates a lot of data over the course of the 250-plus hours of filming accomplished, and I mean a lot of data. Measured in the “tons of terabytes,” Lajeunesse describes it as “an extraordinarily huge amount of data.” Getting all those ones and zeros back planetside requires a number of dovetailing methodologies. At the end of every hours-long shoot, astronauts aboard the ISS would beam down a low-res “unfolded” 3D image — in that it has been rendered as a flat, 2D picture. This gave the NASA and the F&P production team a rough idea of the session’s content (enabling them to give notes and feedback) without tying up the ISS’s comms channels for extended periods.

“When that footage would come down, it would go to NASA first and NASA would have to review the footage before it gets released to us,” Lajeunesse said. “And so that was very efficient, as a process it allowed us to see what we were doing and to be in a mode where we could come back to the astronauts and say ‘look, that's fine, let's try to do this, let's try to do that,’ which allowed us to course correct through the two-and-a-half year filmmaking process.”

Acquiring the high-resolution RAWs was even more of a challenge. “Very often what we did is send time codes so that [the ISS crew] could download separate, selected segments of the high-res files, so that we could start production down on Earth,” Lajeunesse noted. And in the instances where there was a cargo transfer or crew changeout, the collected video content was gathered up on SD cards, plopped into a sealed bag and sent back down the gravity well where they were recovered and treated with higher reverence than the Holy Grail itself, according to Lajeunesse.

Moving forward, Lajeunesse and Rituit hope to head even further into the interplanetary void. “We're at a moment in time where humanity plans to go back to the moon to the Artemis missions, and then eventually to Mars,“ Lajeunesse exclaimed.

As part of the celebration surrounding Friday’s National Space Day, Felix & Paul Studios are offering free screenings of The ISS Experience. Chapter 1, “Adapt” will stream on loop at no charge to anyone with an Oculus Quest, Quest 2 or Oculus Rift. If you don’t have a headset of your own, the company is already working with space centers around the country — including those at Nashville, Huntsville and Houston — to produce 2D versions for their theaters, according to Rituit. What’s more, the source content “can be packaged for multi-platform distribution in virtual reality, augmented reality, mixed reality, mobile, desktop, fulldome, giant screen and other formats,” the ISS National Laboratory mentioned. Episodes one and two are already available to watch, episode three arrives in Q3 while the final installment drops in Q4.