We all know that VR does certain things well, such as games and interactive experiences. But there are some creatives, including Ethan Shaftel, who believe virtual reality can also be used to enhance cinematic storytelling. For the past six years or so, Shaftel has been working on various techniques to exploit the medium in that regard, to improve the overall visitor experience. His latest animated short film, “Ajax All Powerful,” utilizes some of those filmmaking techniques, along with a Maxon- and Unity-based pipeline.
What do we think? For those believing that the entertainment world would ignite the AR/VR/MR revolution, well, we’re still chugging along at more of a tortoise pace rather than that of a hare. A lot of this has to do with the associated hardware. A year ago, there were numerous companies popping up and promising smart glass or headsets, with mostly just the big names shipping product or close to it. Granted, the devices are improving, but not enough to make reality-based activities part of our everyday norm. Software vendors are doing their part to strengthen the medium, as well, with robust tool sets that make generating AR/VR/MR content more fluid. Now, we need daring creators to step outside the tried-and-true and deliver something innovative enough to give the space the added push it needs. After all, look how the cellphone evolved into a smart phone that is now integral to our everyday life and whose functionality goes well beyond a simple phone call.
Cinematic storytelling in the age of virtual reality
When 2022 came up short in terms of AR/VR headset shipments and sales, some market intelligence providers like TrendForce acknowledged the decline but predicted a surge in 2023. We’re still waiting, and likely will continue to wait at least until 2024, when Apple finally rolls out its Vision Pro. Meanwhile, Sony’s PS VR2 released earlier in the year, while Meta just started shipping a higher-priced and more powerful Quest, the Quest 3 mixed reality headset. And there are promises for more devices from other vendors. But, what good are they without content?
Among the content for the Vision Pro is stuff you make yourself, as Apple is bolstering its offering by enabling iPhone 15 Pros to shoot spatial video they can then relive using the device. Google has a dedicated VR platform on YouTube for accessing content, Steam offers VR games, and the Meta App Store is a good place to find a wide range of reality games and experiences for Meta’s Quest platforms. A recent addition to that store is the VR animated short comedy “Ajax All Powerful.”
Written and directed by Ethan Shaftel, “Ajax All Powerful” is a cinematic storytelling experience produced exclusively for virtual reality. Unlike some VR experiences that involve interactivity by the participant, “Ajax All Powerful” removes the barrier of the fourth wall to provide viewers with a uniquely immersive perspective in which they can experience the events and emotions more acutely through the viewpoint of the protagonist. “We’re used to a linear, third-person point of view through the eyes of the protagonist. In VR, it’s a first-person experience. That first-person element can undermine the relationship with the protagonist; it actually makes it more difficult to see the world from the protagonist’s point of view. We tend to forget what cinema does well, which is strip away our own identity in favor of the protagonist. My goal as a storyteller is to balance first person and third person in a way that allows the viewer to leave themselves behind.”
In the short film, a foul-mouthed, double-crossing genie grants wishes to gullible humans, and in exchange, he ends up with their souls. He meets his match, though, when he encounters a little girl, who is backed by a no-nonsense lawyer (voiced by Chris Parnell) who aids her in negotiating her wishes. The devilish Ajax (voiced by Henry Winkler) has met his match.
Shaftel notes that “Ajax All Powerful” is a VR movie, and his goal was to make it an entertaining and enjoyable cinematic experience for everyone, even those who have no understanding or interest in VR. “All they have to do is put on the headset. It’s a storyline and characters they would recognize from the cinema world,” he says.
However, to make that work, there is a lot of behind-the-scenes cleverness and interactivity required to deliver a cinematic story in VR. And that is all optimized and hidden, unbeknownst to visitors. And the movie, which is a seated experience, is optimized for the pace and interest of the visitor. Everything just works around them, the filmmaker says. They do not realize that their own movements and pace is triggering and changing the movie around them.
The story takes place in very specific and nested spaces that highlight what immersive media can do. The story starts inside the genie’s lamp, moves outside of the lamp into the lawyer’s office, and then moves to the city outside that office. Each time, Shaftel explains, there are transformations of scale and points of view that are undertaken not only by the characters as the genie grows and transforms, but also by the viewer. “It was an opportunity to tell a story where space is part of the canvas on which the psychology of the characters is drawn and power in the scene changes and shifts at the same time as size and scale,” he explains.
Buoyed by all the initial buzz VR was generating, Shaftel began creating his first VR project in 2016, calling upon what he knew best—character-based storytelling combined with his experience in media design for immersive spaces, pre-VR, using giant LED walls and video surfaces for concerts and theme park installations.
For the past six years or so, Shaftel has been all in on VR and continues to develop and hone techniques for cinematic storytelling in this space. “Ajax All Powerful” marks the first time he was able to fully realize some of those directorial techniques he had been working on over the course of previous projects. These include fast movements through space that change the visitor’s perspective and scale in relation to the narrative as it unfolds, which he calls “zips.” Zips become one part of a shared choreography between the audience and the creator, with the visitor turning their head to look at things that interest them, which, in turn, triggers the interactive zip movement. Zips always add to the visitor’s interest by either moving towards their object of attention, adding detail, or away from it, adding context, he explains. Like good cinema editing, visitors engrossed in the story take in the added detail or context just as the narrative demands it, feeling the spatial change, while not being fully conscious of it.
Similar, but very different
Shaftel believes in the VR medium for storytelling—in addition to the traditional experiences like puzzles and exploration often associated with VR and which VR does exceptionally well. “When you make a movie in mixed reality, it’s really a whole world, so there is so much more animation to do,” he says. “All of the characters are alive all of the time, so you have to be really brutal about where you put your production time, polish, and energy.”
Shaftel works with small team of close collaborators—an animation partner and Unity developers—and together they perform most of the creation work using a workflow that includes Maxon Cinema 4D and ZBrush, along with Unity for the real-time work. The filmmaker says the advantage to using Cinema 4D in VR production is that it can be used in several different ways; it’s flexible and the process for exporting files into VR is straightforward.
The group had been using Cinema 4D for their past work, so they were familiar with the software. (JPR’s DCC report provides an in-depth look at Cinema 4D and other digital content creation tools.) But after Maxon added robust VR tools, they found it advantageous to continue using it when they began working on VR projects. ZBrush is primarily used for character design, while Cinema 4D is used for animation, which makes it easy to plan and block out action sequences during the writing and animation processes. Also, the VR 360 video tools in the software enable Shaftel to render out frames of 360 video from different perspectives, which are then imported into Adobe Premiere to be edited into a VR storyboard. During the full production process, the dialogue soundtrack is brought into Cinema 4D so the characters can be rigged and animated to coincide with the vocal performances.
“I think VR has the potential to revolutionize our relationship to spaces and our relationship to the real world around us. I don’t know if VR will revolutionize storytelling, but it’s something that is worth investing a lot of experimentation and passion into, because it maybe it will [revolutionize it],” says Shaftel.
So, what is needed to encourage more types of VR projects? It’s easy to get someone to try VR, Shaftel says, but it’s hard to figure out how to get VR to become a regular part of their life. And the current headsets are not at the point to make that happen, yet. But he believes we will eventually get to where VR is ready for mass consumption. That is Shaftel’s wish—no genie required.
“Ajax All Powerful” took about a year to make, with about a third of the time dedicated to writing, prototyping, and pulling together the financing. The VR short film is available on the Meta Store for all Quest and Quest Pro headsets, including the new Quest 3.