START TYPING KEYWORDS TO SEARCH OUR WEBSITE

Challenges, risks, opportunities for entertainment with VR storytelling

Posted: Andy Marken 12.07.17

The annual SMPTE (Society of Motion Picture & Television Engineers) is a technical conference, loaded with technical sessions where senior engineers from across the M&E (media & entertainment) industry catch up on the standards and technology. The stuff that enables them to keep up with the rapidly shifting content development, production, distribution and monetization landscape.

That’s important with new players like Twitter, Facebook, Snapchat, YouTube, Roku, Vimeo, Sky, Amazon, Netflix, Disney, TenCent, Orange, Viu, Muvi, Afrostream, Zype, Baidu and thousands more

Despite the technical and marketing issues they were facing, the SRO (standing room only) audience was eager to get a fresh perspective on how VR (virtual reality) is going to change the entertainment/information landscape.

And cinematographer Andrew Shulkind didn’t hold back in his keynote presentation – Broaden Your Horizons.

Real Virtual – Andrew Shulkind, a highly regarded and articulate, VR cinematographer keynoted this year’s SMPTE conference and explained to a packed house how the industry had to broaden its horizons to meet the wants, needs of today’s sophisticated and involved viewers.

Shulkind set the stage for the attendees by saying how great it is that writers, directors, cinematographers, FX, audio and post pros as well as actors regularly come together to take viewers to a different place, a different reality.

To illustrate the point, he flipped through scenes from memorable films through the ages.

Watching them, I thought to myself, “Damn this is a great industry.”

He said something similar, noting that what traditional media has done so well (visually expressing human emotions) immersive content can do even better. Then he warned the audience, “Our past activities don't necessarily work anymore. The industry still clings to the bottom-line model … and the blockbuster system is broken."

“Making movies as we’ve done them in the past, has become an impossible proposition," he added.

Shulkind, the youngest inducted member of the International Cinematographer’s Guild, is one of the new breed cinematographers we’ve come to know who are serious about melding art and technology to bring viewers into the VR storytelling process.

Because of the breakdown of the old infrastructure and changes that are taking place, he believes Hollywood, motion pictures and the new content opportunities actually have the power to democratize our world.

He explained that today, people in the industry rely more heavily on technology to help tell stories in new ways and empower underrepresented writers, storytellers and talent to tell their own narratives.

He told the audience they need to integrate VR into existing production workflows and help develop and refine a full stack of protocols, techniques and tools.

Production Discussion – Andrew Shulkind (l) discusses some of the “scene” work in the recently completed VR film, The Ritual, which has since been acquired by Netflix.

Shulkind’s insights are more than just observations because he has done immersive commercials for Apple, Google, Chevrolet, Sony, and other progressive firms. His latest film was The Ritual, for Andy Serkis’ studio, which was purchased by Netflix at the Toronto International Film Festival (TIFF).

To amplify the industry shift, Shulkind pointed out that long-held systems are breaking down in our business as content goes more mobile.

Mobile First – Almost born with smartphones in their hands, the younger generation often picks up the device when they want to watch content of any kind.

He emphasized that for today’s viewers, the big screen has been replaced by the small screen in your hands and that the future is short form, mobile first; citing recent mobile industry trends:

  • 50 percent of all video content is consumed on a mobile device
  • Video content accounts for 72 percent of all Internet traffic
  • Over 3B smartphones are in active use today
  • Videos of less than 5 minutes account for over half of all mobile video consumption
  • 10M videos are watched on snapchat every day
  • 92 percent of mobile video viewers share content with others
  • “The future of content is decidedly mobile first,” he emphasized. In fact, with so many options at our disposal, the modern viewer doesn't really have to go to the theater at all.

He noted that not every film will be 360 degrees but now; writers, directors, storytellers, content makers, and advertisers have more effective tools at their disposal.

He added that VR is the new language of storytelling and that with immersive media, AI, blockchain and a tidal wave of technology heading our way; it will totally upend the business.

On the upside, he said the business isn’t refined enough yet to have cogs in wheels and filmmakers are now more than just technicians, they are multi-hyphenated generalists. Following his keynote, we sat down with Shulkind to explain and clarify some of the common issues raised and solutions.

Q – What about the common shortcoming that you can’t script or guide in the storyline?

Shulkind – Lewis Smithingham, of 30Ninjas, was one of the first to explain how a VR film could be scripted and how the viewer could be guided, which he did beautifully with Doug Liman’s mini-series Invisible.

He showed people how he used a laser to guide the attention of a cat and said it is the same with visual and audio guides in a VR film.

Subtle Guide – To explain how filmmakers can use audio and visual guides in scripted VR films, Lewis Smithingham, of 30Ninjas, uses a video of a cat following a laser beam.  It illustrates how creative DPs can guide the viewer with sight and sound in a 360-degree film

Nick Bicanic, of Rvlvrlabs, points out that we live in a 360 world, but we see about 170 degrees and hear/sense 360 degrees with our ears and visual clues. That’s what Lewis’ cat/laser example shows.

People don’t wander around spinning 360 degrees, taking everything in; they focus on areas of interest – a conversation, door, car, kiss— as one scene moves us to another. We’re not like a psycho dog racing through the house trying to take it all in.

That’s what Liman/Smithingham did so well in Invisible and what Smithingham does in so much of his work.

The key to great VR is to guide the viewer in subtle ways through the story. Finally, we have the opportunity to control and be an agent to use the medium to entice, help, guide, reward the viewer as never before.

Q – What about the ‘rules’ of VR production, place the camera and let the action, activity move around it and smooth, careful stitching, editing?

Shulkind – You can’t bring rules to a space this new and uncharted; though there is content out there that looks unprofessional, confuses people, or makes viewers sick. But even those results can be used to serve a certain purpose. Nick Bicanic is one of my favorite creators who challenges those unspoken rules.

Cupid Break – Kayde MacMullen (prone) and Dominika Juillet take a break in the shooting of the first VR episode of Cupid, while Nick Bicanic does trial movement of his Z CAM S1 on the Sperica Rover.  Done right, he has found camera movement is a natural part of 360-degree storytelling.

As with so many of his projects, Bicanic was one of the early filmmakers to move the camera and prove that viewers are people and they move around. Now, that’s standard and we all do it because … it’s natural.

Like Bicanic, good VR filmmakers don’t deal with the rectangular-bounded box we call a frame. Blocking is necessary because it guides and provides suggestions to the viewer with our perspective of the storyline.

Bicanic focuses on active ways of engaging the viewer and helping him or her consume and participate in the content that goes beyond sitting on the couch watching it on a flat screen.

Bicanic stresses that having the right computer hardware that’s fast and ultra-reliable is key and that includes having storage (lots of it) as well as careful finesse of workflow and selectively aggressive use of 360 editing tools.

We’re just now getting the hardware/software needed to render with good refresh rates, high frame rates and stutter-free. And Bicanic has put it all to good use with aggressive editing and rendering that would melt or choke “standard” hardware/software.

Editing 360 video right now is a lot like painting a house with half the bristles missing … it requires a lot of work, rework, and attention to the details.

But … it’s getting better, easier and more fun.

Q – You emphasized short form and mobile VR. When will we have long-form VR films?

Shulkind – I think short form versus long form is a distinction of the past. A collective decreased attention span, new advertising models, eye strain, our busy schedules, and the challenging ergonomics of watching mobile devices for long periods of time hearken a coming age where running time doesn’t have to click into fixed segments of time so common in television, cinema, and advertising. There are some outstanding sub 6-minute pieces and a handful of wonderful 20-minute pieces. I think a scaled version of longer run times are more about users spending as long as they like in an interactive, explorable space over repeat viewings.

Gripping Story – Shulkind has viewed dozens of examples of great films that show the power and impact of VR. He feels that one of the best is Alejandro Iñárritu’s Carne y Arena.  He says without the excellent use of VR for the film, it would have been just another good, interesting documentary.  It earned a special Oscar at this year’s awards.

And if you want to see the real power and potential of VR, you’ve got to see Alejandro Iñárritu’s “Carne y Arena” or “Meat and Sand.” VR films of this power and magnitude show the pure potential of the technology.

Both Smithingham and Bicanic have been doing strong VR long form films as well.

Content providers like Hulu and Discovery have been at the forefront in using the technology to tell stories, immerse viewers.

Andrew MacDonald and his CreamVR team have been creating a solid series of 360 video projects for the networks and OTT’s that have led to funding original VR content.

MacDonald understands that entertainment is evolving past the traditional screen and Cream is expanding its skillset into all of the platforms VR is making available.

Post Process – Cream VR’s Andrew MacDonald sets up one of his OWC Mercury Elite storage units to put the final touches on one of the firm’s project segments—Curious Minds—for Hulu.  For complex projects like this, storage has to be fast and reliable to eliminate workflow stutter, speed production and deliver quality video on time.

“We are living a renaissance of modern media,” he noted. “Information is no longer bound by a physical screen and the limits of media are no longer defined by a little box. Now, filmmakers can take a more holistic, cross-platform approach to the content experience.”

Moves like these are being embraced as the industry looks at what is possible. The heads of Turner and Hulu will probably give us a peek at what’s behind the curtain during their keynotes at January’s CES show.

MacDonald doesn’t control the story in the traditional sense and is taking it a step beyond to immersive theater. That’s what VR filmmakers are pushing for today. They know there are “rules,” but they’re really putting them on the shelf and starting from scratch.

How do you keep control of the story?

Or maybe you shouldn’t?

Maybe focus on immersive theater—let the rules go away and just start from scratch.

Really creative, dedicated filmmakers feed off each other.

They learn from each other.

As a result, they get better because of each other.

Shulkind concluded by saying today’s VR filmmaker/storyteller will reset the bar and shape the future of content that is a balance of traditional media, immersive entertainment, gaming, e-sports and live streaming.

It’s becoming the ultimate canvas … a composite experience.

All MacDonald’s three DP (director of photography) friends added was, “Nailed it!”