Storytelling in medium of film and television is well understood, having been developed over the last century with a large body of work to draw upon for inspiration. What happens when you move to the new immersive mediums of VR/AR where the viewer is no longer constrained by a 2D rectangle, but is in a 360-degree environment? Even in a pre-rendered experience, each viewer will experience the content in their own unique way.
Take for example the difference between how my old mother and my son experience VR. When my mother first put on a headset to see one of our films, she stared straight ahead as if she were looking at a traditional television set on the wall. I had to nudge her and say “mom, take a look around” before she moved her head at all. Even then, most of her viewing took place within a very limited field of view.
On the other hand, as soon as I put the headset on my son, he immediately started looking around the environment, trying to interact with objects and characters. He was mesmerised by small effects like snow falling or being able to make the grass sway when he passed his hand over it, often missing important beats in the storyline.
Traditional concepts of composition and framing are turned on their heads when you remove the frame itself. Cinematographer tools like focal length and lensing, also need to be rethought. The omission of these tools does not mean that composition, staging and shot selection do not matter. On the contrary, it puts more emphasis on the collaborative process of film making since every department, from previz to lighting to animation, needs to lend a hand in directing the viewer’s eye as they compose the shots themselves.
As an industry, we are just beginning to develop the language of storytelling in immersive media. No one really knows anything yet. At Baobab, we have been experimenting with different techniques in each of our films, trying to capture and direct the viewers’ attention, while maintaining a strong narrative experience. I’d like to share some of our initial experiments so far.
Think Like a Magician
Magicians use eye contact to both engage and direct the audience’s attention. This allows them to perform “magic” by using sleight of hand to accomplish the trick, while directing the audience’s attention elsewhere. The age-old adage holds that if you want the audience to look at you, look at them. If you want the audience to look at something else, look at it and they will follow your gaze.
In our first film, Invasion!, you meet an adorable white bunny rabbit named Chloe. When you first meet Chloe, she comes bounding up to you, acknowledging your presence by looking directly at you, then hopping back and forth like she wants to play. Chloe immediately captures the viewers’ attention, with big expressive eyes in close proximity to their location within the environment. When two alien invaders arrive, Chloe tracks their arrival by first looking up when you hear a sound, then tracking the movement of the ship. The viewer naturally follow’s her gaze up without having to be forced.
Give Them Something To Do
Unlike 2D formats, where you don’t expect to be able to talk with the actress on screen or pick up the soda bottle she just set down, immersive media naturally invites the audience to interact with the characters and set of the film. There is no barrier of physical space between you and the screen. No seats in the movie theater or living room floor to the wall. You feel like you are in the film itself.
In our second film Asteroids!, you follow the two aliens you met in Invasion! into their spaceship where you play the role of a menial task robot. While you are not the main character in the story, the audience has the option of actively participating, which changes how the two aliens and their robot dog relate to you. There are various interaction points throughout the ship such as opening the windshield blast doors, playing fetch and operating a defibrillator (spoiler alert). This has the effect of engaging the viewer, drawing them to the right part of the ship at the right time in the story.
Re-imagining cinematography in an immersive environment is a new and exciting challenge for storytellers. Will new concepts and techniques come from storytellers with deep experience in traditional media or by brand new artists with no preconceptions? My bet is that it will be a mixture of new and old, and I’m personally very excited to see what the VR/AR creative community comes up with over the next few years as we develop a whole new language for immersive storytelling.
More about Generation Z and immersive content at MIPTV 2018
Jonathan Flesher speaks on the “True VR: Gearing up for an Interactive Watching Experience” panel at MIPTV, April 10. Full details here…
These posts are produced in partnership with Red5 and VR event Virtuality. More soon!
Image: Asteroids! © Baobab
