The success of augmented reality (AR) mobile game Pokemon Go and the recent reveal of next-generation immersive-media solution Magic Leap One confirm that AR will have a significant role in on-screen entertainment. Its potential use as entertainment marketing and production tools have indicated that, like virtual reality (VR), AR and its sister tech mixed reality (MR) are viable next-generation storytelling formats for reaching and engaging viewers.
In 2016, the location-based mobile AR game Pokemon Go, created by US immersive media and gaming-tech specialist Niantic, enabled players to see, chase and capture virtual 3D creatures that they could see in their real world in real time via their smartphone screens.
“Pokemon Go proved that people chasing virtual creatures that only come to life when viewed through your phone is fun and social,” says Peter Hickman, BAFTA Award-winning drama and children’s producer, digital entertainment and games, at the UK’s BBC Worldwide. “There are already AR games that rely on you being able to move your mobile device around virtual objects and environments to solve puzzles. That really offers a different type of gaming and entertainment experience.”
Karmelina Parouka, vice-president of content and engagement at Viacom International Media Networks (VIMN), also wonders whether AR could become an entertainment format in its own right: “If Pokemon Go taught us anything, the answer is yes — but it just scratched the surface of what will be possible. There is an opportunity for immersive experiences to be stand-alone. We might be surprised by what ends up working in this space in the end, but shared and location-based experiences, such as how we connect within a space and with others, show a lot of promise.”
The ability of AR, MR and VR tech to create a digitally synthetic world in which people can immerse themselves has taken media interaction to a new level. Full-VR users wearing headsets that immerse them completely in this virtual world cannot see anything else, including their physical surroundings. With AR and MR, on the other hand, the technology superimposes objects from a virtual world on to the physical world, which can still be seen. Imagine the characters of an animation series popping out of TV screens to engage with viewers in their living rooms.
The mass-market application of immersive tech is still embryonic, but the possibilities are numerous. Its ability to combine gamification, interaction and personalisation to entertain millions of digital-native viewers is enticing.
While VR has been dominating the mass-market immersive-tech business in recent years, AR appears to be coming into its own in 2018. A report by Vast Media for MIPTV entitled Augmented Reality: Immersive Apps And Campaigns By Broadcasters, noted that the number of mobile-AR users worldwide is forecast to reach three billion by 2021, based on analysis by US consultancy Digi-Capital.
The launch of Magic Leap One shows that AR might be able to offer even more than a mobile experience. The AR headset was unveiled in December after more than six years in stealth mode and almost $2bn in funding from the likes of Google and Chinese e-commerce giant Alibaba Group. Described as ‘built for creators’, Magic Leap One has a controller that enables you to pull content from your computer — anything from a document to an item from an online store — outside the screen and interact with a 3D version floating in front of you. You could, for example, transport characters from your favourite video game from the screen into your real-world living room and interact with them.
The tech sector expects great things from Magic Leap One, which is still being beta tested before going commercial sometime this year. Combine that with the HoloLens (above), an MR headset created by software conglomerate Microsoft that is also still being tested, and it looks as if AR technology has much to offer the media and entertainment business.
The BBC has been extending a number of its brands into the AR space, including Walking With Dinosaurs, the 1999 blockbuster documentary that showed viewers what life would have been like millions of years ago, when dinosaurs ruled the earth. The series’ premise was brought into the 21st century after the BBC collaborated with UK-based Supermassive Games to produce AR video game Wonderbook: Walking With Dinosaurs in 2013.
“For Walking with Dinosaurs, we created a smartphone and tablet app that lets you bring dinosaurs to life on your table top. You can even feed them their favourite food,” Hickman says.
Daniel Wan is lead digital creative (VR/AR) at UK-based Alchemy VR, which has worked with the UK’s Natural History Museum to build immersive experiences linked to the BBC documentary Great Barrier Reef With David Attenborough. “Our clients have backgrounds in museums, charities and cultural institutions,” he says. “These attractions have a real appetite for AR. Not only is AR attention-grabbing for them, it opens up the story of their exhibits in real-world galleries. As our personal phones improve and get more powerful, AR will become mainstream and a natural extension of what we do.”
The resume of US VR agency Moth + Flame includes projects for Google, Facebook’s VR operation Oculus, celebrities Taylor Swift and Karlie Kloss, and TV networks AMC, Discovery Channel and Viacom’s MTV. Founder and CEO Kevin Cornish explains the extra dimension that immersive technology brings to TV content: “With VR, you are looking at something on a screen without edges, so your mind doesn’t know there is a screen. As a result, you remember not only what you saw, but also what you did. If a producer’s idea is to drop and immerse something into a world of visuals, VR is better for that. If you want someone to develop a bond with the characters, either VR or AR would work.”
Cornish says that the TV industry is increasingly embracing immersive content as part of the production, distribution and promotion chain. “When it comes to marketing, now you automatically ask where to include the AR and VR elements, and determine which one serves your show best,” he adds.
The number of VR creative agencies adding AR to their expertise is growing. Los Angeles-headquartered Here Be Dragons, the prolific VR-content studio that has worked on USA Network’s Mr Robot, FX’s Legion and Paramount Pictures’ sci-fi actioner Ghost In The Shell, plans to boost its AR activities following a $10m fund raise last August from Discovery Communications, among other investors.
In December, Jaunt VR, another well-funded VR creative agency, launched its Jaunt XR Platform, which will enable media companies, brands and other rights-owners to create and deliver their own direct-to-consumer VR, AR and MR content.
Meanwhile, tech behemoths including Apple, Google, Facebook and Snapchat have invested in AR technology to enhance the digital experiences they offer to their subscribers. Apple’s most recent iPhone operating system iOS 11 features ARKit, a digital toolset that allows app developers to generate AR content and experiences. And in November, it acquired a Canadian AR-headset developer called Vrvana to further its immersive-media strategy.
Google’s rival version of ARKit is called ARCore, while Facebook’s Camera Effects platform includes AR Studio, a feature designed for building social-AR experiences. Ephemeral messaging app Snapchat has also joined the AR club. To prove what it can do with the technology, owner Snap joined forces with Jeff Koons to allow users in different parts of the world to see gigantic virtual installations of the acclaimed US artist’s works, just by tapping their smartphone’s screen.
Elsewhere, e-commerce conglomerate Amazon is collaborating with smart-glasses maker Vuzix. The US-based company has manufactured the first pair of smart glasses featuring Alexa, Amazon’s voice-activated digital assistant, which display answers to requested information on the glasses’ lenses.
“At this early stage of these technologies, it’s important to consider the pros and cons of each technology and understand if reach [AR] or impact [VR] is the key performance indicator for a project,” says Aurelien Simon, head of immersive at the UK government-backed innovation centre Digital Catapult. “The potential reach of an AR application is currently far greater than VR, but the creativity is limited. CreativeXR, one of our recent programmes, is trying to solve this issue by supporting early-stage creative ideas in AR and VR.”
Michael Boreham, senior consultant at London-based Futuresource Consulting, says the entertainment industry should not understate the marketing strengths that AR can bring: “AR can play a significant role in advertising across multiple applications, particularly through search and discovery, and in the use of marketing campaigns to support products or feature-film launches. For example, the campaign by internet giant Tencent in China to promote the film The Fast And The Furious used AR-based QR codes to unlock content.”
Boreham adds: “AR advertising has also been utilised in a gamification context using Pokemon Go-style games to advertise products and services. Social-media platforms have used AR-based advertising to drive revenue growth, notably Snapchat, which is monetising its Lenses and Geofilters features via advertising.”
Encouraged by the attention being paid by the global tech conglomerates to AR, several tech startups are developing related businesses for the film and TV industries. Augmen.TV, a project developed by Germany-based tech startup eyecandylab, is able to make virtual versions of characters and objects jump out from TV screens, be they in a show or a commercial, via a smartphone app. The company partnered with German network ProSieben to create a dedicated AR app for a one-week marketing campaign to promote weekly magazine show Galileo last November. Via the app, 3D AR objects, graphics and interactive games offered extra information about items featured in the programme.
Another startup is New York-based We Are TV, which creates mobile AR games linked to TV programmes that viewers can play while engaged with the show.
It is also interesting to see other forms of long-established entertainment and leisure activities adopting AR. The Lego Group, the Danish corporation famous for its interlocking plastic bricks, Legoland theme parks, Lego stores and the movies that have earned almost $1bn in box-office receipts, is experimenting with AR. In December, it introduced the Lego AR Studio app built with Apple’s ARKit. Smartphone owners can use virtual Lego bricks inside the app to build animated 3D objects. They then use the phone to produce short video clips of the animation moving around in the physical world.
The professional sports business is also adopting AR to offer fans the type of bespoke personalised content that is becoming increasingly important for retaining their loyalty. An expert is this field is US-based YinzCam, a mobile-software developer that has partnered with most of the top US professional sports teams, including those belonging to American football’s NFL, the NBA, Super Netball and Major League Soccer.
YinzCam’s AR enables spectators to play the same sport on virtual versions of the football field or basketball court via their smartphones during game intervals or while watching the action on TV at home. The AR effect can be triggered by pointing the smartphone at a QR code on a sponsor’s product, such as a soft-drink can.
“You can create virtual goal posts or basketball hoops to play the game on your mobile, or even play against each other,” says YinzCam CEO Priya Narasimhan. “We can show how many fans have engaged with sponsors’ products, thus enabling the teams to go back for some more sponsorship dollars.”
Virtex Arena, a similar AR game offered by California-based Virtex Apps, enabled fans watching this year’s NFL Super Bowl at home or at the stadium to play against each other live for the first time.
For content-owners seeking reach, AR looks best placed to lead the way, since it is much more mobile-friendly than VR, and not dependent on expensive headsets. However, that has not stopped companies such as Moth + Flame from experimenting with the more consumer-friendly aspects of VR. It is one thing to be totally immersed in a virtual world that takes you completely away from your physical real world, but is it possible to have an organically natural conversation in that digitally created environment?
Moth + Flame’s Cornish has joined forces with California-based creative-production agency Tool of North America and VR headset giant Oculus to try to answer this question. They have created Fall In Love VR, an experiment that aims to see whether it is possible to talk to someone and develop a romantic relationship within a VR world.
Cornish believes this could drive VR to “the next generation of cinematic storytelling”. He is also confident Conversive, which recently sealed $600,000 in new funding, will one day answer the following question: “How do you enable a conversation on an immersive platform? When I speak, how do I get the other person in the virtual world to respond naturally?”
As VIMN’s Parouka puts it: “VR and AR can offer transformative experiences and deepen relationships and interactions with consumers. VR is the most immersive, but requires a headset and commitment. Our smartphones, which we carry around at all times, are now capable AR devices.”
This article was written by Juliana Koranteng for the MIPTV Preview magazine, and edited by Kristine Clifford. Read the full digital edition here!
