Sharks or butterflies in the room, and other ways to augment reality

If you’re wondering what AR – augmented reality – has to offer, Edward Miller, co-founder of Scape Technologies makes a big promise: “it gives people high level abilities, almost like superpowers.”

Edward sees AR as offering huge opportunities in a wide range of professions from medicine, to design and engineering.

He was speaking at an event about AR for BBC staff which proved to be a mind-stretching exercise in how new techniques of augmented reality can transform the way the BBC tells stories, whether in fiction, history or news and current affairs.

Zillah Watson, Commissioning Editor for Virtual Reality at the BBC, explained what augmented reality is:

“At it’s most simple, it’s about getting a device - for now a phone - that can read and understand the world and can then augment or change that world with a layer of data so it looks different.”

So the phone becomes a magic lens that changes the world it looks at and can even appear to penetrate deep beneath the surface. 

But there are some hard challenges to make AR work.

Pokemon Go got everyone talking about AR and earned $200 million in its first month. However users can’t yet step into the new reality. For that you need a headset and they are often cumbersome. Over the next ten years the idea is to send the image instead direct to the human eye with a virtual retinal display (VRD). One company working on this is Magic Leap which has already raised a staggering $2 billion in funding before even having a commercially viable product.

Before that kind of technology is ready, work is progressing fast to enable a mobile to recognise what’s in a scene and to react, but Edward Miller said this is very tricky for computers to do:

“We want to have rich digital experiences that are overlaid into our real world in a natural manner and that’s very hard.”

In robotics, this is now being solved with odometry – which is the use of sensors to measure changes in location. The process is called Simultaneous Localisation And Mapping (SLAM) or visual odometry.

Ben Kidd a co-founder of Curiscope sees huge potential in using AR to make exciting educational experiences for children. He showed off a product which, when he pointed his mobile phone at his specially designed T shirt, revealed a moving image of the inside of the chest cavity, complete with pumping heart! He also demonstrated the ShARk app which produced a life size shark that appeared to swim around the room very naturally avoiding objects and members of the audience (see above).

Luke Ritchie of Nexus Studios hopes that AR will soon hit the mainstream. Nexus has developed an app that brings to life the cover of the New Yorker Magazine and builds a 3D image of the White House on any one dollar bill. They have already had success with The Gruffalo Spotter which creates interactive virtual animals which children can follow through any of the twenty-six national forests in the UK. Nexus has also developed the first AR character-based wayfinding app called Hotstepper. It features an eccentric scantily clad character who walks and dances his way to your destination.

The creative opportunities of AR are endless – including the Live Butterflies app (below). BBC producers and journalists are being encouraged to think of innovative ways to use it. 

BBC staff can watch the whole event - Mixing, merging, augmenting: Delve into the world of all things augmented - which was produced by BBC Fusion.