IAN: Aarrhhh.
NARRATOR: 19-year-old Ian was born with cerebral palsy. He lives in Bristol with his mum Eliza.
IAN: 'Spitting toads amazeball… awesome.'
MALE: Hahah!
NARR: Although his mind is unaffected, he has no control of his arms, legs or body and can only communicate with his eyes. His biggest joy is his lifelong love of sailing. But he's never been able to control the boat on his own. Leading this challenge are experience designer Ruby Steel and electronics engineer Ryan White.
DOORBELL RINGS
ELIZA: Hello!
RYAN: Very good to meet you.
ELIZA: Ian's just through here.
RUBY: Hi, Ian!
IAN: Hello.
ELIZA: This is Ruby. And this is Ryan.
RYAN: Hey, good to meet you.
RUBY: Well it's, er, fairly evident that you like music. What kind of music are you into?
IAN: 'Yes…'
ELIZA: It's actually quite hard work.
IAN: '…awesome.'
NARR: Ian uses a communication tool called 'Eyegaze' which enables him to select words with his eyes.
IAN: 'My favourite music is metal, with the volume turned up. 'I like Slipknot, Cannibal Corpse, Napalm Death, and the old stuff like Black Sabbath.'
RUBY: Nice.
RYAN: Right on, man.
RUBY: That's pretty hard core, I have to say.
IAN: Aaagghh…aha!
ELIZA: Some of it is what I call 'noise'.
RUBY LAUGHS
RYAN: What other stuff do you like to do? What other hobbies do you have?
IAN: 'I also love going sailing.'
RYAN: Okay.
ELIZA: So, I think this is getting to the reason why you're…
RYAN: why we're here
ELIZA: - yeah, why you're here.
IAN: Aaayyyaaa!
RYAN: Right on, man.
IAN: 'I like being out in the fresh air and I love the feeling of the movement in the boat.'
RUBY: Yeah, I can totally relate to that.
RYAN: That's supercool. Right on.
IAN: 'I can't wait to get going. I hope you don't mind getting wet.'
RUBY: No.
RYAN: No, I'm all good.
RUBY: See you in a minute, Ian.
RYAN: See you in a sec.
NARR: Cerebral palsy is a condition that affects muscle control and movement, and is usually caused by an injury to the brain in the womb or soon after birth. It affects 1 in 400 children in the UK, but the severity of symptoms vary enormously.
ELIZA: His…affects his whole body, all his limbs and he, he's got movement, but he doesn't really have very much control of his movement, so it's not very purposeful and he's very physically impaired, but cognitively, he's got a good understanding of what's going on.
NARR: Ian started sailing with his dad, Paul, when he was ten years old. Together they enjoyed success in the National Disabled Sailing Championships.
ELIZA: Ian ended up getting his own boat and his dad used to take him sailing. It was their time together. Paul was not just passionate about sailing. He…was an engineer, was passionate on his sports cars and um, Paul…died driving one of his cars. It's, it's…hit Ian really hard. I think partly it was age being 16, but also, you know, when you know that you're dependent on other people as well… so it has been really… really hard for Ian.
NARR: Ruby and Ryan want to see how Ian is able to sail so they've come to Reading where he's taking part in a sailing competition.
ANNOUNCER: …can you please all make your way underneath the balcony for the briefing.
RYAN: How's it going? All right? You're gonna come first, yeah?
IAN: Aahhhh!!
RYAN: Yeah man.
NARR: Tom Holme and his dad Richard are close friends of the family. Tom helps Ian sail by operating the boat and has designed a system to help him become more involved. You got this thing, of course there is no easy way for Ian to communicate with me and tell me what to do, other than with his eyes. And because I'm sat next to him, I can hardly see his eyes, and so I basically, er, just came up with the idea of a mirror.
RYAN: Okay, and there's a whole bunch of velcro on the back so you can stick…
NARR: Tom puts directional stickers on the mirror. Ian can then indicate what he wants Tom to do next, by looking at them. I struggle with it because if you're not looking… if Ian's trying to tell you something and you miss it…well you miss it.
RYAN: Good luck, both. I'm fascinated by the mechanics of it. Really excited to see how it all works.
RICHARD: Where possible, Tom will be able to ask Ian a question and Ian can answer and Tom can see his answer by looking in the mirror to see what his eyes are doing, but it's still fairly crude because you don't really have the time to have a conversation about what to do.
RUBY: Yeah.
RICHARD: So, to give Ian absolute control and responsibility would be brilliant.
RUBY: I'm still really concerned about how extreme Ian's condition is, and what he wants to do. He's kind of…
RYAN: 'this' far apart and we need to bring them together.
RUBY: Yeah.
RYAN: There's a risk that they've got the best system that there is at the moment: you know, a competent sailor and a mirror with some stickers on it.
NARR: Back in London, Ryan and Ruby begin brainstorming possible ways in which Ian could control his boat.
RYAN: Bare minimum…two motors driven by Ian.
NARR: They want to fit Ian's boat with small motors which will control its direction.
RUBY: So, putting motors on the boat: fairly straightforward. What's not straightforward, and the main risk, is how we get what's in Ian's head… into actions.
RYAN: What do we know about Ian's vision?
RUBY: I think it's…
RYAN: Good?
RUBY: I think it's good.
NARR: Ruby and Ryan want to know if it's possible for Ian to use the same Eyegaze technology he uses to communicate… to steer the boat.
JOHN: So, there's going to be some points appear on the screen. And if you could just follow those…
NARR: Ian's system is a few years old, so Ruby and Ryan have invited John Liddle, a specialist in assistive Eyegaze technology, to help them test the latest tech and see if it will work outdoors.
JOHN: Could you just take a moment to look in each other's eyes? LAUGHTER
RYAN: Yeah. You may or may not be able to see a glint, a white reflected light source from the window or something in each other's eyes.
RUBY: Yep.
RYAN: Yeah.
JOHN: Well, what we do with these illuminators is cast, using near-infrared light, a very similar glint somewhere onto your cornea.
NARR: Eyegaze uses cameras to pick up the eye's glint. It can then determine where a person is looking to within an accuracy of 1 degree.
JOHN: Do you want to try it outside then?
RUBY: Yes please! Let's see if it works outside.
ELIZA: Woah, there we go, Ian.
RYAN: So this is still pretty bright, I mean my eyes are still adjusting.
RUBY: Yeah, me too.
ELIZA: Yeah.
RYAN: Let's see if we can get it to work. Shall we see if we can get 'cat', up here.
ELIZA: Okay, Ian, can you look, look back and see if you can find that horse?
RYAN: I so want this to work.
JOHN: Let's just try this.
RYAN: Nothing. Oh, it's so close!
JOHN: If you were to step over there…
NARR: Outside the sun floods the cameras with infrared light, confusing the system.
IAN: 'That is not helping.' THEY LAUGH
RUBY: You're right, it's not helping.
RYAN: Eyegaze doesn't work outside. It can't track the user's eyes. It's clear that it's not, in its current form, good enough to drive a boat with. It's as simple as that. We need to find something else.
NARR: In London, Ruby and Ryan have been researching various forms of assistive technology to help Ian independently steer his boat. They've discovered a variation of the Eyegaze tracking system used in an innovative way.
SCOTT: This is a wearable eye tracker.
NARR: Scott Hodgins has been working with Eyegaze technology for over a decade. He's brought with him a pair of eye-tracking glasses with cameras built into the frame.
SCOTT: So, we've got a couple of cameras pointing at each eye and then from there we compute where you're looking. This is a full high definition camera in the front. You are ready to go.
RUBY: Man, this is so cool.
SCOTT: Okay.
NARR: Eye-tracking glasses record live data to show exactly where the user is looking. They're often used by market research companies to determine where shoppers gaze, giving them information on product placement and advertising.
RYAN: So I'm really excited to try these outside. I know it's a little bit cloudy today.
NARR: As the cameras are close to the eyes, Ruby and Ryan hope they can add filters to reduce the amount of ambient light flooding the cameras.
RYAN: So it's working when you're not looking straight into the brightness of the sky.
RUBY: Yep. This is really exceeding my expectations right now.
NARR: Ryan has an idea that he thinks might help Ian to steer his boat.
RYAN: I've knocked this board up with some QR codes on it. If the computer sees that he's looking in the direction of that particular card with a special code on it, then it enacts some action on the servos in the boat.
NARR: On and off over the next two months, Ryan works closely with computer engineers to build a software programme to recognise the barcodes, so when Ian looks at them while wearing the glasses, he can choose to steer the boat left or right.
RYAN: If I look at that, it goes green, if I look at that…it goes green. It's like… it's really…powerful system.
RUBY: Boom, boom!
CHEERING AND APPLAUSE
NARR: Ruby and Ryan are ready to test their Eyegaze glasses on Ian, in conditions which emulate being out on the water.
RUBY: We might actually get to a point where Ian looks at something and moves an entire boat in the direction that he wants to go in. Hi guys!
RYAN: How's it going? What we've got here are some Eyegaze tracking glasses.
ELIZA: Ooh!
RYAN: When you look at one of these things, what will happen is, it'll light up
RICHARD: And that'll then indicate to Ian that he has triggered that button?
RYAN: That's exactly right.
RUBY: So, these are not intended for anything like this…RUBY AND ELIZA: at all!
RYAN: The last build of this software we ran at 2:30 this morning.
ELIZA: So, you've not been to bed yet!
RYAN: No - well I have…but. My kingdom - for you to look right there in the middle, okay?
IAN: Aaaooouuuhhh!!
ELIZA: Glasses.
RUBY: Have you got that…?
RYAN: So, I'm going to calibrate it now.
ELIZA: Take a deep breath and just chill.
RYAN: Okay, calibration successful.
RUBY: Oh!
RYAN: So, now can you try and make this one light up?
RICHARD: To the left, then just hold it there.
RUBY: Ooh!
RYAN: And I can tell you're really concentrating. Wow, that's awesome.
RICHARD: This time, Ian, here we go.
RYAN: Head up, chin up, chin up. That's good, that's good, that's good.
RUBY: Ohh. So close…
SCREAMS OF EXCITEMENT
RUBY: YES!
RYAN: I'll now go…
RUBY: Ryan, don't jump in the water. That was brilliant! That was really good!
RYAN: Yes!! I've never been so relieved in my life. I mean, that's… I've been making stuff with my hands and my mind since I was six years old, right? The feeling of making something and having it work and testing it and trying it for the first time and seeing it do what it's supposed to is, for me…euphoric, it's the most amazing feeling that you can feel. We don't have a solution yet. This is not a working system. It's a prototype and that's what we were here to show today and that's what we were here to test today and the test, it's passed. It's perfect. It's awesome.
NARR: Ruby and Ryan now need to fit servo motors in Ian's boat to see if the technology can enable him to steer it on his own. Six months after they began work on Ian's fix, Ruby and Ryan are about to discover if their invention can help Ian fulfil his dream of steering a boat.
IAN: Hey!!
RYAN: Good to see you man.
RUBY: Good to see you!
ELIZA: We've been waiting for this! Ah…
IAN: 'I am super excited.'
RYAN: Yeah me too, man. Can we show you how it works? Do you want to try these on?
RICHARD: Yeah.
RYAN: Cool, all right.
NARR: They've added filters to the glasses to keep out more of the light and improve the accuracy of Ian's gaze.
RICHARD: Wow, he looks like Harry Potter!
ELIZA: He does!
RYAN: Should we get out on the water? Yeah, lets. I'm getting nervous now!
LAUGHTER
NARR: If Ruby and Ryan's plan works, it'll be the first time anyone has ever steered a boat with just their eyes.
RYAN: Nervous and stoked.
RICHARD: Okay, hold it there!
NARR: As always, to ensure he's safe, Ian will be accompanied by a sailing partner. On this occasion, it's his friend Richard.
RUBY: It's super windy. There's a storm coming in, so everyone's a little bit on edge. We are really pushing it to the limit of what it's able to do, and I'm not… totally convinced it's going to work.
RYAN: It's as good as it's going to get. We've got a good view of Ian's eyes for the minute. And…it's now or never.
ELIZA: Ian!
RUBY: Good luck, Ian.
See you later!
IAN: Yeahhh.
RYAN: Tear it up, man.
RICHARD: Now I'm going to plug in your glasses. They're not plugged in yet.
RICHARD: Okay, Ian. You have control. Okay, Ian has control. Right…right…
RUBY: We want green lights. Green lights means it's turning.
RICHARD: Ok, here you go.
RUBY: Oh, it's red!
RICHARD: He's doing it.
IAN SQUEALS, THEY LAUGH
RYAN: I heard that.
RICHARD: Look, no hands!
RUBY SQUEALS
RICHARD: Yeah they did, they saw that!
RYAN: Yesss!
RICHARD: They saw that.
RYAN: Ian, give me one more of them, man.
RICHARD: Go left, go left.
RYAN: Seriously…I love that colour green.
RICHARD: There you go, that's it. It's a lot of concentration but, you know, Ian's doing it.
ELIZA: Oh! Agh!
RUBY: Is that you?!
RICHARD: That was Ian, not me!
RUBY: I know, I can see!
ELIZA: I think… I think he's enjoying it, don't you!
RUBY: That was fantastic! I just got like… a proper view of his face, and he was just grinning from ear to ear, just like, "Look at me mum, look at me!"
UPLIFTING MUSIC
NARR: Ryan and Ruby's fix will be permanently housed in Ian's boat, and with Richard's help, he'll be able to use it on all of his future sailing trips.
RUBY: It's the first time that anyone has ever sailed with their eyes and it was Ian and I'm so happy it was Ian.
RYAN: Such a privilege. An absolute privilege.
Two designers develop assistive technology to help Ian, with cerebral palsy, sail independently.
Engineer, Ryan White, and experience designer, Ruby Steele, team up to see if they can come up with a solution.
If they’re successful it’ll be the first time anyone has ever sailed a boat using just their eyes.
They’re hopeful that their idea to use the eye gaze technology that Ian already uses to operate his computer will work if they hook it up to motors on the boat.
See how designers research and develop a new product.
More from The Big Life Fix
A smartphone fix to help Greg. video
A new mobile phone design helps with Greg's paralysis
Helping Ayala play. video
A new robot is designed to help Ayala play with her sister
Helping Kyle follow his dream. video
A product designer creates a prosthetic to help Kyle cut hair
