So, some time ago, a company called LEAP Motion produced a demo video that looked truly incredible.
They claimed to have a new piece of technology, a tiny little box, which would track your hands and usher us all into the world of Minority Report.
Like a lot of folk, I bought one. In my case, I bought one specifically because motion-capturing hands is a Hard Problem: it’s possible to get inertial motion capture gloves, but they’re not cheap for $10k values of not cheap, and in my opinion the output quality isn’t that good.
The big day arrived. The LEAP Motion was unboxed. I tested it out, and lo!
It wasn’t much cop.
Wobbly, laggy, inaccurate. Back into the box it went, and that was the last I thought of it. Until two days ago, when the company behind the LEAP – how to put this?
Leaped into action.
Yes, in defiance of the normal “release, be crap, go bust” cycle, the guys and girls behind LEAP have been quietly working away at making it better. And amazingly, they claimed that they’d fixed most of its problems without even forcing us to buy a new device. Instead, they just had a new software release called ORION, and they claimed that installing it would make the LEAP work as promised.
One frantic dig into the Big Box O’ Unused Tech in my cupboard later, I was ready to test their claims.
A Great Leap Forward
Two spoilers. One, there are going to be a lot of “leap” puns in this post.
Two, yep, it works now.
Not perfectly, for me, so far – it’s a little wobbly during the daytime, as the tech might be good, but it uses IR light to track, and there’s a damn big IR emitter outside my window which kinda fucks with that during the daytime.
In addition, when using it for VR, I heartily recommend clearing your desk before beginning. With a headset-mounted Leap the minimum close tracking distance isn’t particularly close. However, it DOES work fine at arm’s length. Unless what’s sitting at arm’s length is an expensive condenser mic, in which case you will a) fail to grab the block you’re reaching for and b) punch your microphone.
On the downside, calibration is about as much fun as it is with most optical devices. For anyone who has never had that particular joy – it’s not much fun. Essentially, you wobble the LEAP around and point it at reflective things, waiting for the Magic Software to tell you it’s calibrated. The first few times you try it, it won’t be.
Tip if you’re doing this – rotate the Leap, don’t just move it around. Also, I had to iterate through about three allegedly reflective surfaces to get one that worked – ironically enough I ended up using the screen of my Surface Pro.
But it’s orders of magnitude better than the last time I tried the Leap. I duct-taped it to my Oculus DK2 (the Leap site recommends their proprietory adapter, but if you search around a bit you can find their unofficial mount thread including duct tape instructions.
Mine did not look as neat as the one in their official thread, but it worked fine:
I tested out their “Blocks” app, which allows you to pick up and throw around blocks in a virtual environment, using your hands. No latency I could feel, and I could throw blocks around and build towers in VR fine.
Very impressed indeed.
Later in the evening, once the irritatingly persistent IR emitter in the sky had gone away, I gave it another shot (whilst also standing up to avoid the punch-the-mic problem). It worked extremely well, even better than before, to the point that I was getting some “presence” in VR.
I did try to shoot a short video of the process, but unfortunately the LEAP does rather hog the USB bus, and as a result didn’t play at all well with my webcam. So here’s a video from someone else who clearly got over the problem:
I did a bit of more rigorous testing with the LEAP’s debug visualiser, trying to find hand positions it didn’t like. Amusingly, the only one it seemed to have trouble with was the classic middle-finger-in-air gesture – however, it would appear that problem’s not universal.
It did still struggle a bit with my girlfriend Becca’s hands – she has rheumatoid arthritis, and so her hands don’t move perfectly and are semi-locked in a position with the back two fingers curved. It did much better than when it was released, but still, the tracking didn’t work as well as it did for me. Obviously, this is an edge case, but it’d be good to see it improve, particularly given how useful VR is for various accessibility / disability applications.
Unfortunately it seems that the LEAP Orion improvements don’t carry over to all of the LEAP’s app store. I tested out their non-VR sculpting tool as well, but was almost completely unable to get it to play nicely. In short, it seemed to be working about as well as it did before the update.
However, in the new VR apps, it’s extremely cool and very usable. And some of the showcases that we’re seeing coming out of the VR community look super-cool – I’m particularly looking forward to the various wizard simulators.
Overall – this is a remarkable improvement to what was formerly a neat but pretty useless toy. I’ll definitely be hooking it up to their Motionbuilder plugin for some testing as soon as I get the chance…
And if you’d like to be updated on what I’m doing, testing and making on a semi-regular basis, click this here linky thing.