LEAP Forward - misconceptions (not a mouse replacement).

September 18, 2013

Down arrow black
Melbourne Demons premiers 2019

Karl.

Technology Director


LEAP Motion finally hit the public market in July, and my technology thirst was quenched when my pre-order arrived. I’m not going to write up a review containing the pros and cons and my experience of plugging it in, like you might see elsewhere, but thought I’d express what I think this device means for the human computer interaction (HCI) space and why some people are getting the wrong idea.

After many discussions about this technology, it seems a lot of people think the LEAP is supposed to be a replacement for the mouse or trackpad, or creating a touch screen in the air, and then argue that there’s no feedback and it feels less natural. For me, I think this is completely missing the point, and I’ll explain why below.

I have played a bit with the device, checked out the apps on Air Space (their app marketplace, which I could write a completely separate piece about software distribution), I’ve also checked out the SDK and looked at some of the active open-source community projects enhancing some of the gesture recognition libraries. But the device spends much of it’s time on the shelf. The reason this has not become part of my day to day and a replacement for the mouse, is because the mouse was innovative genius when it came out and a big jump in HCI, and the interfaces we use today on our computers lend their hand to mouse interaction completely.

The LEAP is not in any place to compete with this. There are a few clear reason’s why it can’t. 1. It is tiring to hold your hand up and wave it around all day when interacting. 2. Although the device is very high fidelity with precision of movements, human motion is not as precise as we would like it to be, but can be more accurate when moving an external device, I.e. moving a mouse, or touching a screen. We are not all surgeons. And 3. interfaces we use today are concise, structured and generally high resolution, and small in physical size. The feedback from pressing a physical button on a mouse for example gives us the confidence that we are selecting what we want with the precision we want.

The reason the LEAP is iconic is because this is the first time since the release of the mouse that there has been a large innovation in HCI brought to personal computing, and this potentially can expose other applications which are more suited to the LEAP. When I say first time, I know there is voice control, touchpads, joysticks (excluding touchscreens, as this is a new device type), but I believe these have less potential to make an impact the way the LEAP can. Similarly to the Kinect, Nintendo Wii, and Playstation PowerBall, the LEAP can grow to become in touch with the user on a human or body language level. This does not need to give haptic feedback if the software you are interacting with can reciprocate with human gestures that we’ve grown up with and understand.

If evolution is to occur in the way humans interact with computers to empower the potential of technology. Computers will need to have a better understanding of our movements, human gestures, and body language. This can manifest itself in a new breed of applications including sign language learning tools, health sector assistance technology like gesture controlled operating theaters, and potentially applications or systems that react to your emotion.

This transformation to encompass practical utility using this new method of HCI will not be a quick process and may not occur at all, but opening up the potential is definitely progressive. Interfaces would need to be designed specifically for LEAP interaction and it may require working with other pieces of technology in harmony, i.e. an operating system which includes facial recognition for expression tracking, voice recognition for natural communication, LEAP monitoring for body language recognition, and UI or device feedback returning mood colours or interface skewing to convey an expression. I would also hypothesize that this type of technology and interaction caters for computers communicating with audiences or passers by, as opposed to a personal computer. This is because greater benefits in this automatic engagement are realised by users who have never used the system before, but if it’s your own computer, operations can be optimised by direct keyboard control.

I will leave it there, as I’m getting carried away with HCI. But the bottom line is, the LEAP should be not attempt to be something it’s not, but it’s definitely a LEAP forward in the next era of computing.