Page 1 of 1

DIY Vive Controller using Leap and Arduino

Posted: Thu Dec 01, 2016 7:51 pm
by flo2
UPDATE: Working source code
I've managed to pipe the data from an (arduino) orientation sensor, leap, and oculus into freepie and emulate a HTC Vive Controller using the Razer Hydra drivers:

Here's the overview of how system works:

Currently it's just raw data, but I'm trying to do the math transforms of the various position/orientation data and having a lot of trouble importing numpy into freepie.

I've placed it in the pylib folder of FreePIE install, but get the following error when I do "import numpy as np"

Code: Select all

cannot import multiarray from numpy.core
Any advice in getting the matrix math functions from numpy working in FreePIE?

The hope is I can position the emulated Vive controllers as a function of the HMD position, orientation, and Leap's hand data. From previous experiments, using orientation data in conjunction with the Leap positional data produced very good results:

Re: DIY Vive Controller using Leap and Arduino

Posted: Fri Dec 02, 2016 1:33 am
by flo2
I got around needing numpy by finding premade functions around the web using standard python. My linear algebra knowledge is almost nil and the code has a lot of issues, but as a proof of concept it functions.

The position of the right controller is a function of the HMD's position + orientation and the Leap's right hand position. I'm placing the code here if someone figures out the transformations faster than I can. I have a lot of linear algebra to learn.

Code: Select all


EDIT: 12/7/2016 - code was fundamentally incorrect in too many ways to risk confusion, and it was an eyesore

Re: DIY Vive Controller using Leap and Arduino

Posted: Sat Dec 03, 2016 6:57 pm
by flo2
Improved code, explanation, and request for help: ... formation/

Re: DIY Vive Controller using Leap and Arduino

Posted: Wed Dec 07, 2016 2:42 am
by flo2
It works!
There were many pitfalls (e.g no documentation on the hydra driver, and I had no idea how it calibrated so everything was inverted for a while) but I hope someone has these numbers. What's the distance between the hydra base station and eye-center when re-centering the hydra base to 0,0,0?

If you look closely at the gif, you'll see the hand be stuck inside a "virtual bounded box." That's the Interaction Box function of the Leap sensor: ... action-box

Unfortunately, the author of the Leap plugin for FreePIE is no longer active as this was made over 2 years ago. All that would be needed is output the raw hand position data for the full range of motion the Leap captures. I hope this is a simple fix, but I know nothing about writing plugins. If anyone could help here, that would be awesome.

Next up is calibrate the Arduino-IMU orientation, and map the joystick+buttons. Should be in a playable state before the weekend :)

Re: DIY Vive Controller using Leap and Arduino

Posted: Wed Dec 14, 2016 12:51 am
by flo2
Project files for DIY HTC Vive motion controllers, using the Leap Motion mounted to VR headset for positional tracking:


Over the weekend I finished 3D printing a second DIY motion controller and playing a few Steam VR apps. There were a few glitches that were very specific to developer implementation, but overall very satisfying to finally have access to the Steam VR stuff with motion controllers. Without any industry standards for VR controllers, your mileage will vary in app support. Rec Room didn't recognize the pickup action, but paintball was fun.

While researching and reading through forums I found a lot of people with wii nunchucks trying to do this. With a few edits to the FreePIE script, it should be easily converted to any device/buttons/orientation data they want. The hard part was figuring out the 3D transforms, now it's just a matter of mapping the data.

More info and photos: ... -emulation

The biggest limitation is the 2014 FreePIE plugin for the Leap Motion. It outputs the normalized (and limited) hand positions of the interaction box: ... action-box. Developer thread:

The x,y,z positions of Left and Right hands are given in the InteractionBox's normalized space instead of the raw mm values of the full FOV from the Leap sensor. Until the driver is updated/recreated we're stuck using the limited InteractionBox volume.
If anyone with Windows DLL plugin development experience thinks it's straightforward, this driver could enable a flurry of VR applications for FreePIE community. Thank you again for everyone here helping out and the developer of FreePIE for such an awesome tool.

Re: DIY Vive Controller using Leap and Arduino

Posted: Wed Dec 14, 2016 7:55 am
by MeteorFalling2
Did you manage to figure out how to use numpy? I'm trying to smooth out the bumps in the y axis using the Wiimote and it seems like I need it as well.

Btw, i like your project. Motion controls really are special. I wish there was more documentation on FreePIE as it is hard to figure out how to do things.

Re: DIY Vive Controller using Leap and Arduino

Posted: Wed Dec 14, 2016 12:15 pm
by flo2
I couldn't get numpy to import. I rewrote the matrix transformations in pure python. It wasn't that bad since all I needed was a matrix multiplication function and the rest was performing the transformations in the correct order. Stepping through it helped me really understand and debugging was easy since only one function touched the data.

Documentation is a bit sparse but the source code is all there so it was straightforward to answer my own questions.

Re: DIY Vive Controller using Leap and Arduino

Posted: Wed Dec 14, 2016 9:03 pm
by cybereality
That's awesome, flo2!