FreePIE Wiimote iR plugin
Posted: Mon Sep 30, 2019 7:09 am
Hi everybody,
I've been lurking for a long time (since the article came out around 2010 about EDID overriding the first generation passive 3D TVs for NVidia support ) but have finally made an account to join in the discussion. I can see the enthusiasm for DIY and budget VR is still going strong here, so I thought I'd say a little about what I'm working on, and also seek some help. TL;DR at the end.
I, as well as many others, are working on unifying input systems to deliver the best VR experience depending on what hardware is available. I'm sure you all know about PSMove on PC, or RiftCat for cell phones. I'm trying to take things a step farther using things that may be laying around the house or obtained cheaply, such as Wii-remotes or Kinect or Windows/Android phones.
The current stumbling block is using wiimotes for 3-DOF. It's common knowledge that the Wii tracked pitch and roll, perusing years of forum posts makes it apparent that not many know about the "motion plus" addition a few years after the Wii's release that added a gyro to track yaw as well. Eventually Nintendo started selling their wiimotes with motion plus built in, which was a headache for early developers because of the different bluetooth protocols they used- this is no longer an issue with Windows 10's bluetooth stack (or a MayFlash Dolphinbar, which I'll get to).
FreePIE has native support for MotionPlus, but it doesn't have support for the iR camera in the wiimote which can fix 'yaw-drift' when a 'sensor-bar' is present. Glovepie, on the other hand, had sensor-bar support but didn't support the wiimotes with built in motion plus (called -TR due to the updated model name) and has long since been discontinued. The Dolphinbar mentioned above is a bluetooth adapter and iR light source for all versions of wiimotes.
Using a Dolphinbar and -TR wiimotes in my development and playing environments, I am receiving very good data from the wiimote into Freepie. Once calibrated (sometimes 2 or 3 times for the best initial calibration) the values are very stable, and the yaw tracks remarkably well for only having itself as a reference point. I mean it's really truly impressive, you could use it as a mouse for internet browsing with no trouble. Until you start hacking away at an enemy or a box in Beat Saber, then even with perfect calibration you'll be spinning on the yaw axis faster than the wiimote can register, and wind up with your controller and cursor in different directions.
Now the good news for many is that the motion plus add-on module is functionally identical to the wiimotes with it built in, aside adding unnecessary bulk and weight to the controller. The downside is that GlovePIE is dead, and while you may be able to script something together that works today, it may not tomorrow or ever again and we will focus on a platform that includes support for the most modern hardware (-TR) and software (FreePIE) to make VR accessible to all.
Now comes the call for aid. There are so many Wii remote libraries available, but plugin development is outside the experience range that my wife and I have (she's a developer for a very large corporation, I've been homebrewing tech for ~20 years). We looked into it over the weekend and we think we're going to need some outside help. Having a Dolphinbar is great for having a known-good bluetooth adapter and a sensor-bar for the current retail price of $19.93 USD via Amazon, but the plugin will not depend on having one; once synced to the PC we only need to get the iR camera to track an infrared light source and allow a button press to reorient to e.g. 0/180, in case you're playing aimed away from the sensor-bar.
Now for those wondering how they can use this without compromising their current setups, this should be fully compatible with whatever you're already using, GearVR, Trinus, VRidge Riftcat, OSVR, ALVR, Driver4VR, Kinect2VR to name a few, even actual Vive users who have broken one remote and want to use a wiimote. I'm not suggesting you invest heavily into this if you already have or can cheaply purchase a PSMove setup, but this has more mappable buttons and should provide more accuracy than the magnetometer in PSMove so if you have the hardware collecting dust, this may provide a superior experience.
TL;DR:
We need help making a FreePIE plugin that will allow wiimotes to utilize their built-in infrared cameras. With 3DOF perfected, a Kinect or PSMove camera will make them 6DOF controllers for a sliver of the cost of Vive/Index controllers.
I've been lurking for a long time (since the article came out around 2010 about EDID overriding the first generation passive 3D TVs for NVidia support ) but have finally made an account to join in the discussion. I can see the enthusiasm for DIY and budget VR is still going strong here, so I thought I'd say a little about what I'm working on, and also seek some help. TL;DR at the end.
I, as well as many others, are working on unifying input systems to deliver the best VR experience depending on what hardware is available. I'm sure you all know about PSMove on PC, or RiftCat for cell phones. I'm trying to take things a step farther using things that may be laying around the house or obtained cheaply, such as Wii-remotes or Kinect or Windows/Android phones.
The current stumbling block is using wiimotes for 3-DOF. It's common knowledge that the Wii tracked pitch and roll, perusing years of forum posts makes it apparent that not many know about the "motion plus" addition a few years after the Wii's release that added a gyro to track yaw as well. Eventually Nintendo started selling their wiimotes with motion plus built in, which was a headache for early developers because of the different bluetooth protocols they used- this is no longer an issue with Windows 10's bluetooth stack (or a MayFlash Dolphinbar, which I'll get to).
FreePIE has native support for MotionPlus, but it doesn't have support for the iR camera in the wiimote which can fix 'yaw-drift' when a 'sensor-bar' is present. Glovepie, on the other hand, had sensor-bar support but didn't support the wiimotes with built in motion plus (called -TR due to the updated model name) and has long since been discontinued. The Dolphinbar mentioned above is a bluetooth adapter and iR light source for all versions of wiimotes.
Using a Dolphinbar and -TR wiimotes in my development and playing environments, I am receiving very good data from the wiimote into Freepie. Once calibrated (sometimes 2 or 3 times for the best initial calibration) the values are very stable, and the yaw tracks remarkably well for only having itself as a reference point. I mean it's really truly impressive, you could use it as a mouse for internet browsing with no trouble. Until you start hacking away at an enemy or a box in Beat Saber, then even with perfect calibration you'll be spinning on the yaw axis faster than the wiimote can register, and wind up with your controller and cursor in different directions.
Now the good news for many is that the motion plus add-on module is functionally identical to the wiimotes with it built in, aside adding unnecessary bulk and weight to the controller. The downside is that GlovePIE is dead, and while you may be able to script something together that works today, it may not tomorrow or ever again and we will focus on a platform that includes support for the most modern hardware (-TR) and software (FreePIE) to make VR accessible to all.
Now comes the call for aid. There are so many Wii remote libraries available, but plugin development is outside the experience range that my wife and I have (she's a developer for a very large corporation, I've been homebrewing tech for ~20 years). We looked into it over the weekend and we think we're going to need some outside help. Having a Dolphinbar is great for having a known-good bluetooth adapter and a sensor-bar for the current retail price of $19.93 USD via Amazon, but the plugin will not depend on having one; once synced to the PC we only need to get the iR camera to track an infrared light source and allow a button press to reorient to e.g. 0/180, in case you're playing aimed away from the sensor-bar.
Now for those wondering how they can use this without compromising their current setups, this should be fully compatible with whatever you're already using, GearVR, Trinus, VRidge Riftcat, OSVR, ALVR, Driver4VR, Kinect2VR to name a few, even actual Vive users who have broken one remote and want to use a wiimote. I'm not suggesting you invest heavily into this if you already have or can cheaply purchase a PSMove setup, but this has more mappable buttons and should provide more accuracy than the magnetometer in PSMove so if you have the hardware collecting dust, this may provide a superior experience.
TL;DR:
We need help making a FreePIE plugin that will allow wiimotes to utilize their built-in infrared cameras. With 3DOF perfected, a Kinect or PSMove camera will make them 6DOF controllers for a sliver of the cost of Vive/Index controllers.