Meant to be Seen

I've mentioned a few times that my wife and I have implemented a character movement scheme using a balance board, so here it is!

[youtube]http://www.youtube.com/watch?v=IOTnRoi1HDA[/youtube]



In this video you can more clearly see the variable speed that is possible, it's not as smooth as it is when it's dialed in for play, but it can be quite smooth, not quite analog smooth, though I am sure that is a problem that can be fixed.

[youtube]http://www.youtube.com/watch?v=vNygP13hNYg[/youtube]


We haven't done any work on this in over 9 months as we got as far as we felt we could without an HMD, which will change the experience significantly.

***update may 2***

We've had our Rift for a week and done some more work on this, we've implemented the scheme into the Tuscany VR demo scene that ships with the Rift. Once we get a few Unity build issues figured out we will be releasing this as a demo so others can try it out with their Rifts and see the potential of the balance board!

[youtube]http://www.youtube.com/watch?v=OlSRSfHaRaQ[/youtube]

I was hours away from starting such a project, and here it looks like you've already finished it! I'm rather surprised at how intuitive and precise it appears; this really could be a major component of VR control. Well done to the both of you! Does it also work while seated? And have you experimented with mapping the X axis to strafing, tying turning to a thumbstick or something? I've given up assuming what will feel proper inside a Rift until I actually have one to experiment with; different approaches are likely to be preferred by various factions, and you've definitely anticipated one of them here.

Is this all Input Manager and custom scripts in Unity, or is GlovePIE or something serving as middleware? Are you planning to release any code from this project, or make it a unique centerpiece of some games? I really don't think I could ever refine this concept past the point you've reached, but I may take a crack at it if necessary. Any pointers, at least?

I'm convinced foot control is going to be widely adopted over the next few months, and Balance Boards are cheap, widely available, and (apparently!) quite capable. I look forward to seeing where you go with this.

Interesting concept. Looks like its work OK too.

I haven't actually tried it seated yet, but my wife is doing that just now lol. The scheme is really designed to measure and interpret balance, not just pressure, so I am skeptical if it will work for seated use.

I wanted to have a foot control that freed hands from the task of movement completely, so no I hadn't considered mapping any movement to a key, other than jump as I don't like the idea of jumping up and down while my view of the real world is completely blocked out by a wired device, besides how many of us mere mortals can jump like a game character can jump? I imagine it would be pretty weird to do a little jump and land while your character is still in the air.

Ok she just finished testing it while seated, and it works very well, however it could be optimized for such use. You do lose the inner ear sensations from using your actual balance, but she says it's actually easier to use.

This implementation is in UDK, we have not moved this to Unity yet, it is currently a .dll of the WiiYourself library and some Unreal Script, we also had some Kismet nodes as well at one point but I don't think they currently work.

Not sure what we will release yet as there are a ton of rough edges, particularly surrounding bluetooth and battery usuage. The board does some mighty silly things when it gets low on power and it's not always obvious that it's low on batteries. Wii and Bluetooth are also temperamental bed fellows from our observations so far. While VR enthusiasts have no problem jumping through a few hoops to get a peripheral up and running, the average game player won't be nearly as forgiving.

Brilliant, consider me thrilled. I just picked up a used BB to start my own experiments yesterday, so let me know if you need an additional tester.

You've probably already pursued this, but one thing that's helped me mitigate some of the maddening Wii+Bluetooth problems: Try the Toshiba stack. Microsoft's is a pile of frustration and sadness and I've read BlueSoleil is so demanding about compatible hardware I avoided it entirely, but the Toshiba stack seems about as stable as these things get and relatively forgiving of the weird controllers and commands I've been throwing at it.

Nice implementation. It'll be interesting to see how balance is affected (if at all) when you're wearing the Rift.

Awesome! Let me know as well if you need another beta tester! Can't wait to try this in combination with the rift, the hydra and the posittron

Finally it seems like all the VR pieces are falling into place.

Good stuff! Glad you posted these videos.

So it seems to use tanks-type controls for turning (on foot forward, one foot back), correct?

I would definitely like to see a PC equivalent of the Wii balance board. I still can't help but to think additional yaw sensors for each foot would be of benefit as it would allow for a wider variety and separation of control "gestures" that could be used.

great device! definitely a perfect combination with the Rift for who can't afford Omni (cause I guess Omni will be more expensive than a Wii Balance Board, sorry if I am not correct) and I will definitely buy a Wii Balance Board just for this.

MSat wrote: I would definitely like to see a PC equivalent of the Wii balance board.

Wii fit balance board can be bought under $50 on eBay, a whole Wii with a balance board and controller can be had under $100 on craigslist, why make a new product?

QUAKE wrote:

MSat wrote: I would definitely like to see a PC equivalent of the Wii balance board.

Wii fit balance board can be bought under $50 on eBay, a whole Wii with a balance board and controller can be had under $100 on craigslist, why make a new product?

1) As I mentioned, some additional input capability would be nice, such as having a mechanism to detect foot twist

2) Using a wii balance board on the PC is still kind of hack-y

Can you exclude the rotation from the control scheme? Also, is board "soft" enough you can use it while sitting?

Yes, rotation can be excluded from the control scheme, and yes it is sensitive enough to be used while sitting down.

MSat is correct on why a new device would be desired, the Wii Board is great for what it is, but it could be better, a lot better. Especially when it comes to developer support on the PC. Hopefully someone with some hardware know how and a VR centric perspective will be able to make and bring something similar to market.

Interestingly enough we are thinking of implementing this control scheme with one of the Primesense family of depth sensors instead of the Wii board.

MSat wrote:
So it seems to use tanks-type controls for turning (on foot forward, one foot back), correct?

I would definitely like to see a PC equivalent of the Wii balance board. I still can't help but to think additional yaw sensors for each foot would be of benefit as it would allow for a wider variety and separation of control "gestures" that could be used.

It looks like tank type controls but it's not. We tried that initially and it was unintuitive, you had to stop and think about how you were going to move, and that was not going to work for us.

I love your ideas on yaw sensors, though it could be a little tricky to use that data as we all have different angle preferences for our feet but I am confident that such information would be useful in some manner. And a PC native piece of hardware, yes it's really needed! There are some patents associated with wii board but I haven't looked them up yet. Hopefully it's not anything that would prevent bringing a real VR solution to market.

Aabel wrote:

It looks like tank type controls but it's not. We tried that initially and it was unintuitive, you had to stop and think about how you were going to move, and that was not going to work for us.

Ok, just admit it - You're using magic!

Aabel wrote: I love your ideas on yaw sensors, though it could be a little tricky to use that data as we all have different angle preferences for our feet but I am confident that such information would be useful in some manner. And a PC native piece of hardware, yes it's really needed! There are some patents associated with wii board but I haven't looked them up yet. Hopefully it's not anything that would prevent bringing a real VR solution to market.

I figured it wouldn't be completely straight-forward when you incorporate yaw sensing (at least displacement) as that would affect the sensed weight at each corner. However, I'm confident it wouldn't be terrible difficult to account for. Another thing though is that you would have the benefit of ensuring that each foot is properly located during use via some sort of bracket around your foot, so such corrections is easier to implement.

This looks excellent. Nice work!

updated first post with a new video

Very cool Aabel, i want to try it now!!

And how about the Sixense Tuscany demo + Wii board?!

We'd do the Sixense Tuscany if the source was available, but it still isn't as far as I know. However looking at the progress that has been made with the Quake3, Doom 3, and Half Life 2 VR make overs perhaps those would make better candidates for compelling demos?

is there a way to strap the pressure sensor to your shoes so you're not tied to one location?

ah yes, here it is http://hacknmod.com/hack/sneakers-wii-b ... nike-wiis/