Using feet for movement in a VR world

[WhoIsRich]

So while I wait impatiently for my rift to arrive, I have been thinking about movement and immersion.

Currently the rift demo's have been with an Xbox controller, there are cool posts here showing off omnidirectional walking platforms, shuffling dishes, but I live in a flat where space is limited.

So I was wondering if any of the DIY hackers here had tried any direct foot controls. I was looking at flight simulator rudder pedals, where the sliding could be used for turning, and the tilt used for forwards and backwards.

Anyone tried this or thoughts of other USB devices for tracking feet movement while sitting?

[Direlight]

Glovepie + wii balance board. It works, but haven't made any scripts yet. Script I downloaded was weird.

It's wireless too, I have this one.

http://www.amazon.com/gp/product/B004DP ... UTF8&psc=1



The scale gauge seems to not work correctly.

[STRZ]

I had some thoughts regarding feet control http://www.mtbs3d.com/phpBB/viewtopic.p ... =30#p99406

I think the idea with a jogwheel/rotating disc for turning around for one foot, and for the other foot to emulate the WASD movements is the best combination considering that you'd have free and independent aiming/handcontrols and the headtracking of the Rift as well. The idea of the jogwheel is basically putting the precision you have when turning your character with your mouse movement into foot controls, by having a rotating disc you're also able to do more than you could with your mouse, with a little push into a direction and letting the disc spin, you could turn very quick into a certain position or let the character turn on his axis only with two small inputs (pushing and stopping the disc).

The idea basically evolved out of foot mouse concepts like this one, only that it isn't based on optical tracking and doesn't need aiming, or up and down movement of the cursor, the left right movement is enough for turning your character, so that a turning disc is better suited than a pedal with a optical tracker on a ball bearing:

[youtube]http://www.youtube.com/watch?v=If-jkWIa1mg[/youtube]

http://www.dossierd.nl/ap_voetmuis_detail_eng.html

[MSat]

The following thread is a discussion on the topic:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15730

[crespo80]

I have this idea of a foot controller but I realized I don't have the time nor the skills to design and build a prototype.
You place both your feet on this controller and simply:
pitch forward/backward to go forward/backward
roll left/right to strafe left/right
yaw clockwise or counterclockwise to rotate your avatar's body like on a gamepad analog stick

[youtube-hd]http://www.youtube.com/watch?v=Wz_Ue1amC1Q[/youtube-hd]

[nanicoar]

crespo80, that looks like a really cool idea. - I mean, like it would actually be practical and fun. It's certainly much better than my idea of a huge squirrel wheel with breaks and motors (for realistic slopes).

Some ideas;

• Put your smartphone between your feet, since its gyro should be up for this task. We really need a plug-and-play sensor fusion library for smartphones, so someone is likely to write the code.
• Attach a pair of those plastic, shower stall sandals to the balance board so you can hook your feet into it, for use with a spinny chair. This way you can push off to spin and lift your feet with the board attached, which makes for a great deal of physical engagement while remaining safely seated.
• Snowboard, skiing, or surfing game for demo. This should separate the idea from all those shooter games, making it stand out more. These games might really be easiest to implement in a proprietary game engine like Unity. In a skiing game you could actually do tricks, if your sensors are up to it. - Just close your eyes and let your imagination be your VR!

[PatimPatam]

I really like this idea crespo, something similar to what you show in the video could work very well, and free your hands to do more interesting stuff than simply moving your avatar around.

Personally for my daily VR use i think i would prefer something like this over something like the Virtuix Omni, but i believe both type of controls can perfectly coexist if VR takes off as i think it will.

Now we only need to find someone with the time and the skills to design and build a prototype

[cybereality]

@crespo80: Looking good. I think something like that could work.

[crespo80]

The original idea was a foot controller for standing on your feet and give 1:1 rotational mapping.
Basically a really cheap and simple alternative to ODTs or sliding devices.

[youtube-hd]http://www.youtube.com/watch?v=RXl7BwhIHeg[/youtube-hd]

[Flassan]

I like it @crespo80

Versatile and might even benefit some handicapped players as well. Keep working on it.

[crespo80]

as I said, it's just an idea I had, I'm not working on this, the best I could do with my skills is that video on youtube

[WhoIsRich]

@Direlight
Wii board, excellent suggestion, being flat would make it easy to combine with something else for rotation.

@STRZ
Thanks for the thread, will have a good read.

@crespo80
That is how I also imaged the controls in my head, being able to tilt and rotate with both feet,
and was trying to find a simple way to implement it using off the shelf hardware.


In my latest search, I found there are joysticks that have twist action: ( see 30 secs in )
[youtube]http://www.youtube.com/watch?v=n3PUS8IvyAw[/youtube]

I am thinking if the stick part could be replaced, or even pedals fixed to
either side of the stick, you could re-create the movement you posted.

Looking at my current leg postion, I would need a higher chair or to mount the joystick
on somesort of 45 degree block for it to be comfortable.

[Benjimoron]

This looks cool. I wonder if it would be more immersive if your character in the game was also on something similar, kinda like a hoverboard? It could probably be added to some games as mods. Just change the way the character moves a little so you'll feel like you're actually on a hoverboard. So it won't feel weird that you're using one to play the game.

[sunvsmoon]

The original idea was a foot controller for standing on your feet and give 1:1 rotational mapping.
Basically a really cheap and simple alternative to ODTs or sliding devices.

Hi,

Someone posted a similar idea here, combining a balance board with a U-HID-G three-axis accelerometer interface board , it looks really cool.

http://www.youtube.com/watch?v=IuBHwmEuxl8

[crespo80]

and the very very original idea was a standardized VR input device the consumer Rift could have shipped with.
I mean one single device that includes:
- positional HMD tracking
- full body tracking
- foot controller
- compactness
- no need to find a place for the cameras

But maybe this would be too bulky (and it remembers me of the infamous SEGA Activator 2

[geekmaster]

In my latest search, I found there are joysticks that have twist action: ( see 30 secs in )
...

Sadly, most of my 3D joysticks only work on Windows 98 or earlier (not USB). I only have a few 3D joysticks with USB plugs on them. They were great for playing Descent, until I switched to a SpaceOrb 360 6DoF controller (which used a serial port, like most mice of that era):



3D joysticks dropped off of store shelves when Windows XP did not support them, and the newer USB replacements did not really catch on after so many people got stuck with unsupported joysticks.

Unlike all my non-USB joysticks, my SpaceOrb 360 may still work if I connect it to a USB serial adapter. Now I need to dig it out and try it... Or I could gut it and hack it into a 3D foot controller device.

[cybereality]

Oh, I remember that thing. Always wanted one but never got to try it.

[WiredEarp]

@ Crespo: the idea for the 2 footed controller is a very good one - while its not a walking replacement like we will eventually have, it looks like it could be a great walking interface for NOW, and probably the next few years. I think its quite possible people would get used to using it very quickly.

IMHO, you should work on that idea, get a prototype together, and if you find it feasible, I think it could be a great kickstarter idea. The big advantage is using both feet to control it, I think this will be easier and simpler than learning to do one tihing with one foot and a different thing with the other...

I've got a slightly similar but way more complex active system concept, but this seems the most practical consumer priced solution to date I've seen. The fact it is small and portable and requires no building/setup, are all points that make it more attractive than some of the ODT solutions I've seen.

[langmyersknow]

My question is this:

What if you just want to stop , and not move.

The video version of you only having one foot on it shows that one part of the dish will be hitting the floor, because I don't think anyone would be able to balance perfectly level to stop moving....

[crespo80]

The dish I was using in the video is just a "fitness balance board" that is normally used for proprioceptive balance exercises.
A proper foot controller would be less high, with less range of motion, would not touch the ground, would have some return spring to autocenter;
and optimally it would pitch only forward/backward, allowing to strafe by a center-to-side sliding movement, thus allowing better combined forward/strafe movements, or maybe even better replace the whole tilting concept with a sliding concept, so slide forward/backward or left/right to move accordingly, no need to balance, less high from the ground, more intuitive

[WiredEarp]

Your ideas are excellent there crespo80 (sliding etc). Since the turning with your device would be similar to how you turn on a console using the stick controller (analogue input controls turning speed), it would probably be quite intuitive to many people.

[nanicoar]

I'm working on my derivative of crespo's idea. I'll attempt to use node.js, socket.io, PhoneGap and three.js to make a demo. I have a Nokia phone which uses something much like PhoneGap, but I might be able to make the demo completely cross-platform.

Being a newb to these technologies, I'd love any help, support, or kind words. =)

ED:
Hit a snag. The rotation sensor reports in 15 degree increments. - However it does this really fast, so I can see the numbers jitter, which means I might be able to write a smoothing algorithm.
Before doing that I'll see if I can get the raw data to a node.js server. If that works then the other sensors should be usable whether or not I can write a sensible algorithm.

[WhoIsRich]

Update:

I have been looking into how to use these devices to control movement in a game.
My knowledge is mostly based around C# and general scripting.

A lot of examples I found were based around sending key presses,
but I want to send analog movement which means joystick data.

Had a look at 'GlovePie' but joystick emulation seems to require 'PPJoy' which seems
to have since been abandoned by the developer.

Then I found this great joystick emulator called VJoy
Not only was it simple to install, the SDK comes with several language examples,
including C# which I was able to have a play around with and verify it works.

Then I found for games that have limited controller support, there is a program
called 'x360ce' to emulate an Xbox 360 controller from a generic joystick.

Hardware:

I was looking at motion sensors like the vid with the guy controlling tribes,
then I realised, I already have one in the form of my Android smart phone.
As a bonus it also a compass which should be able to provide rotation.

Found an Android app called 'Sensor Ex' which has a DotNet support.
So plan to use this to wirelessly link the phone sensors to VJoy.
The latency will probably be terrible, but it will be good as a proof of concept.

I also order the 3D joystick and Wii board to see how mechanical and pressure
based controls will compare to the tilt, so fun and headaches ahead once I figure
out more of the programming side of things.

[nanicoar]

Of course! The magnetometer should do a fine job.

[crespo80]

You can use some generic joystick USB interface that works without drivers on any OS (Windows,macOS,Linux) and can control multiple analog inputs (for potentiometers or hall effect sensors) and multiple buttons

http://www.leobodnar.com/shop/index.php ... cts_id=203

http://www.derekspearedesigns.com/circuits

[geekmaster]

Of course! The magnetometer should do a fine job.

I just played with the magetometer compass app in my Nexus 7, and it seems to completely ignore a strong rare-earth magnet (from a hard drive servo actuator) regardless of where I hold it, or even when touching my Nexus 7. The magnet is strong enough that I can pick up my Nexus 7 with it, and yet the compass app still points north as I rotate it. Accurately sensing the weak magnetic north while within a strong contrary magnetic field seems like magic. How does it do that?

Oh, wait, I tried turning it off and on, and the compass app starts out pointing the wrong direction until I move it a little, but it corrects rapidly even with the magnet nearby. It must be using accelerometer fusion in the compass app, to ignore temporary magnetic interference (or something like that). The rapid correction still seems magical though...

[WhoIsRich]

@nanicoar

I had not heard of PhoneGap before, looks like a great way to have compatible with a variety of phones.
Let us know how you get on with averaging the raw sensor data and getting it quickly to the PC.

My javascript knowledge is limited, so when it comes to a constant stream of data, I would have no
idea if XmlHttpRequest, WebSockets, or some other means is the best for transmission.

[VRus]

cudos to Crespo for this dead-simple minimalist solution..

One reason i'm very glad there's so much fruitful brainstorming going on around the future VR peripherals
among us enthusiasts, is that these ideas and cool small demos
can be used as prior art against those mega-companies which later in the game will come trying
to monopolize/distort VR market by pouring money into patenting anything and everything in sight...

One such example is Jeff Han' little multi-touch tech-demo that was used in court
to show prior-art in Apple vs Samsung patent debacle:
http://www.youtube.com/watch?v=QKh1Rv0PlOQ

*hint hint* to Cyber and Nail - guys, please, keep this great stuff well indexed and archived on every
search engine that exists, this may come in handy later, to fight a good fight when VR revolution goes into full swing...

[bobv5]

Of course! The magnetometer should do a fine job.

I just played with the magetometer compass app in my Nexus 7, and it seems to completely ignore a strong rare-earth magnet (from a hard drive servo actuator) regardless of where I hold it, or even when touching my Nexus 7. The magnet is strong enough that I can pick up my Nexus 7 with it, and yet the compass app still points north as I rotate it. Accurately sensing the weak magnetic north while within a strong contrary magnetic field seems like magic. How does it do that?

Oh, wait, I tried turning it off and on, and the compass app starts out pointing the wrong direction until I move it a little, but it corrects rapidly even with the magnet nearby. It must be using accelerometer fusion in the compass app, to ignore temporary magnetic interference (or something like that). The rapid correction still seems magical though...

GPS?

[geekmaster]

I just played with the magetometer compass app in my Nexus 7, and it seems to completely ignore a strong rare-earth magnet (from a hard drive servo actuator) regardless of where I hold it, or even when touching my Nexus 7...

GPS?

No, I only rotated my device. I did not move it. Due to inaccuracies in GPS coordinates, you have to move at least a few meters to calculate a motion vector, but that still does not tell you compass directions relative to the device unless you are facing in your direction of motion.

It cannot be GPS, so it must be magic!

Actually, I already figured out that it must be using the accelerometers to cancel out stray temporary magnetic disturbances (even strong ones like I tested). Starting it up near a magnet does cause a "great disturbance in the force".

Luke: "Are you all right? What's wrong?" Obi-Wan: "I felt a great disturbance in the Force... as if millions of voices suddenly cried out in terror and were suddenly silenced. I fear something terrible has happened. You'd better get on with your exercises."

[Hooves]

What about a Dance Dance revolution gamepad rewired to be slightly larger to accomadate the size of a human "step". The only problem remaining is orientation so stepping forward is always forward no matter what direction you are facing.

[Mystify]

What about a Dance Dance revolution gamepad rewired to be slightly larger to accomadate the size of a human "step". The only problem remaining is orientation so stepping forward is always forward no matter what direction you are facing.

Plus you have a huge delay between "I want to move forward" and "I have taken a step and the button has registered it".

[geekmaster]

What about a Dance Dance revolution gamepad rewired to be slightly larger to accomadate the size of a human "step". The only problem remaining is orientation so stepping forward is always forward no matter what direction you are facing.

It has been discussed before, and there were problems with trying to use if for VR:
http://www.mtbs3d.com/phpBB/viewtopic.p ... 479#p89479
http://www.mtbs3d.com/phpBB/viewtopic.p ... 134#p49321

Hmm... I see that YOU created this other thread where this was discussed before:
http://www.mtbs3d.com/phpBB/viewtopic.p ... 639#p71448

The same problems still apply, just the same as the last time you participated in a "DDR" discussion. A larger gamepad will have the same problems as before...

[nanicoar]

There may have been an error in the API documentation published by Nokia which means all my progress yesterday didn't happen. I'm waiting for their response on that.

Until then I will probably have to implement the client in platform-specific C++, just so that I can start development on the server. I will still write the server on node.js so that it is as cross-platfom as possible.

[WhoIsRich]

I was able to mashup Sensor Ex and VJoy, so uploaded a vid for you guys

[youtube-hd]https://www.youtube.com/watch?v=Nzl93ncBONU[/youtube-hd]

It's by no means perfect, Sensor Ex is a CPU hog, both on the phone and PC,
and even then every so often the data flow stutters. But I have been unable to
find anything similar that has a ready to go API, so that only leaves custom coding
something to interact with .NET which would take a chunk of time and effort.

I am thinking that it would be more practical to get a Playstation Move controller which
apparently has the same sensors, and could be accessed directly by blueooth.

There are other interesting issues such as with a magnetometer, for accurate
rotation data, you also need compensate for any tilt going on at the same time,
which involves some complicated 3D math ( available on some Model Aircraft sites )

[nanicoar]

But I have been unable to
find anything similar that has a ready to go API, so that only leaves custom coding
something to interact with .NET which would take a chunk of time and effort.

Is that TCP/IP traffic it sends or is it bluetooth? Could you capture some traffic between the phone and PC with for example wireshark? It's unlikely, but if it uses SSL you can use Fiddler to read it. I'm about ready to start developing the server, and your work is relevant to my interests. =3

[nateight]

I've been lurking for months, so I'm thrilled to finally have something relevant to add to a current discussion. I've long wondered how well a Balance Board would handle this task, but never felt optimistic enough about it to buy one (upload some demo videos, please!), and while the Crespodisc™, that “foot mouse”, and Rich's accelerometer concept have some promise, I'm reminded of a (warning: typically vulgar) Penny-Arcade comic. I'm confident VR is going to change a great many things, but mankind's predisposition toward sitting on a comfy chair while gaming is not one of them. As much as I'd like to avoid becoming a pariah on my first official day here, Project Holodeck, for all its ambition, is unlikely to achieve commercial success on account of being too much too soon, and gamers across the globe, blinded by a VR rig or not, don't particularly want to flail around knocking every lamp off of every end-table. All the omnidirectional treadmills and force-feedback haptics we can imagine won't get the VR revolution off to its proper start if a robust input scheme isn't available while sitting down. After all, “From the mightiest Pharaoh to the lowliest peasant, who doesn't enjoy a good sit?”

Even without first-hand experience with a serious VR rig, I strongly suspect tying character rotation to head tracking is the wrong approach here. If, within virtual space, we literally cannot move in one direction while looking in another, how does the experience meaningfully differentiate itself from playing in front of a 3D monitor? Descent-style 6DOF games will have their unique challenges and rewards and are likely to take their control schemes from the colorful world of flight simulation hardware, but for all those great many more game with a humanoid avatar, “pitch” and “roll” cease to have meaning, and “yaw” is just turning your head (someone please prove me wrong with a long-overdue The Specialists remake – I'll say “yaw” as much as you want if I get to do side-somersaults and shoot people while ass over teakettle). Luckily, in any virtual space in which Cartesian coordinates are more appropriate than flight dynamics, Z axis motion will be limited to crouching and jumping (two buttons mapped, no Kinect required), and head tracking does fulfill the role of mouselooking up and down, so the problem becomes mercifully two-dimensional. I'm all for a foot control scheme that a seated player can use, absent any other inputs in his hands, to describe 100% of character motion, and all while being intuitive, precise, comfortable to operate for extended periods and cheap to produce, but I'm skeptical that such an engineering challenge will ever be met by us mortals, and we need something to fill this very large void in less than a month. It's possible I've been blinded by fanboyism for the VR pioneers at Virtual World, but I think the ultimate winner here is going to be as simple for us as it was for them: analog pedals.

Transcription machine pedals have been mentioned, as have rudder controls, but the former are unsuitable for being strictly on-off, the latter for being an elegant solution to a subtly different problem. It so happens that my desire to replicate (albeit poorly) the Battletech: Firestorm experience led me to build something that I think we're all going to want under our computer desks, and it has the advantages of being so simple as to not warrant an Instructable in addition to -philicity-'s and so cheap I barely spent $10 on the non-Arduino parts. Here's the complete process: Take one of these, gut the electronics (preferably while scoffing at the entire premise of Guitar Hero), replace the weird potentiometers inside with cheap linear 10ks, stuff in some washers if the pots aren't snug and trim out whatever plastic necessary to secure an Arduino inside and access its USB port. If you're smart and have done this with an Uno or later and also happen to either have an AVR in-system programmer laying around or have the patience and nerves necessary to do a lot of firmware flashing, you can use Dimitri Diakopoulos' HIDUINO firmware to turn this into an HID device that most Windows PCs will recognize as a joystick without even installing a driver. Test for the endpoint values of the pots in their current orientation, upload a sketch so simple even a novice programmer seasoned by mere weeks of Arduino tinkering could write it, and you have a fully functional and satisfyingly analog single-axis joystick. I'm going to wait until my Rift arrives to decide whether it feels better to assign this joystick to character rotation or strafing, but I suspect using pedals to turn yourself in place will feel more intuitive. Of course, this leaves strafing and forward and backward motion to deal with, but in my view that's something suitably solved by a single thumbstick, be it on a 360 controller, Wiimote nunchuck, or (ideally) Hydra wand. At this point you should be looking around with your head, turning with your feet, and moving through virtual space with your thumb, each entirely independent of the other, and still have nine fingers free – and back in meatspace you're seated comfortably and not even accidentally strangling yourself with a video cable. I'd love to be proven wrong by a better control scheme, but I'm not confident any such thing can even be conceived of until the world is blissfully awash in production model Rifts, if ever; pedals may not be sexy, but I believe they'll get the job done.

Like many of us here, I've been fantasizing about VR for most of my life, so I don't think my conclusions are faulty due to a lack of scrutinizing the problem, but at this point I would prefer a spirited debate about why I'm wrong over agreement or deafening silence. Am I even correct in assuming that I've explained this adequately, or would you guys want to see some videos demonstrating how smooth and responsive these pedals feel? And regarding Kickstarter, I have been mulling over seeing how this prototype would do, but it feels entirely too simple and obvious, and the only reason I can fathom for a lack of some similar commercial product is a lapse in my Google-fu. If I ever manage to get my Power Glove 2.0 prototype together, though, the pedals will make a great companion reward – stay tuned.

TLDR: GUYS LET'S USE PEDALS FOR TURNING

[TheHolyChicken]

Am I even correct in assuming that I've explained this adequately, or would you guys want to see some videos demonstrating how smooth and responsive these pedals feel?

From what I understand, you are suggesting that you have a left pedal to turn left, a right pedal to turn right, and the player would simply use a standard gaming analogue stick (as in a gamepad) for movement. That actually seems to me like the most attractive control setup I've seen suggested so far for seated playing.

[nateight]

you are suggesting that you have a left pedal to turn left, a right pedal to turn right, and the player would simply use a standard gaming analogue stick (as in a gamepad) for movement . That actually seems to me like the most attractive control setup I've seen suggested so far for seated playing.

Thanks for the feedback. I'm hardly the first to suggest such a thing, but because there doesn't seem to be an existing product ideal for this I thought my pedal rig might be of interest. As much as I want to encourage exploration and experimentation with innovative new control schemes, I'm worried the collective VR movement might get lost in the weeds in the process. Having a simple and largely proven setup as a anchor seems advisable - I'm not certain this is it, but I'm hoping we can determine that in time to establish something as the preferred default setup devs will be building upon when their Rifts arrive.

[TheHolyChicken]

A video would still be good, nateight, to let people see such a device in action.

I think what will be extremely important for any device controlling movement is accurate, reliable precision. Especially given that there is risk of simulator sickness, you really REALLY need users to have superb control over their movements at all times. With other proposed solutions I would be worried about unintentional or inaccurate movements, which could easily lead to vomit!