Great work, brantlew! Very interesting ideas and implementation. I will follow this closely from now on.

Very impressive. Looks like the system is working pretty good.

My group did some of the academic work on redirected walking and I'm really impressed by your set-up: I haven't seen many people actually try this scale before. I think I gather you are just doing a constant rate of turn whilst moving? There is lots of work, outside VR, on how much constant rotation people can detect and I think the research predicted you would need an area about what you are using for this. However its easy to generate degenerate paths where you would walk out of the tracked space if you had constant rate turning, so you will either need a (virtual) safety net ,or you'll end up with very strong steering in certain situations which could cause discomfort/ You can, however, significantly shrink the space if you are more agressive in steering depending on the acceleration of the head. The basic theory about this was pointed out in Sharif Razzaque's PhD thesis at UNC-CH (where redirected walking was introduced), but I haven't seen much development of this, other than to note that you can significantly up the rate of turn in situations where the head is accelerating (to 10s of degrees per second).

If you release this, I can think of half a dozen colleagues of mine who would get one so they could study human locomotion (and play some games no doubt). How accurate do you think you can push the head-tracking?

@profvr

This is exactly the type of research I want to do with the system once Brant lets us play with it. Do you have any other resources about what specific work has already been done in this field?

One thing I am really curious about is whether it is possible to steer the user toward a particular region of the play area, in order to interact with props or particular terrain, like a staircase or doorway. Towards that end, I will need to develop a quiver of techniques for more aggressively steering the user.

@profvr: Thanks. I read all the academic literature I could find on the subject. A lot of it seems to deal with optimized paths in special situations when the virtual environment is known in advance (ie. 90 degree turns, etc). For the general case, it seems that constant turning is the only viable solution.

I knew about the methods for accelerated turning during head motion. When I was working within basketball court size areas I was planning to implement it that way. But my curiosity and the challenge of it led me to aggressively push the scale of the system instead. Once I was able to achieve football size areas, I just decided to keep the algorithm simple with a constant turning rate. Besides - I saw some potential pitfalls to linking turning rates to head rotation. Since the tension of your neck muscles and shoulders serve as a fixed reference point you would be able to detect that the world was rotating if you turned your head from side to side and back again. In the pathological scenario, you could make the world spin around you by shaking your head back and forth.

At the size and parameters I have now, I only encounter a boundary about once every 6 minutes (give or take). My simple solution is just to amplify the turn and aggressively steer the player back into bounds. It's noticeable but infrequent enough that it barely detracts from the overall experience. And currently there are plenty of other technical glitches that are much more annoying and attention grabbing than boundary enforcement.

How much can I improve the performance of the system? I'm not sure. There is still a good bit of work left to do on the sensor fusion and signal processing algorithms. But, since I am currently just trying to support PC games right now, the game interface itself is a big limiting factor. I don't have to work as hard to support the course granularity of the keyboard and mouse. Later however, I would like to expand the project goals to see if I can achieve finer grain control. Sorry if I am being vague. In a few months I will be able to talk in more detail.

Have you thought about finding where Skyrim stores the view angles for the player in memory and directly writing to those locations, or is it the movement that's more of an issue?

One idea I had for hacking analog motion into games designed for keyboards was to strobe the key presses you send into the game at a high frequency, and with a duty cycle that matches the unit vector of the player's motion, then scale it by their speed. So for instance, to achieve a heading of 37 degrees, your duty cycle for the 'Forward' key would be 1, and for the 'Left' key, it would be .75. Then scale that vector by the player's speed.

I never got around to trying it out, and whether or not it would actually work completely depends on how a particular game captures user input.

@Krenzo: It's the motion that's the issue. If I could write directly into a motion vector then that would be cool but I wouldn't even know where to begin to find that. It's not like hacking into the rendering pipeline (ie. TriDef). Those hacks change the viewport but not the character position in the game model.

@FingerFlinger: That's a pretty good idea. I have no idea if it would work or not. It's worth a shot though because it would be pretty easy to implement.

I'd be worried about weird effects when you overrun the input buffer. You could also introduce some nasty lag as the buffer fills up faster then it's consumed by the application. It probably depends on how the game uses the inputs from keyboard, since it may only read them once a frame, in which case, the frequency would have to be much slower then the framerate for it to work. I'd love to see the results of any tests though.

@brantlew: I'm sure lots of people would be interested in the control system. I wouldn't worry too much about the shaking the head and returning to the same direction, in our experience this is not a problem if your control feedback is reasonable; you would anyway only apply the accelerated turning during certain frequencies of head turn where the vestibular and visual cues are not sensitive (roughly equal to "looking around" rather than "shaking the head").

Of course there are lots of other interesting control problems such as which direction to steer them to avoid the boundaries. I am not up to date, but turning to steer them through the center of the tracked region is almost certainly not optimal.

@Android

Yeah, that's pretty much my concern as well. A lower frequency can still be useful for isolating chest and head orientation. It wouldn't be terribly smooth, but you could then walk forward and turn your head without changing the direction of ingame motion.

@FingerFlinger: Well it was an interesting idea but unfortunately it just doesn't seem to work out. I did a quick test with a constant W key and a 50% pulsed D key which should move you at clockwise 27.5 degrees. It works at low frequencies - moving the character in the correct direction, but it appears as a shaky-cam effect. As I increase the frequency I cannot get rid of the vibrational qualities. Instead, the game poller just starts missing pulses and I start losing the angle - but it happens irregularly as the poller and key-pulser move in and out of harmony. So the end effect is that the intended angle is way off, plus there is a vibration in the camera.

It also operates completely different from engine to engine. In SkyRim a frequency of about 12 Hz was the maximum I could push it and maintain the correct angle (with a huge shaky effect). With the Avalanche engine however I could not push it past 4Hz because they employ a small acceleration effect.

So unfortunately it doesn't appear to be a viable option....damn.

Hmm, that's too bad. Do you think it could still be useful for separating head direction from the player's actual direction of motion?

Possibly as a transitional effect but I wouldn't count on it. For example if you were walking forward and made a 90 degree continuous pan with your head, you might be able to hit some of the transitional angles with this pulsing effect. But I wouldn't want to linger too long at one of these angles because I think the vibrational effect would be annoying at best, and nauseating at worst. And there is a very good chance it would just look really bad - even during transition.

thats very cool. i knew where i could use it.
if u turn around 10 times on a soccer field u could walk for a km. the immersion would be very nice. imagine runnig away with top speed from a monster when u ran out of ammunition.... .
it would be cool if there will be some kind of Virtual Reality Theme Parks. with halls full of omnidirectional treadmills where u pay 10 dollars for an hour and big open courts on which u fight against other gamers.

I haven't calculated how far I walk, but I sometimes go for several kilometers on the field. Running would be cool but you would really want some compact, ruggedized hardware like a smart phone instead of a backpack and laptop to do that. Plus you would need to significantly increase the size of the play area to accommodate the increased player speed. But it would be a great way to exercise. It would be ironic if hard-core video gaming could make you physically fit.

it would be so cool if u did a 2 hour workout and game.
i am currently looking for a used treadmill at ebay.
i also will buy an arduino controller. i will try to control the motor by foot contact.
if you raise one foot it would automatically start to move. if u put both down it will stop.
sounds very easy....
for moving left and right i would use the nunchuk controller on a wii gun.
later i could locate the peak of the gun with Leap Motion.

Sounds good. You and zalo can probably share ideas for treadmill control.

The real irony is in 50 years, people will be thinking how mad it was that we had to sit around and twiddle gamepads to play our games, getting fatter and fatter doing it.

Just read through this post & wanted to say that I am blown away by FriiSpace!! Very exciting work. I can't wait to try this one day soon.

Thanks for the encouragement.

I haven't really updated in a while, but I have been hard at work on the project. After QuakeCon, I decided to design the system specifically for the Rift and have spent the last couple of months reworking and optimizing for that platform. Designing for a specific hardware platform has really helped solve some issues I was having with the older design and the performance is getting better and better each month. I get a pre-kickstarter Rift in November and I am anxious do the final integration. But I'm a little nervous because it's either going to be the most nauseating activity you can do with a Rift, or it will be the most immersive experience that you can have outside of a military installation.

Right now I am wrapping up a major round of improvements, so I can send some private demo kits out to a couple of forum members at the end of this month. And then I need to rewrite and polish the prototype software as a professional application. So the goal is still to release the product shortly after the Rift. Maybe in December, but most likely at the start of the next year.

I also want to film another set of in-game videos soon so people can better understand what it's like. I am a huge advocate for natural motion systems now. I feel like they complete and amplify the experience a lot. Notice that the Oculus guys often have to seat or restrain the players because the tendency to start moving is so strong. So that's what I'm trying to offer - the ability to just naturally follow your instincts. But I am having a hard time convincing anyone (even in this VR forum), that it's worth it to go out and try this.

There is definitely a scaling problem with this technology. Once VR is mega-popular, we clearly can't have everybody running around soccer fields; there aren't enough of them. But until that point, this will be the best omni-motion solution available to the average person.

One of the things that I want to experiment with before release is scaled turning to see how much I can comfortably reduce the playing field. It's still going to be a big area. I suspect that a basketball court will be too small, but you might be able to squeeze it into 2 side-by-side basketball courts (or roughly 100' x 100'). That is only partially helpful I realize, but any amount you can reduce the playing field opens up more options for people.

I suspect that you can train the player to accept more aggressive turning as they become used to the setup, also. Perhaps you could globally scale the "turning factor" with the amount of time they've spent playing, up to maximum level, or at least allow user adjustments.

brantlew said:


I know this is a challenge with VR... everything has to be right, or it is useless.

One way to address some weakness in a system is to accept a less complicated set-up. I have always thought that one way to address the issue of trying to allow someone to walk, is to give up walking for the idea of riding a vehicle. An electric mobility scooter can become an ATV you are riding, or even a tank.

I wonder if it would be safer to use a mobility scooter than actually walking. This way you don't fall down if your sense of balance is upset for any reason.

Note that I am suggesting only that you keep this scooter idea in mind as you design your system. This is so that you don't accidentally prevent this sort of system as a fall back because of some sort of technical limitation. For example, if your system depends upon foot steps to allow someone to advance, can you turn that feature off?. It also may benefit those who have disabilities if they can use a scooter.

This next comment is more speculative. A future alternative system may depend upon stationary simulators. Things like the car simulators. It would be good for your system to integrate with those as well. I am not sure what sort of data exchange would be needed.

I guess what I am getting at with the last few comments is that your software would ideally allow for input about the type of system it is being used with. E.g. Free Walking, Walking in place, Moving vehicle, Stationary Vehicle. Are there other categories?

Joe Dunfee

Flying (wingsuit), swimming, crawling (military FPS), running, falling (BASE jump), etc.

@cadcoke5: That's funny. I was just thinking about a stationary mode today after looking closer at the Wizdish. I think my motion software and that device could complement each other well since I handle the separation of torso and head. That's something that I have not seen yet in the Wizdish demos. I haven't thought about vehicles but I don't see any reason it couldn't be done. In fact, the problem is simpler because there would less noise in the signal.

A small update. One of the things that will be great about the Rift is that game developers will combine the analog motion controls of the XBox 360 Controller with the absolute coordinate head tracking of the Rift. Up till now, I had to choose to interface the game with either the mouse/keyboard or the XBox 360 (but never both simultaneously). Mouse emulation offers fairly accurate 1:1 head motion but the WASD keys only allow for 8 discrete directions. This makes motion in the game a bit clunky sometimes and can ruin the illusion of control - especially if your head is just slightly off center as you are walking. It feels like the character is drunk sometimes. The XBox 360 Controller on the other hand has analog direction control, but the head motion is velocity based and so very difficult to synchronize properly with your real head. Here is a video demonstrating the difference between the two interfaces. Here, I kept the head vector fixed to demonstrate directional motion, but the same issues apply if the body vector is fixed and the head is moving (looking around while walking).



But now with the Rift, I can get the best of both worlds. Perfect control of the head along with analog control of the motion vector. This should make Rift-enabled games operate much smoother than unsupported games. Unfortunately for games that do not directly support the Rift, I will probably default to WASD and you will have to deal with a somewhat drunken character.


PS. Interfacing the game with the XBox 360 emulator is not simple. A device driver must be written to accomplish this since Microsoft does not supply an API for it. Currently I am using a third-party prototype driver, but I will be writing my own driver soon and will open-source it with FreePIE to support gamepad emulation.

... as they sit there playing games controlled by their minds alone! lol

@brantlew: Cool.

Check out the ppJoy plugin, it allready emulates gamepads and joysticks edit: a missed the 360 part only read the gamepad part, don think ppJoy does 360 emu

I was fortunate to meet up with Brantlew yesterday and try his Red Rovr system. It was spectacular. The feeling of physically walking around in Skyrim is the most immersive experience I've had so far. We used the ST1080, but can't wait to try it with the Rift. There is a ton of opportunities for systems like Red Rovr that translate the physical movements of the user to actions in the virtual world. I think such systems, coupled with hardware innovations like the Rift and Kinect, have the potential to propel VR into every day mainstream life. The Red Rovr immersion I experienced last night was impressive and trumps any form of 2D or 3D entertainment.

Nice.

Sick! There's a highschool football field 4 blocks from my house; I can't wait to try out his system.

I haven't read the whole thread but do you think this is something that can be used at home? Having a huge area to walk around would be killer but i cant see many practical opportunities to use it. The idea comes to mind to make a high definition version for home use that will track absolute position for 1/1 game position fine movements combined with another tracking method like walking on the spot/ODT to move large distances? That way a small room Rovr and a head tracker can be used to reproduce 10dof tracker?

$10 sounds way too cheap. I'd think you'd need to make more than $10/hr out of all the hardware invested in omnidirectional treadmills, etc.

Back in the last boom of VR, it was $5 for 5 minutes!

I think $10 for 10-15 minutes would be ok however... I mean, people pay $15-$20 for 15 minutes of laserstrike.

@JanVR: Thanks for the positive feedback. Its always fun to meet up and geek out with people that share enthusiasm for VR.

@Okta: My goals were never to make it really big - just to make it as realistic and natural as possible which unfortunately seems to require a minimum amount of space. But yes, I think there are ways to adapt it to home usage as long as you are willing to trade realism for space. I have been exploring stationary usage quite a bit lately, and I'm planning to offer a two-tier system. A smaller version with reduced functionality designed for indoor usage and a full blown version meant for outdoor usage.

Also - one thing that has improved tremendously since my first prototypes is the simplicity of setup. It's bad enough to require players to travel to a large outdoor area, but then to require a complex 30 minute setup and calibration procedure would be too much to ask. The current setup procedure can be accomplished in 5 - 10 minutes and my goal is to make that no more than 5 minutes.

brantlew was nice enough to lend me his RedRoVR system about a month ago, and I am just now getting around to doing a short write-up. (I'm very lazy...) I was only able to do 2 long sessions because of time and weather, but I think that I got a decent feel for it. The first few minutes in the system take a little getting used to. Forward and backward walking both work very well, but side-stepping/strafing is a bit funky currently. One pretty neat trick is that I could walk forward and look around with my head, as you would walking down a sidewalk, looking at storefronts and whatnot. The user must take an authoritative first step for the software to correctly recognize his/her intent. This is easy to get used to, and after the first step, walking feels completely natural, with a few caveats:

1. Sharp/violent turns can confuse the accelerometer.
2. The user must lead with their hips, which is mostly how one naturally moves anyway, but there is some nuance.

The only time I really noticed these issues was during “moments of immersion”. For instance, a wolf would attack me in Oblivion, and I would frantically try to get away, only to end up confusing the accelerometer with violent, bouncy motion. Because of this, RedRoVR is best with exploration games for right now. Although I know that brantlew is continuing work on his sensor fusion algorithms.

The quality of the redirection is very dependent on the size of the play-boundaries. I used an American football field and a square-shaped field about 60 meters on a side. Both of these worked well, and I think that the redirection would be too aggressive in a smaller area. At the same time, a much larger area would be even better. As it was, the redirection was still slightly noticeable, and I approached the boundaries maybe once every 10-12 minutes. I think that much of this can be remedied with levels designed specifically around this technology, and this is something I want to experiment with once I have a Rift.

I was somewhat limited by my setup. I used my netbook as a backtop and I was also borrowing brantlew's Vuzix Wrap 920. Despite framerate issues and a low FOV, I had a good experience; the potential is certainly there.

I saw just an early prototype, so obviously it was not in a state ready for Joe-gamer. Despite that, I think that the RedRoVR portion of the setup was fairly painless; it was mostly the general hassle of a backtop that I had an issue with. And using an HMD with an integrated headtracker should simplify that further.

A big limitation is software that is not designed with considerations for VR. One of the big issues is dealing with WASD controls. brantlew's system can detect motion over 360 degrees on the ground, but is artificially limited to only 4 directions. Similarly, games typically only have two movement speeds, walk and run. Being able to use true motion vectors will help quite a bit, I think. Something else that never occurred to me before actually experiencing it is the importance of sound effects. RedRoVR can detect the user's footfalls, and could communicate that information to the game engine, thus synching the sound effect with the user's action. Obviously, this isn't a feature that any games support yet, but it is important.

One last curiosity that I'll mention is that, once I got settled and was freely walking around, I had the sensation of not physically moving anywhere, as if I was walking on a treadmill. I suspect that this is due to the HMD's low FOV. My biggest frame of reference was the screen hanging in front of my face.

Thanks Finger. I still feel bad that you were saddled with the Vuzix for this demo. It really dampens the effect.

I've had a dozen friends and forum members try out the system so far and all the impressions have been very informative and valuable to me. Generally the response has been positive. Even guys that have used professional VR systems have commented that the freedom of unconstrained movement feels like something "new" and adds an additional layer of immersion that they had not experienced before. So it works great as a VR demonstration. But I am also acutely aware that the system is not ready for active gameplay yet. The motion detection is simply not robust and accurate enough. Consequently I think I need to delay/suspend any plans for release until I can work through another round or two of basic R&D.

I think it's definitely worth having as mature a product as possible before releasing it. And a purpose-built demo would push it to the next level.