First Person Shooter Simulator Build Log

[Synexious]

I have identified a system of components which, combined together, should enable an affordable, compact, more or less plug and play body motion simulator which creates a compelling feeling of actually inhabiting three-dimensional virtual space, for use in any nonvehicular simulation - first person shooters, roleplaying and action games, and virtual worlds:

Head-mounted Display
The HMD is the central component of the system. I want to be able to move my head freely without worrying about my eyes drifting offscreen. While multiple projectors would allow a wider FOV, they are static, so looking up, down, or behind is not possible. Projectors are also too large for me to fit in my dorm, and would be more expensive. Since seeing an Icuiti (now Vuzix) 920 for $600USD in SkyMall magazine on a plane as a kid, I've always wanted an HMD, but have been put off by the abysmal resolutions. Greater than 1080p resolution monitors are cheap, but until recently, the cheapest HD HMD cost ~$12,000. However, two consumer HD HMDs have been shown recently - the Sony "Headman" and the Carl Zeiss Cinemizer OLED. The Headman is confirmed to be 720p, but it has no release date or price, and may never be released. Cyberreality heard a rumor from someone in Sony that there is corporate interest in bringing this to market. Conversely, the Cinemizer is not confirmed to be HD, though some sources are reporting it as such, but it does have a release date and price - $515 summer 2011.

Surround Sound Headphones
A true surround sound system is of course superior to simulated surround sound, but would not be compact or as affordable. High quality surround sound headphones are available for $200 or less. Surround sound headphones combined with an HMD provide ultracompact - even portable - total audiovisual immersion. Though I have wireless Turtle Beach X41s, I will probably buy wired PC headphones to eliminate the hiss.

Manual Treadmill
This is the most active part of the simulator. Instead of sitting in a chair or on a couch, I will move my body through virtual space. Driving and flight simulation is relatively easy because the user is supposed to be stationary. Walking complicates the situation. This can be solved with professional applications with devices such as the VirtuSphere, CyberCarpet, and CyberWalk, but none of these are anywhere near affordable. A bidirectional manual treadmill is the best - and only - available compromise, allowing the user to walk or run forward and backward, with turning handled either by head movement or controller. When the act of walking is synchronized with movement in virtual space, the user feels a surprising haptic connection with the avatar and gameworld, further increasing immersion. The ability to physically turn is of course desirable, but is not as important as simply moving forward/backward. Manual treadmills allow the user to stop at any time (this is not intended as an exercise device, so high speeds are unlikely, though even slow walking can improve health and focus), and are extremely inexpensive - ~$140.

Motion Tracking Gun Controller
A rifle style controller, possibly with the option to switch to a pistol alternate for smaller weapons. Something like Jim Blackhall's PC Game Gun, which is a driverless solution, meaning it is 100% compatible with all games. I'll also probably use a PlayStation Move rifle. The Novint Falcon is stationary, but seems to provide superior force feedback, so that is another option. The pistol grip can be removed to simulate picking up objects and touching them. At $280, it's expensive, so I'll only add this if I'm sure I'd use it frequently.

Body-worn Haptic Feedback Devices
TN Games' 3rd Space Gaming Vest ($140) allows the user to feel bullet, grenade, and knife impacts, up to 5psi. Not all recent shooters work with the vest, and the Trigger box for console support seems to be vaporware, as does the HXT helmet and the rest of the system, but the vest does work with all the Call of Dutys save the first one, S.T.A.L.K.E.R.: Call of Pripyat and Clear Sky, F.E.A.R. 1 and 2, and several others I'm interested in, like Quake 4, Halo 2, and UT3. It also supports three RPGs - Fable, Fallout 3, and Mass Effect.

ForceTek's XIO gaming exoskeleton is very interesting. Currently only the arms are available, but they plan to have a full exoskeleton available. I contacted the company through their site and received a reply within a day - they're planning on an early 2012 release. I plan to build my simulator as soon as the first HD HMD is made available, which I'm hoping is the Cinemizer OLED this summer, but I could add the XIO later.

Ambient Feedback Devices
This is the most supplementary category. Someone mentioned here on MTBS that peripheral vision could be faked in an HMD by including ambient lighting, via LEDs I'd guess. Perhaps an HMD could be modded to include that, but I probably wouldn't want to break into my HMD unless I was sure it would greatly increase immersion. One more tenuous idea is using fans, as is done in a racing simulator. Perhaps they could be triggered during explosions and while sprinting...

This potential simulator is highly immersive while still being affordable and compact enough to fit in a dorm with a roommate. I just came to a rough estimate of $1100 in my head. That leaves me a few thousand for a new computer. I might just upgrade my graphics card if my current motherboard can support a high level card. I want to max out games like World at War, Black Ops, enhanced Morrowind and Oblivion, Skyrim, GTA, The Witcher, Fallout, and Mass Effect, and maybe even Crysis, but the low resolution of 1280x720x2 means I'll need much less processing power than with 3DVS. I'll also use it with console games ranging from Reach to Heavy Rain.

[Synexious]

(Reserved for future build developments).

[cybereality]

You've got some good ideas there. I too want to build the ultimate simulator, but it seems very hard to find the right equipment (for affordable prices).

In terms of HMDs, most of the stuff out there (like VR920) is probably not immersive enough to really justify a setup like this. I am looking into creating a DIY HMD based on pico-projectors, but the rig might be too bulky for free-movement. I don't think resolution is that important, FOV is way more essential. Even at 720P, I think some of the headsets will still be lacking due to low FOV. I think you need to get into the 90 degree range before you can call it immersive. Still, we have to work with whats available and the upcoming Vuzix 1200VR sounds like the best option at the moment (that doesn't cost an arm and a leg).

Sound is pretty much owned at this point. There are a variety of headsets that will do fine for this purpose.

The manual treadmill sounds promising, and could add a lot to immersion. However then you lose the ability to turn. Personally I think it would be better to just walk in place (for example with special shoes or a mat) but have the ability to turn 360 degrees. Might be hard to walk backwards (or sideways) but if you restricted it to walking forward it could work. But I guess ultimately you would want both things, but you can only get that with those more expensive setups. The manual treadmill does seem really cool since they are actually affordable. But that does take up a lot of space, not sure I want one of those in my room. Another idea is to wear rollerskates and a harness that keeps you stationary. That way you feel like you are walking but you stay in one place. That might also let you turn around too.

I am still looking for the perfect VR controller. So far the Sixense controllers look the most promising ( http://www.razerzone.com/motionsensing/" onclick="window.open(this.href);return false; ) since they offer extremely accurate absolute positioning. Most other motion-controllers use relative positioning and this can drift pretty quickly. Using a PS Move or Kinect may be another interesting option. I guess a Wiimote could also be used, but I question the accuracy. I've got the Novint Falcon, and the force-feedback is awesome, but it is not practical for a mobile setup. However it is pretty cool, and haptics do add a lot to the experience. The XIO thing looks intriguing, but I get the feeling its never really going to come out. Someone really needs to come out with a good VR controller.

Haptics suits should be pretty fun, never got to try anything like that myself.

I have some plans to build a wearable computer, and I want to use the 1200VR as the headset. I'm looking at getting this Asus 3D laptop ( http://www.amazon.com/G73SW-3DE-Republi ... 230&sr=8-4" onclick="window.open(this.href);return false; ). Still not sure what I am going to use as the controller, or if I need to build something custom. Should be really fun if I ever actually do this.

[Okta]

If you really want to go ghetto maybe mounting up a cheap 3d monitor with a fresnel lens at head height right in your face on the treadmill would work very well. You could even build a shroud over it.

I cant help but think the linear movement on the treadmill will kill immersion though, and using a motion pointer type gun just wont work as it will kill immersion again not being able to turn with it. It will be like using head tracking on a fixed monitor

[cybereality]

Or you could build some special shoes like this guy:

[youtube]http://www.youtube.com/watch?v=UYE6dqaereU[/youtube]

[Synexious]

I think simulating walking, even if only linearly, might be better than being able to turn fully. Using PlayStation Move, full turning isn't possible, but the ability to aim a real gun still increases immersion. The GameRunner seemed to work well:

http://gamerunner.us/index.php?media" onclick="window.open(this.href);return false;

Those VR shoes are an amazing concept, Cyberreality! I've never seen those before. They'd be a cheap alternative to ODTs and spheres, but I couldn't find anything at all about them beyond that video and a few blog mentions of it. Do you think it would be possible for us to build our own VR shoes? I can't code, so it would have to be plug and play once built, or else we'd have to a more skilled person/people to help. How would we make the wheels reverse? Also, once full movement is in play, the system must be completely wireless, like in the VirtuSphere, because wires would tangle quickly. What HMDs are wireless, or can be made wireless? There are ways to transmit AV signals wirelessly, but from what I've read, they're unreliable.

[cybereality]

I watched some of those GameRunner videos, and its a cool concept but I don't think that's immersive. It adds realism, but I am not sure about immersion. Immersion is more about being engulfed in a experience, which usually translates into things like high-FOV HMDs or surround sound audio systems. The key to immersion is the peripheral, seeing an enemy at the corner of our eye, or hearing a sound behind you. That is the exact opposite of being forced to look and move only forward. So the physical aspect of walking on a treadmill should add realism, yes, but I think it won't add much for immersion. Let me rephrase that: in very limited cases, the moving forward idea would work (lets say in a track & field game where you could only move in a straight line anyway). But in more complex situations (say a first person shooter) I am not sure it would work good at all. It just seems really restricting. I think I would much rather simply run in place, but have the ability to turn around freely. That at least feels like you would be surrounded by a world, not merely looking at a TV screen.

The VR shoes thing seems promising, but would require a really complex setup. I mean, the video shows that the concept at least works. But to make it usable in an action-orientated situation (aka first person shooter games) would be another story. But its certainly worth exploring at some point. I can program, but I'm just a software guy. I don't know much about building hardware or any of that stuff.

In terms of wireless solutions, there are a few options. First, you can simply have a gaming laptop in a backpack. HMDs like the Vuzix 1200VR use USB power, so the only wires needed would go from the headset to the backpack. Most controllers (mice, wiimotes, etc.) seem to be wireless these days, so that shouldn't pose a problem. There is also a wireless video card available, see this thread: http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=12477" onclick="window.open(this.href);return false; . However you would probably still need to power the HMD somehow, so you need a backpack in any case.

[Synexious]

That wireless card isn't upgradeable, though, and a 3D laptop is much more expensive than a custom desktop, and still can't max out recent games. If I was to develop a wireless system, I'd try to find a method of transmitting video from a desktop in high quality, without being stuck with a special card. That may not be possible currently, though. The treadmill concept is compelling to me because most of the time I'm moving forward, and turning is just a momentary adjustment. Also, as I said, with Move, Wii, and standard PC motion controllers, the player can move the gun to the sides to turn, not actually turning all the way around. A manual treadmill is only $140, though, so I could easily use both. In fact, my parents have a powered treadmill they never use, so I could test both options (360 degree turning, no motion/forward-oriented aiming/motion) without spending any extra money.

I've always hated math, but my software engineer father's always told me he thinks I'm talented at it, and I'd like to be able to develop my own projects for VR, so I'm considering changing my major from English to something related to VR or game development. What kind of coding do you do?

[cybereality]

What kind of coding do you do?

Different stuff really. For work I do web development, game programming, etc. On the side I do more experimental stuff like this ( http://www.youtube.com/watch?v=hcOdFAGXMTk" onclick="window.open(this.href);return false; ). I also wrote a utility to convert MPO images into standard JPS files ( http://www.mtbs3d.com/phpBB/viewtopic.php?f=3&t=4419" onclick="window.open(this.href);return false; ). I do have a lot of ideas and aspirations, but there is only so much time in the day.

[cybereality]

This looks like just what I was looking for. Should be easy enough to get working on PC and the price is right!

[youtube]http://www.youtube.com/watch?v=E5-NNzaL_1g[/youtube]
http://www.blazeeurope.com/mel-b-fitnes ... d_126.html" onclick="window.open(this.href);return false;

Now the only question is: does it come in black?

[Synexious]

Does that allow for 360 degree turning? Do you think it reacts fast enough for twitch play? Something else I've been thinking about - how to handle independent head and gun turning. The Sixense can do it, but some games, especially console games, can't, so I suppose turning would have to be linked to the gun, and I guess the HMD couldn't be used to look around.

[cybereality]

Does that allow for 360 degree turning? Do you think it reacts fast enough for twitch play? Something else I've been thinking about - how to handle independent head and gun turning. The Sixense can do it, but some games, especially console games, can't, so I suppose turning would have to be linked to the gun, and I guess the HMD couldn't be used to look around.

Yeah, its just tracking that you're making a walking motion, the direction doesn't matter. However you would lose the optical tracking on the Wiimote, so it could only be used as an accelerometer (which isn't too accurate for aiming a gun). But you could use it with a gun peripheral as the physical trigger and then use some other method for tracking. You could combine this with headtracking on an HMD, but you would need to be using custom software or have a special driver that would allow you to use this in games.

[Okta]

Wow thats interesting. Would work well with my setup. Heres what i use-
Huge ugly huge FOV hmd with gyration air mouse fixed on top for head tracking.
Wii gun holder with wiimote,motion plus and nunchuck all fixed together for controls and gun aiming. The accuracy is acceptable for this with the motion plus.

Its a dual tracking setup- you can look with your head or gun and it takes some getting used to but is an acceptable solution considering bugger all games have dual tracking.

I have been looking for a walking simulation so the wii pedometre might do the trick, have to see if there is anything in glovepie for it.

Edit- does that thing actually do anything or have they just programmed the accelerometer to move forward when it jiggles? Could do that with glovpie anyway...

[cybereality]

does that thing actually do anything or have they just programmed the accelerometer to move forward when it jiggles? Could do that with glovpie anyway...

Interesting point. It may simply hook into the existing sensors to trigger the forward motion. That would be the cheapest solution, right? For the price, though, its probably still worth checking out.

[3dvison]

What about this,,
Iif you like the running in place idea for games like computer first person shooters. You could use some old wireless keyboard or mouse to take apart and place a button into the very far forward toe area of a pair of shoes and another button in the heel area.
It could be done in such a way that when you stand flat footed the buttons do not get pressed, but running on your toes would press the toe button down which would be mapped in the game as forward so the fasster you run in place on your toes the faster forward you go. The Backward button would be in the heel and work the same way, you run in place backward on your heels.

Or could you just rig up a DDR dance pad for the same thing ?

[cybereality]

Or could you just rig up a DDR dance pad for the same thing ?

Yeah, I've done that. It was pretty ghetto, but actually quite fun.

[Synexious]

Okta, is your setup wireless? Do you have any documentation available? Cyberreality? Anyone else? I'd like to see some VR setups in action. You're right that almost no games have dual tracking. Are there any besides the Sixense-modded ones, like Half Life 2 and Black Ops? I don't think Sixense has actually been released though; I can't find a place to buy it.

[cybereality]

I don't really have much of a VR setup anymore, just a 3D monitor at the moment. But I have some plans to build one in the near future.

The only game I have played with dual-tracking was UT2004, with the VR920 mutator on it. It wasn't something that came built-in to the game, but Epic made it relatively easy to mod. I imagine you could do this with a driver, but you would likely have to customize it for each game.

I don't think the Sixense is out yet, maybe later this year. I will probably get one, although the first model is wired, so that may be troublesome for a VR setup.

[mAchiNE]

Interesting Topic, I have been thinking of doing this myself. A few things that may be of interest:
Firstly Sound. the most realistic 3D surround sound you can have is Binaural. This is basically 3D for your ears and only requires you to have stereo headphones, but will ONLY work when listening on headphones, on speakers it will sound like normal stereo. If you have not listened to a binaural recording i suggest you listen to this: http://www.youtube.com/watch?v=IUDTlvagjJA" onclick="window.open(this.href);return false; Binaural recording is done simply by recording sound from either a real head or dummy head with the microphones positioned where your ears are, this works the same way stereo 3D does for your eyes, it works by providing each of your ears a separate sound that they would get if you were actually there and your brain does all the hard work of combining those sounds together allowing you to pinpoint where the sound is coming from based on the slight differences in sound between left and right recordings. How does this apply to VR? well I am no programmer but I don't think it would be difficult (correct me if i am wrong) to emulate this in any game or virtual environment, you simply need to add a "dummy head" into the sound engine and output your left and right stereo channels from the left and right side of the virtual dummy head. I believe the Creative X-Fi CMSS-3D Headphone mode does this in games with open AL but i have not tried it for myself to confirm. This combined with a good HMD would provide a very immersive experience as far as audio visual goes. You can get Laptops with the X-Fi chip built in if you are going down the PC in a backpack road or your other option is to get a Dolby Headphone compatible sound card which simply uses an algorithm to convert a 5.1 signal into "binaural", this is far superior to so called "5.1 Headsets" that actually have 5 speakers in them, but as with depth enhanced pictures this is not as good as a proper Binaural recording. You can get a great pair of sennheiser headphones HD555 for a reasonable price and if combined with one of these options will give you good 3D Audio without compramising on sound quality.

Secondly Haptics I have the TN games 3rd space gaming vest, it has an air compressor so is noisy and only suitable for use with headphones, if you are going for a mobile rig you will need to attach the compressor to yourself and power it off a 12V battery. I suppose you could attach a backpack to the vest and use it as the base of your rig. As for the experience of using it? Very immersive, obviously it doesnt feel like getting shot in real life but none the less the impacts it provides make the game more realistic and allows you to feel where the damage is coming from, currently there are only 8 impact zones 4 on the front and 4 on the back, this is enough but it would be nice to see at least double that in future revisions of this product.
I also hvae the Novint Falcon with pistol grip, this device is awesome, the learning curb is steep but the feed back is excellent. It is however usless for a mobile rig only suitable for a static setup (as it is designed for use on a standard desk PC setup). Although if you did get something like the Gamerunner treadmill thing then attaching a Novint Falcon to it might work would be alot better than the gamerunners standard control.

Thirdly Environmental Effects I think a ButtKicker Gamer (or Buttkicker Hometheater) would be perfect, it is sound based (runs off the analogue sub out on a 5.1 signal) and so will work on any game on any platform and as such requires no support (drivers etc) of the company in the future should they dissapear. It essentially is a sub sonic subwoofer and as such it provides force feedback for low frequencies at very little sound, this for example allows you to feel explosions in FPS or the rumble strip on the side of the track, or a crash in a racing game. I have one of these and I cna tell you it works very well and is perfect witha good set of headphones. You could attach it to the Gamerunner or if going semi mobile (i.e. bacpack rig but running on spot with pedometer or kinect) you could build a smallish (1mx1m or whatever you have space for) platform and put rubber feet on it and attach the buttkicker to that. The buttkicker gamer is relatively cheap and definately worth the money, a lot of racing sim people use them and swear by them. I have used it in First Person Shooters and combined with the 3rd Space vest gives a very good overall feeling of haptic connection to the virtual environment for not alot of money. If we could get the recoil feed back of the Novint Falcon on a wireless hand held device (even to a lesser effect) then it would be even better, I had thought about using a small solenoid attached to a Wiimote (replacing the FFB rumble motor in the Wiimote) but I do not have the programming skills to "borrow" the feedback information from the Novint driver and feed it to the Wiimote, maybe Carl the creator of GlovePie could do that but I think he is nolonger developing that program anymore?

Sorry for the long post I hope this info is useful

[mAchiNE]

Cyber that Wii Pedometer looks like it has great potential for Home VR setups I thought about using kinect for tracking when I am running walking reloading stabbing etc but other than walking most of that can already be done by a Wiimote and with this Pedometer attachement you have the freedom potentially to move 360deg and still track weather you are walking properly where as you would have to be facing the kinect for it to track you properly. Does anyone know if the Wii Pedometer can track the difference between walking and running on the spot?

[PalmerTech]

Machine, if you make two separate threads for both binaural sound and environmental effects, I have a lot of good stuff to post in them.

[cybereality]

@mAchiNE: I really like this haptic/force-feedback setup you are talking about. I also have a Novint Falcon, and its pretty good for the few games that it works with. However it is pretty useless for a full-freedom VR setup. A butt-kicker in a backpack sounds like fun though. It would also be nice to have a wireless gun with real kickback (like the Time-Crisis arcade unit). Might be hard to write a driver for that, but I guess it could work for custom applications. I have some aspirations of writing a VR input/output kind of driver (sort of like GlovePIE, but more advanced). The idea is that on a basic level you could use various input devices (3DOF/6DOF) controllers, headtrackers, Wiimote/Move/Kinect, etc. but also support output like for force-feedback, vibration, haptics, motion-simulation, etc. Even more experimental stuff like treadmills, fans, water, smell-o-vision, whatever you want. But who knows if I will ever start that. I've got so much stuff I want to work on.

[mAchiNE]

@Cyber I'm not sure attaching the buttkicker to yourself would improve the experience, if its attached to you directly you would feel where the effect is coming from (the device) instead of the illusion of the effect coming from the environment, and as it is mainly environmental effects that it provides, its better that it shakes whatever it is that you come into contact with rather than shaking you directly, however I could try this out and let you know how well it works. I Agree the Novint Falcon is useless for full freedom VR but i still think a wireless device with similar recoil FFB is possible only problem as you said is the drivers, if only we could get the FFB info out of the Novint driver and send it to the Wiimote it wouldnt be hard to connect a solenoid instead of the Wiimotes FFB motor, but i guess there might be legal issues in doing this.
Here is an example of how solenoids can be used for haptic feed back in small mobile devices: http://www.youtube.com/watch?v=Sk-ExWeA ... r_embedded" onclick="window.open(this.href);return false;
That VR Driver Application sounds like a great idea, could be very useful!

[PalmerTech]

http://www.mtbs3d.com/phpbb/viewtopic.php?f=120&t=11970" onclick="window.open(this.href);return false;

I used haptics in one of my HMD projects, and it works really well. It shakes your whole head, and down your spine, pretty awesome for explosions and the like!

[mAchiNE]

This could be of use for this setup when it comes out: http://www.novint.com/index.php?option= ... Itemid=178" onclick="window.open(this.href);return false;