Truly immersive (AKA "Holy crap this is real") VR simulation

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Truly immersive (AKA "Holy crap this is real") VR simulation

Post by PalmerTech »

The following writeup is the result of the gear in this thread: http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=13779" onclick="window.open(this.href);return false;
You stand in the middle of a large, brightly lit warehouse. Rows of high end computers hum on the far end of the room. Cameras, blinking lights, and stacks of expensive simulation gear litter the walls and rafters, a sight that on their own would normally inspire a feeling of awe. Today, though? Your focus is intently concentrated on the backpack being tightened around your waist, and the lightweight headpiece that you are adjusting on your head. The lightweight fabric that drapes from the sides blocks out the harsh glare from the overhead lights, and you are completely in the dark, feeling the typical loss of spacial awareness that accompanies sensory deprivation.

You hear a loud, distinctly mechanical "click" as the backpack you are wearing is switched on, and the cooling fans carry a slight vibration into your body. Split seconds later, though, a bright flash of light appears in front of you, and what appears to be a gravelly floor stares you in the face, at a twisted angle that very much contradicts what you would expect to see, looking straight ahead. What gives?

WHOOSH! All of a sudden, the image spins at a breakneck pace as the tracking cameras along the roof of the warehouse lock onto your body, shifting your view to the correct position. After a few seconds of jitter... the view suddenly stops shifting, settling on a perfect, dead ahead view. Your entire field of view is engulfed with an enormous scene, and your eyes take a moment to focus. You do not feel any eyestrain, and the image before you is even larger than what the average soldier gets to see while wearing a standard issue combat helmet.

You seem to be standing on a post-apocalyptic bridge, what used to be a roadway that carries cars. Rust runs the lengths of the thick iron beams above you, and the road is littered with debris; A few old tires, weeds rustling in the wind, and a burnt out humvee all sit in their respective spots. The asphalt of the bridge has some superficial damage, but is in good condition, for the most part. You glance down, and quickly step back; your foot had been mere inches away from a sharp, rusty spike protruding from the ground, and your instincts want it as far away from your foot as possible. Slowly, you raise your head, looking past your immediate surroundings.

Underneath the bridge runs a strong river, though it is clear from the expanse of riverbed laying dry alongside it that there had been better days in the distance past. Alongside the river, and at either end of the bridge, is a forest. Beyond the forest, many miles in the distance, you see a range of tall, snowcapped mountains. It must be Fall, as many of the trees are mottled in shades of red and yellow, dropping leaves in the wind. Wait, the wind?

Turning slowly around, you can feel the wind press at your body, hitting different sides as you take careful steps to full circle. Wizardry! Or, more likely, computer controlled fans lining the stage. But at this moment in time, it certainly does not feel like it! You decide to walk along the bridge to get a closer look at your surroundings. Taking slow steps, you look up to the sky, rotating your head. As you rotate your head, your surrounding stay in absolutely perfect orientation. You try to convince yourself that you are looking at a screen, but staring into the blue sky, focusing into the vast distance beyond, it is hard to believe that is the case. You look down once again and, growing more confident, take longer strides down the bridge. After about 70 feet of walking, you stop once again, crouching on your knees to get a better look at the large tire that has rolled up against a length of twisted, rusting steel. You lean forward, peering inside the tire, and note the small puddle of water that lies inside the wall. Very, very slowly, you pull your head back out, lest you hit it on the side of the tire.

Up till now, you have been alone in the simulation. All of a sudden, though, you hear someone calling out from where you originally spawned, on the other end of the bridge. Now fully confident of the world you are in, you spring about 60 feet back to where you came from, meeting another avatar portraying a US Army soldier in full desert gear, carrying a large handgun. You salute, and reach out to shake hands. As your hands meet, you feel the real touch of warm skin, which makes sense, considering you are really shaking hands with the physical person that this avatar represents! "So, pretty cool, eh?" You answer in the affirmative with a mirthful voice, betraying youthful enthusiasm that makes the grizzled soldier laugh. "Well, lets see how this new Kinect based tracking system works. Not wearing any body markers, so if this works, we are in good shape." You take a step back, and as he makes a series of arm and leg movements, you let him know that everything seems to be working fine. Without warning, the soldier jumps forward, and swings a fist towards your face. Startled and scared for your safety, you leap back, tripping a bit onto one knee. "Hah! The near field visual cues are pretty realistic, eh? Was that voluntary, or instinctive?" You assure him that it was in no way voluntary, and wonder at the wisdom of scaring people so badly when they are wearing a $100,000+ simulation suit.

And then as quick as he came, the soldier thanks you for the help, and blips out of existence. You know in your mind that he was really just a software engineer controlling a virtual body, but your subconscious is having a pretty hard time believing that. You look around the bridge once again, and walk over to the side of the bridge. Crouching underneath a crumbling sheet of iron and out over a small concrete ridge, you step onto a metal grated walkway on the edge of the bridge, with just a thin metal railing between you and certain death. You lean over the railing without touching it, peering at the rushing waters far below, and a sense of vertigo kicks in. You are usually not afraid of heights, but standing on a bridge that feels ready to fall apart and crash into the river? Well, that is a special situation. You, without thinking, reach out to the railing to steady yourself; To your surprise, your hand clasps around it, the cold surface of the metal draining heat from your hand. You instantly reach away from the sharp temperature change, then slowly reach back out. Bracing both hands against the railing, you lean far out over it, getting an even better look at the waters rushing below you. You can see your hands tightly holding the railing, and you shift your arms further along the railing, watching them with astonishment, as you marvel in the flawless accuracy that they are tracked with.

Seconds later, you hear a a voice echoing out of the sky. "Okay, looks like the cameras we added are syncing to the tracking server just fine. I am running all the shutdown procedures now, so stand still. If you are walking around when tracking goes out, you might get unbalanced and trip on yourself." You follow the command of the voice, dropping your arms to neutral position and staring at the other end of the bridge. You know the tracking has shut off when the slight movements of your head vanish, and take a final look at the scene, trying to see the edges of the scene. Try as you might, though, you can only barely see the edges of the lens, even as you strain your eyes to rotate as far as they can. Growing tired, they snap back forward, once again engulfed the torn world of the bridge... And that is it. It winks into blackness as fast as it blinked into brightness, and the entire world is put into suspended animation on a solid state hard disk. You hear a now familiar click as the backpack powers down, and reach up to your head. You are surprised at how far the headset sticks out from your head, having completely forgot that it was even there for the past 20 minutes. Twisting the ratchets that loosen it from your head, you carefully tilt it off, clipping it to a large stand on the side of the room. You slide the backpack off, and clip it to the same stand.

You take another look around the warehouse, and marvel at how real it all seems. Wow, the shadow effects on the bright lights are amazing, and the tracking is flawless!

In the next moment, you remember something incredible: This IS the real world. The bridge that you were on? That was just a simulation. But even as you stare at the gear all around you that made it possible, you have a very hard time believing yourself. As far as you are concerned, the bridge, the forest, and everything around it is just as real as the hard concrete floor you have your feet planted on.
I told you guys I would give a writeup on what it feels like to use the Wide5/body tracking/Unity based engine setup we have at my work, and I think this is the best way to do it. It sounds crazy, I know, but The Matrix is so much closer than we all think. Right now, I am astounded that nobody else who has used these systems has ever written about them! These are life changing experiences we are talking about, keeping it all to high end government research labs is a travesty! I know why it is the case, and I do not begrudge anybody in the VR industry (These setups cost a lot of money to make, and they are not making fat profit margins off them, either), but as a gamer, as a DIYer, and as a person who dreams... I want these everywhere. People need to experience this to believe it.

Any questions?
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by cybereality »

WOW!
I'm speechless.


So how long do you think we will have to wait until something like this could be sold as a kit for $999?
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by PalmerTech »

Funny you ask! I was actually thinking quite a bit about that.

Optical tracking would be the only cheap way to go, I think. Magnetic would be better, but the setup and calibration would be too hard, not to mention that scaling it gets expensive per square foot. Now... Consider that in 2005, tiny cell phone camera modules were capable of 640x480 at 15fps. These days, we have ones that do 1280x720 at 60fps, all in a module that costs less than $10! If the current rate of progress keeps up, we will easily be seeing 4k cameras at 120fps by 2016, at which point optical tracking can be pretty darn nice. And best of all, you do not need great low light performance or color accuracy like you do with a phone camera! All you need is resolution, and speed. Nothing else matters. Let us assume that you outfit a small room/garage with 4 cameras, one for each corner. Assuming it is a wired setup (Like most surround speakers, really), I am going to assume you could do it for $50 a camera, total of $200. Yes, that is a total BS number I pulled out of nowhere, but this is just a mental exercise anyways.

HMD? Oh man, this is where things get crazy. RIGHT NOW, today, were I given $1,000,000, I could put together a relatively small run of 640x480 HMDs with 120 FOV using off the shelf optics for somewhere around $400 each, including all the expenses like an injection mold for the case, and packaging. But in the future? We already have (Right now, as in today) 4.7", 1280x720 panels in phones that only cost a few hundred dollars, and the screen probably costs less than $100 in bulk. If I had the aforementioned $1,000,000, that would be enough to design and produce a driver board for those panels, one that is capable of shifting the image. Feed it a 1080p image, scale it down, shift the image to the side of either panel, and BOOM. You could have 140+ horizontal FOV, with 720p per eye, and 70% or so binocular overlap. Once you got the NRE (Non Recurring Engineering) costs like optical design out of the way, an HMD like that could cost under $500 a unit. 5 years from now, who knows? 5 years ago, the cell phone standard was a paltry 320x240, and look how far we have come.

If someone invested a few million dollars in it, and especially if game studios became interested, there is no reason we cannot have this kind of experience in 5 or 6 years. The problem? If nobody does... Well, it is going to be a loooooooong time. :(

Another thing to mention: Most people do not have a 120 foot long area they can dedicate to VR. If they did, they probably do not have a wife, nor much of a shot at one! :lol: That is why I think VR arcades could make a comeback.
User avatar
Bishop51
Binocular Vision CONFIRMED!
Posts: 243
Joined: Thu Sep 22, 2011 11:05 am
Location: Vancouver Island
Contact:

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by Bishop51 »

I think we all knew this level of immersion existed out there locked away in research circles (I mean how could it not after all these years) but it's absolutely awesome to have an insider spilling it for us here!

Related to the VR market; I don't think active mobility capture will have real relevance until there is a competent AR system involved which accounts for Joe Blows unique living space constraints. For that reason alone I can't see the average consumer ever dedicating the kind of space required to make that sort of setup practical. I would imagine in the 20 year range there would be sensory systems which trick the body into feeling motion without actually moving at all and that's when the real breakthrough will come (a bit like this system but used in a stationary rig http://youtu.be/ZlNfBrXYYTc" onclick="window.open(this.href);return false; ). The most likely success in the short term (with a fully immersive/visually obstructive display system) is stationary head and arm tracking. You want the average Joe to be able to sit comfortably and still have that deep sense of immersion but within the safe confines of his desk or armchair.

Palmertech's HMD: There are other ways to get the kind of funding you're looking for but you would have to reduce your bottom line a bit and maybe go with a very limited run. I actually think there's some wisdom in sticking with the homebrew/DIY base on this one to start, provide them with the disparate bits of hardware but package things in a way that makes the process attractive, intuitive and marketable. Your best bet would be to approach it like the driving/flying simulator pit market in which you go for the niche crowd and grow it organically out from there. I'd actually really like to talk to you about that!
User avatar
Aeroflux
Binocular Vision CONFIRMED!
Posts: 217
Joined: Mon Jul 09, 2007 9:00 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by Aeroflux »

:woot Somebody likes Inception...lol. That was a badass summary!

I'm all for this setup, but to get it to people like us...well that depends on the whether we blow each other up by then. Technology is already here, it's society that needs to evolve. There has been, is, and always will be a gap between the bleeding edge of tech and the acceptance of people. Norms have to be reconsidered. Were we not investing in war, would this tech even have advanced this far? I don't think so...necessity breeds innovation. We want to keep our soldiers alive, and training in a non-lethal way is the underlining fundraiser for VR. Still, I feel the same responsibility that Dr. Angelo felt in Lawnmower Man. We train to kill, quickly, efficiently. Survive. But do we invest as much in the soldiers going into war as we do the soldiers who have seen too much coming out of war?
Last edited by Aeroflux on Mon Sep 26, 2011 7:06 pm, edited 1 time in total.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by cybereality »

Well I wasn't thinking about just having a stage. That looks great, but most people do not have the kind of empty space. Ideally it would be some sort of pod or orbitron type of thing (Lawnmower Man style). And a haptic suit would give the feeling of walking on the ground or something. That way you could walk forever without any hard boundaries. It could also fit in any normal sized house or apartment.

Yes, I am sure things will come down in price quickly. Especially with so much focus on mobile devices. But I feel there has been progress with consumer VR, its just really slow. For example, we have amazing trackers these days for *cheap*. The Razer Hydra has almost perfect 1:1 magnetic tracking and its $140 (w/ a game). Previously something like this could be in the high thousands of dollars. Even the Vuzix 6TC tracker is pretty good for $150. Not to mention Wii/Move/Kinect. There actually has been progress. Its just that HMDs are holding everything back. Without a good HMD there is no point in VR. I'd also like to see more progress in haptics/force-feedback. We can do better than rumble.

@PalmerTech: Have you ever got to try a CAVE system (or do they have one at your job). I would wonder how that would compare to an HMD setup like the one you are talking about. I understand you would lose movement and the body connection (ie looking at yourself in the virtual world) but I wonder if it might be more practical for someone with limited space to do in their house. I imagine with 5 or 6 projectors (at 720P+) the resolution would probably trump even the best HMDs today.
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by PalmerTech »

@Bishop: I have been looking at that path, I still need to figure out if there is even significant demand from the simulation crowd. Maybe I should make some threads on simulation forums to ask, any good places to start?

@Aeroflux: Actually, there is a lot of money going into soldiers coming out, too. One of the biggest programs at our facility is treating soldiers for PTSD (Post-traumatic-stress-disorder) using VR simulations.

EDIT:

@Cybereality: Yes, I have tried CAVEs before, we do have one. They are really great, and possibly more realistic for home VR, especially for things like data visualization (Google Earth, for example). Problem is, they are not so great for gaming. :( We have some giant rear projection plastic domes sitting in storage right now, I am going to see if I can get approval to pull one out and make a dome display with it.

Now, if gaming drivers would support CAVEs... That could be a very interesting experience.
bobv5
Certif-Eyed!
Posts: 529
Joined: Tue Jan 19, 2010 6:38 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by bobv5 »

It would be little use for flight sims (due to resolution), thats why I was asking about mrg style lenses. Could probably get away with it for driving sims though. The kopin 640x480 displays are ok for driving at about 70 (in game) fov. Never tried at 100+
"If you have a diabolical mind, the first thing that probably came to mind is that it will make an excellent trap: how do you get off a functional omni-directional treadmill?"
User avatar
Aeroflux
Binocular Vision CONFIRMED!
Posts: 217
Joined: Mon Jul 09, 2007 9:00 pm

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by Aeroflux »

PalmerTech wrote: @Aeroflux: Actually, there is a lot of money going into soldiers coming out, too. One of the biggest programs at our facility is treating soldiers for PTSD (Post-traumatic-stress-disorder) using VR simulations.
I had no idea that such a program existed...I know a few people that suffer in different ways from having served in the armed forces. All I can say is thank you!
Synexious
Sharp Eyed Eagle!
Posts: 381
Joined: Mon Aug 09, 2010 6:08 pm
Location: Houston

Re: Truly immersive (AKA "Holy crap this is real") VR simula

Post by Synexious »

Post Reply

Return to “General VR/AR Discussion”