Virtual Reality test chamber (Real, not speculation!)

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Virtual Reality test chamber (Real, not speculation!)

Post by PalmerTech »

Hi guys! Man, I am on a threadmaking role! :lol:

Here is the thread that describes what using this system is like: http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=13780" onclick="window.open(this.href);return false;

I have mentioned that I have a new position working in a military research lab, and I am able to show off some of our stuff. We actually have other motion capture stages/test chambers that are larger, but I am going to just go over what I am familiar with, which are the two stages we have next to my workshop.

Here is Stage 1:

Image

Not a great picture, thanks to the low FOV of the 3DS camera, but it gets the basic idea across. Stage 1 is about 120 feet by 60 feet, pretty large! If you direct your attention to the ceiling, you will note the scaffolding system that runs along the top of the entire stage. Mounted to the scaffolding are 40 motion cameras, each of which captures 3600x3600 at 480fps. The layout and overlap of the cameras means that at any given moment, you have at least 12 cameras tracking you, which means you get sub-millimeter accuracy in all direction, jitter free. The cameras are used for body, head, and movement tracking across the stage. We can have 1:1 movement mapping, or "enhanced" movement, resulting in much less effort, but with a modicum of realism lost.

Here is the far end of the stage, littered with stuff (Some of which we use as physical props for virtual objects!). Don't worry, it is cleaned up when we run simulations.

Image

Here is a (stunningly bad, curse the 3DS low light performance) picture of the motion capture suits we use:

Image

They use glowing LED trackers to each joint. They could be IR in theory, but ours are in the visible spectrum, bright red! I need to get a picture of the suits in operation, but the occasion has not come up. They are not even used most of the time, actually, since it is rare that you need perfect tracking of every single limb in your body. We also have some Kinects rigged up on the side of the stage, and have used those for body tracking when dealing with touching physical objects that are mapped to virtual ones.

Here is a picture of Stage 2:

Image

Truth be told, I do not do a lot of work in Stage 2. It is a lot smaller, perhaps 30x60 feet, and is used for projection projects. Only 12 cameras, I think.

As far as software goes, we use Unity with a lot of addons we have coded that give us the flexibility we need. Turns out, our entire system works perfectly fine on the free version of Unity! The Pro version can make the graphics look a little nicer, but everything works perfectly fine on the free version. If you have any specific questions about the engine (Cybereality?) post them here so I can ask the software engineers. If there are people who would love to have a copy of all our addons... Let me know.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by cybereality »

Wow! Looks pretty sick the setup there. I can only imagine what it would be like in action. What do they use for controllers (are there controllers)?

I am surprised to hear they are using the free version. I would have thought with all that custom hardware you would need the pro version, but maybe they found a clever workaround. Still not sure if I am going to use Unity any more. I think I will be better off doing something a little more low-level with C++. Even for just the learning experience. Also there is some stuff I want that Unity doesn't support (namely hardware PhysX) and I don't think I can complete the demo I want with software physics (even with hardware PhysX I may be limited). But we will see.
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by PalmerTech »

We use all kinds of stuff for controllers. Assault rifles, handguns, gamepads, sticks, you name it. :lol: All we have to do is stick some LED tracker beacons and buttons on whatever you want, make a profile for the object, calibrate it, and you can use it however you want.

We actually do use the Pro version, but in talking to our software engineers, I found out that the engine still works on the free version, which they usually use on their home PCs. Unity is used almost universally in this field at the moment, it seems, but people are working on integration with other engines. Without saying too much... Well, a certain company that just released developer tools for their engine, used in a certain 3D blockbuster title, is the prime suspect at the moment. ;)
User avatar
Bishop51
Binocular Vision CONFIRMED!
Posts: 243
Joined: Thu Sep 22, 2011 11:05 am
Location: Vancouver Island
Contact:

Re: Virtual Reality test chamber (Real, not speculation!)

Post by Bishop51 »

Fascinating! Thank you so much for a very rare look into military research Palmer!
User avatar
Syntax
One Eyed Hopeful
Posts: 34
Joined: Wed Sep 23, 2009 7:33 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by Syntax »

I get goosebumps when I see the pictures and read your experience in the other thread.
Just plain awesome man :woot
It reminds me of the X-files episode: First Person Shooter ;)
(season 7 episode 13)
In this episode Scully and Mulder investigating a "homocide" in a military research lab for virtual reality which aims to sell it as acrade setup for big malls and stuff 8-)

ok offtopic I know :roll:


Anyways you mentioend that u using the unity 3d engine.
Do you think its hard to transform the tracking and 360° Camera-perspective to lets say the Unreal UDK Engine or the Crysis SDK?
with that you wont need to travel to get a nice holiday on a tropical island.
keep up the good work

peace
bobv5
Certif-Eyed!
Posts: 529
Joined: Tue Jan 19, 2010 6:38 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by bobv5 »

Just wondering, could you explain what you meant in this sentence a bit better? (Hope I'm not being dumb, it is stupidly late here.)

"We can have 1:1 movement mapping, or "enhanced" movement, resulting in much less effort, but with a modicum of realism lost."

Also, would love to see any software you are allowed to share, as long as it won't get me put on some list, or have men in dark suits following me or some such insanity. Only half joking when I say that. I wouldn't be surprised if they already have people watching this site. Think I best go get my tinfoil hat, can't be too careful!
"If you have a diabolical mind, the first thing that probably came to mind is that it will make an excellent trap: how do you get off a functional omni-directional treadmill?"
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by PalmerTech »

@Syntax: It would certainly be possible, it just takes a lot of work. If you can get the developer to cooperate, especially! One of the hardest things is adding rotation/tilt, since most engines have no support for it.


@Bobv5Of course! I should have explained it better, as it is a bit of an abstract concept.

See, generally, we can map the real space to the virtual space, 1:1. For every foot you walk in real life, you move one foot in the virtual world, as well. Here is where things get fun, though... You see, people are actually pretty terrible at judging how far they are stepping without using their eyes. Try guessing how far you have walked over any significant distance with your eyes closed, it is pretty hard! What we can do is increase a person's stride length, so that for every foot they walk in the real world, they walk, say, 1.5 feet in the virtual world. Lets you move a lot more rapidly and with less effort, and has the added benefit of giving you more "space" than you really do.

We have some other really nifty tricks, too. For example, suppose you are walking down a hallway, and walk through a door and into the room. The door is located near the corner of the room. You walk over to the other end of the room, interact with whatever is there... And in the meanwhile, the door has shifted 90 degrees onto the corner next to where it used to be! Turns out that people almost never catch this trick, our minds ignore obviously conflicting data like that. Most impressive of all, the test subjects were still able to draw a map of all the rooms they went into that matched what the simulation wanted them to perceive. Why is this a useful trick? Well, you can have people walk into a room, back out again, into another room, back out... And all of a sudden, they are facing back towards where they started, with plenty of room to walk. ;) With indoor environments, you can effectively gain infinite space to walk around.

Other tricks: If someone crouches down to interact with an object (Picking up a gun, opening a box, etc), you can rotate the world around them a few degrees without them noticing. Repeat this every time somebody has to crouch, and you gain a little bit of extra space.

When people turn quickly, we rely almost entirely on our visual system for positioning, as our balance/vestibular system is not so good at rotational tracking. Because of this, we can vastly overstate how far people turn! Events can easily be engineered into the game to force this. For example, suppose a loud explosion goes off behind someone; When they quickly spin around, we can either understate or overstate their rotation, pointing them 30 or 40 degrees off axis from where they tried to turn. Nearly impossible to notice, though! :lol:

Vehicles can also be used to increase the perceived size of an area. Have a mockup humvee in the corner of the room, being represented as a vehicle on the road in the simulation. You can walk over to the humvee, drive a few hundred yards, and then get out, feeling as if you are in a huge area.

Yet another trick you can use are "elevators". Have a sheet of metal with haptic feedback mounted to the floor, and represent it in the simulation as an elevator. The player walks onto the pad, it lurches and vibrates slightly like a real elevator floor would, and they step off. In the simulation, however, they feel as if they have walked onto an elevator, gone many stories up, then exited into a whole new environment! Because of this, you can make a single indoor stage feel like a massive skyscraper, with potentially hundreds of levels.

1:1 motion mapping is what is used most of the time, but the techniques above have proven very effective in maximizing the potential of the space you have.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Virtual Reality test chamber (Real, not speculation!)

Post by brantlew »

Those are really slick perception tricks to reuse your space. It seems like physical space is one of the most insurmountable problems for free-movement VR. I've seen the giant hamster-ball things but those would really only work for a limited set of motions. The tricks you outlined might at least be good enough to do a corridor crawler type of FPS, but in general - how could you every really simulate an open ended environment?
bobv5
Certif-Eyed!
Posts: 529
Joined: Tue Jan 19, 2010 6:38 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by bobv5 »

look at omnidirectional treadmills.
"If you have a diabolical mind, the first thing that probably came to mind is that it will make an excellent trap: how do you get off a functional omni-directional treadmill?"
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Virtual Reality test chamber (Real, not speculation!)

Post by brantlew »

bobv5 wrote:look at omnidirectional treadmills.
Holy crap - that's badass! Talk about expensive though...this other VR equipment looks like peanuts compared to that thing. A case of beer to the first person who can build a DIY version of that!
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by cybereality »

@PalmerTech: Wow dude! You're blowing my mind!!!!!
User avatar
Okta
Golden Eyed Wiseman! (or woman!)
Posts: 1515
Joined: Tue Feb 12, 2008 5:22 am

Re: Virtual Reality test chamber (Real, not speculation!)

Post by Okta »

That is awesome. I had always wondered at the limitations imposed by physical space. Now i see how a "real" holodeck could work, at least using a hmd and some props.
"I did not chip in ten grand to seed a first investment round to build value for a Facebook acquisition."
Notch on the FaceDisgrace buyout.
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Virtual Reality test chamber (Real, not speculation!)

Post by PalmerTech »

@Brantlew: Simulating an open environment is definitely tough! Which is a shame, because I like outdoor ones the best. :lol: I believe we have an omnidirectional treadmill in one of our other buildings, but I have never seen it in action. I will see if I can change that. Of course, the tracking situation gets a bit more complex, as you need to compensate for the treadmill. Someone else told me in a PM that they will make a thread for treadmills, so those can be discussed further there.

@Cyber: Not me! I get to help build some cool stuff, but all those software tricks have nothing to do with me. Thank some of the brightest researchers and programmers that acadamia and the military have to offer. ;)

@Okta: Indeed, it can work pretty well. You want a large space to start out with, but once you get to, say, 100x100 feet, you have so much leeway that as long as the simulation is designed well, you can keep people going in circles forever. Yes, it is not as open as a sandbox style game where you can roam anywhere, but you still have more freedom than most linear games give you anyways.

The funny thing about props is that you can have someone who has the sole job of running around and moving props during the simulation to match the new environments. Most people actually do not touch very much when they are walking around, anyways, they respect the boundaries of virtual walls, and rarely test them. Now, a mechanical system (Like in the Portal 2 test chambers!) would be ideal, but we are a long way from that. :lol:
Synexious
Sharp Eyed Eagle!
Posts: 381
Joined: Mon Aug 09, 2010 6:08 pm
Location: Houston

Re: Virtual Reality test chamber (Real, not speculation!)

Post by Synexious »

Here's the thread. I've assessed every type of OSM (omnidirectional stationary motion) system - ODTs, spheres, tiles, and ball carpets. Of the four, only the ball carpet has any chance of making it into our homes. I'm trying to determine if one could be perfected (need to compensate for ball popout at high speeds) and built for a few thousand: http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=13784" onclick="window.open(this.href);return false;
Post Reply

Return to “General VR/AR Discussion”