Conflicts between full body capture and no "true" movement?

[virror]

How do we handle the conflict between full body capture if we dont have any way of simulating real walking behaviour?
Is using a "standard" walk animation for the legs when we have controller input and overriding it with the capture when we dont?
Anyone have been thinking about this kind of problems? We will probably have decent comercial body capture long before we have decent priced ways of simulating real walking.

[cybereality]

Well they are some solutions right now.

One of them, the simple one, is simply walking in place and using the Kinect to track movements. In this case you probably can match the virtual movement pretty closely with the real movement (except the actual stride, since of course you are not moving in real life). But I think it could work OK.

Another option is something like the WizDish, which I think has a lot of potential. You could still use a Kinect to track movements but, in this case, you would only have the strides to go on and you would have to simulate the lifting of the leg in the game. I think this would provide the better experience, but I haven't tried either honestly.

Ultimately we would want an omni-directional treadmill so that we could actually just move naturally and be tracked. Not sure those will ever be cheap, but some have been built already.

[virror]

But using the wizdish and then transferring the movements of the legs to the ingame character would still make it look strange when looking at your own body inside a HMD.
The big issue here is to make it look like you are walking normally ingame and still allow for captured data to be used for animating the legs, combining those two might be tricky.
With body tracking i mean to transfer your skeletons movement into the characters skeleton ingame, might have been a bit unclear there.

[MSat]

I still believe the WizDish would make a pretty decent omni-directional "treadmill" given that you have a harness to hold you in place. The treadmill itself could detect movement, while something like an HMD-mounted Leap Motion could show the correct representation of your legs when you look at them. Of course, there would always be some sort of disconnect when you walk on uneven surfaces like a flight of stairs. I don't see a cost-effective solution ever overcoming that last challenge until we can jack our nervous systems right into the matrix.

[Aabel]

Every video I have seen of the wizdish there has been a lag when starting and stopping walking, changing direction. It looks like you have to take a full step first, then it recognizes you are walking the walking is also very jerky, I am sure that can be fixed but it's highly unnatural. Also I don't think I've seen anyone able to use the wizdish without swinging their arms like a wild ape. Is there a reason for that? does it require a decent amount of force to get your legs sliding? I could see the wild arm swinging making other actions like aiming a virtual gun a lot harder.

The obsession with virtual walking is an interesting one, from the perspective of games development I don't find it overly important. Fun is what's important and while it might be fun at first to walk around in a virtual environment I am betting the novelty will wear off pretty fast and it will become very frustrating to be limited by actual locomotion. For simulations however where creating a true to life experience is absolutely critical I see the value.

[virror]

"show the correct representation of your legs when you look at them", this is the key question im asking here, how to take whatever inputdevice you have, does not matter if its a gamepad or wizdish and combining that with looking at yourself inside the virtual world doing a "real" walking animation? Also how to combine it with the possibility for something tracking you do a kick as an example and also show that kind of tracked movement at the same time?

[MSat]

Every video I have seen of the wizdish there has been a lag when starting and stopping walking, changing direction. It looks like you have to take a full step first, then it recognizes you are walking the walking is also very jerky, I am sure that can be fixed but it's highly unnatural. Also I don't think I've seen anyone able to use the wizdish without swinging their arms like a wild ape. Is there a reason for that? does it require a decent amount of force to get your legs sliding? I could see the wild arm swinging making other actions like aiming a virtual gun a lot harder.

The obsession with virtual walking is an interesting one, from the perspective of games development I don't find it overly important. Fun is what's important and while it might be fun at first to walk around in a virtual environment I am betting the novelty will wear off pretty fast and it will become very frustrating to be limited by actual locomotion. For simulations however where creating a true to life experience is absolutely critical I see the value.

It's probably a lot like sliding around on ice, so people probably "swing their arms like a wild ape" (lol) to stay balanced. Definitely not a good thing for gaming, not to mention that I can't imagine sliding your feet giving a very compelling sense of walking. Any apparent in-game jerkiness is likely a result of the sensing mechanisms and software. As is, I don't think the WizDish is a particularly good device (and one just begging for a lawsuit in the U.S. once someone slips and cracks their skull open). Again, the only way I could see it being of any use as a treadmill is if it incorporates a body harness to keep you in place, letting you walk more naturally along the surface.

Being able to actually use your legs to move around in a game can be cool, and potentially good exercise, but I agree that it's not particularly necessary to enjoy a game - and in some ways may even be detrimental. But I guess it comes down to if people do want it, then what is the best way to achieve it? A low friction surface is definitely the easiest and cheapest way to implement an omni-directional treadmill.

[MSat]

"show the correct representation of your legs when you look at them", this is the key question im asking here, how to take whatever inputdevice you have, does not matter if its a gamepad or wizdish and combining that with looking at yourself inside the virtual world doing a "real" walking animation? Also how to combine it with the possibility for something tracking you do a kick as an example and also show that kind of tracked movement at the same time?

Well, I guess there's not much of an option if you're not actually moving your legs. Is a game unplayable because your virtual legs are moving while your real ones are not? Or perhaps that you don't even have a body in the game? The only way your question can even be relevant is if you're physically moving your legs in a natural walking motion, and then applying it to your virtual legs. There are many different ways you can track your actual legs - IMUs, Kinect, Leap Motion, or many other different options. Take your pick.

[Aabel]

"show the correct representation of your legs when you look at them", this is the key question im asking here, how to take whatever inputdevice you have, does not matter if its a gamepad or wizdish and combining that with looking at yourself inside the virtual world doing a "real" walking animation? Also how to combine it with the possibility for something tracking you do a kick as an example and also show that kind of tracked movement at the same time?

This kind of thing can be handled by a game engines animation blending/state machine. For example you have a game where the player can use a walking in place peripheral to move about the environment however high kicks to the head are a huge part of the game. Now I don't know about you but I can't do any high kicks to the head in real life which means I am not going to do them in a VR game with 1 to 1 mapping of my body controlling the avatar. The developers foreseeing my lack of high kicking ability put the high kick attack on a button on a controller, so all I need to do to high kick is press a button and the game engine blends from where my legs are to the high kick animation.

[virror]

Kicking was an extreme example, its just the thing about immersion. Most games wont need you to actually move your legs, but to be able to would be very cool in some cases.
Blending in an animation when another animation is pretty trivial, but blending in external sensor data is probably a lot harder to accomplish, thats why i want to find a good middleground or workaround : )

[brantlew]

The obsession with virtual walking is an interesting one, from the perspective of games development I don't find it overly important. Fun is what's important and while it might be fun at first to walk around in a virtual environment I am betting the novelty will wear off pretty fast and it will become very frustrating to be limited by actual locomotion. For simulations however where creating a true to life experience is absolutely critical I see the value.

Coming from a slightly different perspective - true walking really does create an overwhelming sense of immersion and locational awareness. It may not necessarily increase your scoring ability, but if implemented well and done in a properly paced game it adds quite a bit to the experience. I would argue that true locomotion adds an equal amount of immersion as wide FOV does.

[JanVR]

I agree with Brantlew. Walking in a game vastly increases the immersion. If you have a good locomotion device, combined with a good HMD like the Rift, then you'll actually BE in the virtual environment. It will be mind blowing.

[Flassan]

How do we handle the conflict between full body capture if we dont have any way of simulating real walking behaviour?
Is using a "standard" walk animation for the legs when we have controller input and overriding it with the capture when we dont?
Anyone have been thinking about this kind of problems? We will probably have decent commercial body capture long before we have decent priced ways of simulating real walking.

Its a good question that affects all parts of the body. If anything you may be less worried about your legs as you don't pay much attention to them, but if you are holding a wiimote and look down to see a large gun its noticeable. As the pareto rule says a small change like standing and walking can have a massive effect on immersion but I would expect diminishing returns as you try to refine it. It would be nice if your animated right leg moved forward when your real one does but you may not notice if the articulation isn't absolutely precise. You also have to be careful of gaps. For instance in free-walking the running might feel right but when you jump you physically hit the ground long before your avatar does. Equally, one hand holding a sword and the other a support rail could affect the illusion. But these things are still worth doing because the overall affect is far better.
I'm currently changing the Kinect skeletal viewer to work with Mirror's Edge and have already noticed some interesting things. The game forces you to rapidly press sequences of buttons at precisely the right time to progress which is a tedious way to present a challenge. Raising both hands to jump and climb feels plausible and using both arms outstretched to balance is a vast improvement on pressing A or D. But what caught me by surprise is that I found I stop and take a couple of deep breaths before doing a run up to a jump. You don't get those kinds of sub-concious reactions when sat down pushing buttons.
If you think of Virtual Reality as 'simulating what's real' then its easier to see why you would want to stand, turn and walk in an First Person game or experience. It helps that our minds seem capable of filling in major gaps so long as we maintain consistency. Re the WizDish comments you are absolutely right about the latency and intermittent motion detection. That can induce nausea as well as damaging the sense of presence so it is a crucial thing to get right. However, its also one of the simpler things to fix. Likewise the "swinging their arms like a wild ape" should go away with the new lower friction version.

[virror]

Finally one that actually responds to the original question : )
Very good feedback, i agree that the big things have to come first, even if one solution introduces another unnatural behavior i think its ok as long as the improvement is bigger than the negative effects. Sounds like a interesting project you are working on, have a thread or some more info about it?

[Flassan]

There have been lots of insightful and useful comments about it on this forum and where appropriate I've tried to respond to them. It's probably easiest to use the search at the top of the page as they appear in many different threads. I try to update www.wizdish.com and http://www.youtube.com/user/wizdish but this is where the more in-depth discussion happens. Any comments or feedback are more than welcome!

[virror]

Ahh, i thought you were talking about some other project using the Kinect, i should have paid more attention to your signature : )
I have been looking at the Wizdish before and it seems like a cool think, not exactly like walking but probably the best in-place walking simulator yet thats consumer friendly.
When can i buy it to try it out? : D

[Flassan]

We are hoping to do something like Kickstarter in the near future. We think we can make it better and cheaper soon and don't want to disappoint people by doing this just after they bought one. Of course if you are an institution that would prefer to trial it now that's a different matter.

[Lookforyourhands]

[youtube]http://www.youtube.com/watch?v=v0xg_kHnYKs[/youtube]

[cybereality]

Looks sweet, Lookforyourhands!

[virror]

Looks very cool even though the latency is pretty high.

[Zoide]

Likewise the "swinging their arms like a wild ape" should go away with the new lower friction version.

Interesting! When is the new version going to be ready? Do you have a video of it? Wouldn't reducing the friction make the dish more slippery and akin to walking on ice?

Also, what happens if you actually try to lift your feet to walk on the Wizdish instead of sliding them back and forth? Is this avoided due to balance concerns? If so, perhaps you could make it so that the center region of the dish has higher friction. That way you can keep one foot stable so that you can lift, extend, put down, and slide back the other one. This would probably require increasing the radius of the dish, but the increased naturalness of the movement would be worth it.

Thanks