I don't know how to have fun with the kinect and egoshooters

[Mitbekommer]

Hi guys!

So I spent a lot of time trying to use my kinect with current games but no matter what I try I always come back to playing using my keyboard an mouse. The only game that really works, which I only play using the kinect is street fighter

So do you have any suggestions how one could use the kinect to really enjoy ego-shooters or in general games which make you walk through huge areas and that require precise aiming at times.

[pierreye]

I had a kinect but I don't think kinect is useful where games need precision. I even struggle trying to use Kinect as mouse replacement. If you use it for gesture that will trigger some action, that would be fine.

[Mitbekommer]

I am addicted to creating new ways of using the kinect with games. I found the biggest problems to be

1. Rotation. If you don't face the kinect anymore it does not recognize you.
2. Space. You can hardly emulate walking in huge areas with just walking in place or moving one leg forward. You have to actually move.
3. Fast precise movement. See above.

Do you know a forum where people exchange ways to use the kinect with games? I feel like reinventing the wheel here.

[bobv5]

I assume you are using it with pc? Are you using a HMD or monitor? A projector would be good if you have the cash, set the ingame fov to make people actual size, and have a one wall cave.

Personaly I would use keyboard/mouse for the main controls (or maybe wiimote and nunchuck), and try to use the kinect for things like grenade throwing, issueing squadmate orders (either with voice commands or hand signals), ducking, jumping, leaning round corners, reaching over shoulder to grab a different weapon, that sort of stuff.

Either that or sell it before it loses too much value and find something more suitable for PC use. Perhaps Razer Hydra.

[Mitbekommer]

I assume you are using it with pc? Are you using a HMD or monitor?

Both but yesterday I found the HMD to not work with a kinect because of the rotation issue.

A projector would be good if you have the cash, set the ingame fov to make people actual size, and have a one wall cave.

Nice. I don't have the cash.

Personaly I would use keyboard/mouse for the main controls (or maybe wiimote and nunchuck), and try to use the kinect for things like grenade throwing, issueing squadmate orders (either with voice commands or hand signals), ducking, jumping, leaning round corners, reaching over shoulder to grab a different weapon, that sort of stuff.

How would you use the keyboard while making those movements?

Either that or sell it before it loses too much value and find something more suitable for PC use. Perhaps Razer Hydra.

I just watched a video about Hydra. I don't see the value in it. Apart from gaming I use the kinect's 4 microphones for DD5.1 encoding of my podcast which works quite well. So I won't sell it.

[bobv5]

Not impressed with my ideas then?

"How would you use the keyboard while making those movements?"

For some you would have to take your hands off. Leaning, you just lean. Same for ducking. I was trying to think of a compromise that allowed you to play the game well and still use kinect.

"I just watched a video about Hydra. I don't see the value in it. Apart from gaming I use the kinect's 4 microphones for DD5.1 encoding of my podcast which works quite well. So I won't sell it."

Fair enough. The impression I got from the title of the thread was that you had bought kinect and were very disapointed with it.

[Mitbekommer]

Fair enough. The impression I got from the title of the thread was that you had bought kinect and were very disapointed with it.

Don't be sad that I didn't choose your ideas.

I am disappointed with the kinect's ability to be used as a firstperson-game controlling device. I felt so close to having developed something but having passed all the hurdles of coding I found each method to not be more entertaining than actually playing using my keyboard and my HMZ-T1.

Some things I found out about gadgets in the past are:

1. You can use the kinect very well for Dolby Digital Encoding
2. You can use the HMZ-T1 very well for 3D gaming.
3. You can use the Kinect very well for Street Fighter using FAAST
4. You cannot use Android's position sensor to be used as a head tracker.
5. The HMZ-T1 is very heavy and makes my nose red

[Zaptruder]

I am addicted to creating new ways of using the kinect with games. I found the biggest problems to be

1. Rotation. If you don't face the kinect anymore it does not recognize you.
2. Space. You can hardly emulate walking in huge areas with just walking in place or moving one leg forward. You have to actually move.
3. Fast precise movement. See above.

Do you know a forum where people exchange ways to use the kinect with games? I feel like reinventing the wheel here.

Hello. Are you able to provide more info about how the 'walking in place' function was lacking? Was the feel of walking in place unconvincing because it fails to map movement in a 1:1 fashion? Do you think it could be done better and more convincingly given enough time and resources?

Like... if you could make it so that lifting up your knee and putting it down, would constitute a step in the game - with little to no lag... would that remedy the problem in your opinion, or is the problem deeper than that? Is it a case of latency? Or the way it's implemented is more gestural than it is a '1:1' mapping of your physical actions? (i.e. putting one leg forward).

[Mitbekommer]

Hello. Are you able to provide more info about how the 'walking in place' function was lacking? Was the feel of walking in place unconvincing because it fails to map movement in a 1:1 fashion? Do you think it could be done better and more convincingly given enough time and resources?

I think you need a sphere to walk in or above. The former already exists but is naturally huge. Basically you want to feel in the game, so movements of the figure are to be equal to your movements.

Like... if you could make it so that lifting up your knee and putting it down, would constitute a step in the game - with little to no lag... would that remedy the problem in your opinion, or is the problem deeper than that? Is it a case of latency? Or the way it's implemented is more gestural than it is a '1:1' mapping of your physical actions? (i.e. putting one leg forward).

You want to include movement to the game as natural as possible. Maybe you can accept walking in place but being forced to not turn more than 45° spoils it. Especially using a HMD.

I think one should replace the kinect with a suit with gyrosensors close to each joint for this.

Using no HMD the kinect could work again as you want to watch the screen. And when you are watching the screen you are facing it. So maybe there could something be done, but how would key mapping have to look like to get the most realistic feeling.

In which situation in real life are you confined to this situation? Maybe in a car. You cannot move but control the movement.

Or maybe it is really necessary to go deeper into game hacking to transfer coordinates.

Am I the only one who is into all of this?

[Zaptruder]

Hello. Are you able to provide more info about how the 'walking in place' function was lacking? Was the feel of walking in place unconvincing because it fails to map movement in a 1:1 fashion? Do you think it could be done better and more convincingly given enough time and resources?

I think you need a sphere to walk in or above. The former already exists but is naturally huge. Basically you want to feel in the game, so movements of the figure are to be equal to your movements.

Like... if you could make it so that lifting up your knee and putting it down, would constitute a step in the game - with little to no lag... would that remedy the problem in your opinion, or is the problem deeper than that? Is it a case of latency? Or the way it's implemented is more gestural than it is a '1:1' mapping of your physical actions? (i.e. putting one leg forward).

You want to include movement to the game as natural as possible. Maybe you can accept walking in place but being forced to not turn more than 45° spoils it. Especially using a HMD.

I think one should replace the kinect with a suit with gyrosensors close to each joint for this.

Using no HMD the kinect could work again as you want to watch the screen. And when you are watching the screen you are facing it. So maybe there could something be done, but how would key mapping have to look like to get the most realistic feeling.

In which situation in real life are you confined to this situation? Maybe in a car. You cannot move but control the movement.

Or maybe it is really necessary to go deeper into game hacking to transfer coordinates.

Am I the only one who is into all of this?

The obvious ideal for a VR system is a complicated exoskeleton that can restrict the user's movement (providing tactile feedback along the way), but also allow them a full range of movement while walking in place (i.e. you have the exo skeleton parts your legs fit into, and the actual exoskeleton legs that stay stationary). Such a solution could even solve angled hill walking - the exoskeleton would restrict the range of motion to match the perceived virtual environment; restraining the degree of ankle turn, and even how far down the knee can travel.

Of course, this technology will also never be consumer level hardware, and will as a result never serve to spur consumer level content development.

While it's not ideal - walking in place seems to me like the most obvious solution for marrying some degree of proprioception feedback with interaction in the virtual environment.

Under the most typical cases (i.e. foward walking or running), it would provide the user with the greatest degree of synchrony. As the game required more complicated movement (strafing particularly), the convincing synchrony between movement of the limbs and visual in the game space would be reduced...

But the real question is - is the quality so unsatisfactory as to not serve as a viable replacement for simply moving around with a gamepad? Because that's what any realistic, practical VR control system has to beat out in terms of function, immersion and practicality.

[Mitbekommer]

What we need to find are answers to the question how to map movement of a user to accepted keystrokes / mouse movement in an average 1st person game.

Let's use dead island for instance:

We have

- voice commands
- movement

movement boundaries:

- movement only within captured area in between 4 meters and about 1,50 meters from the kinect
- user can turn at most 30° in each direction till movement detection gets lost


Keys to map:
Attack
Activate
Aim/Counterattack
Jump
Crouch
Strafe Left
Strafe Right
Walk back
Walk forwards
Run
Kick
MedKit
Reload
Fury
Light
StartCar
Drop Weapon
Upgrade Menu
Map
Inventory

Plus Mouse movement within menus - activating within menus, moving the charaters point of view.

[Zaptruder]

Oh there's no way you'd limit yourself to just Kinect motion controls.

You'd want to have a set of Razer Hydra motion and Kinect motion controllers - for limb, button and stick based goodness.

Kinect by itself was always a turgid idea, even while kinect would've made the perfect supplementary control device.

So with that restriction lifted, you can do things in a much more intuitive fashion.

From a modding point of view, you're going to see some awkward overlaps - swiping your arm to swing a melee attack will also work to fire off a bullet - where you'd really want to pull the trigger on the controller to do that.

But from a ground up game design perspective, if you were designing for such a scheme, then there's no reason why you wouldn't do limb tracking and object modelling - allowing you to do things like swing a melee weapon naturally (from any angle, at any speed, just as you would IRL), and pull a gun up to iron sight it.

As for movement, which is why kinect is useful - you would chain movement speed to the degree of lift of their knees, and the speed of that lift. In modding terms, that would be like having an external program that captures kinect inputs and transforms it into an analogue stick output - the faster you run on the spot, the faster your character runs in the game. If you pump it fast enough, the external program can add a modifier key for 'sprint' - for games that use a specific sprint button in addition to the analogue stick.

Strafing left and right can be accomplished either by stepping left and right of a 'home' area, or simply having the direction of movement indicated by the analogue stick for the user.

I don't think on the spot camera based movement controls will ever give maximum immersion - but if you do them right, with the right set of controllers to supplement, it'll be at the very least, a worthwhile alternative to simply sitting on a couch and twiddling thumbsticks.

[Mitbekommer]

Zaptruder, I don't understand your message.