Meant to be Seen

Hello Rifters,

I've been working on a few new control schemes for the Oculus Rift with the Razer Hydra, and have come up with this;

http://www.youtube.com/watch?v=u-92rvc_Y-E

Edit: v1.1 download link
Download here: https://mega.co.nz/#!AchXVIDK!MoYP3jENn ... KK1vfEPc_I

Edit: A few people have asked me for the source code to this, so here it is;
https://mega.co.nz/#!4cAgTZgT!dK2XYzjgi ... JoGMGGRxFU

You attach one Hydra wand to your chest and use the other Hydra joystick to walk forwards backwards. The camera is fully controlled by your head & body, so you literally have to turn around to walk the direction you want. I've tested this out on a few friends who got motion sickness in other demos, and they didn't feel any!

The chest Hydra also allows you to move around (e.g. duck down) and inspect the environment close up. I think this gives a really good feeling of immersion.

I've created this little exploration demo in UDK's Citadel level to show this off. I've hidden 10 golden orbs around the starting area, which you need to collect before you can proceed to the castle. Some of the Orbs are hard to find, and you'll need to reach up or in to grab them with your Hydra wand. A bit like an easter egg hunt!

Once you get to the castle, there's another 20 golden orbs to find. Collecting all of these will open the gate to pick up the JETPACK HANDS (like IronMan) and let you fly around the world.

WARNING: Be careful there's nothing breakable around you, my wife already knocked over a glass of wine last night!

Also, check the Readme.txt for a bunch of .ini settings you can tweak, if you're so inclined.

Nice Teddy0, I'll have a play with it tonight.
The fact you don't get nausea with the setup kind of backs up the conclusion that nausea can be caused by the virtual body not mirroring the physical one...

That's cool! I'm starting to get used to the idea of having the positional tracker on the torso. It does solve a few problems concerning weight (when compared to having it inside the rift).

Are you sure there's no nausea whatsoever after applying your technique, or is there still some residual nausea from the latency, refresh rate, IPD, focus, etc.?

Great work!

Does the hydra have an accelerometer?

I've almost finished my Android app (should release an APK today if not then tomorrow) which acts as a directional pedometer and you also mount to your upper torso.

It measures when you take steps (walking / jogging in place) by checking force spikes in the Y coordinate, as well as how you're leaning in the X/Z coords for strafing / walking backwards / crouching.

You can disable having to walk by jogging in place and instead by just leaning if you so prefer (Added this features as I have a sore tendon at the moment )

If the hydra has an accelerometer we can query than we can do the same thing with that.

In theory, they should be able to add this to the Oculus Rift as well (prior art, no go patenting it now ORV).

Great work. Cant wait until propper wireless VR control system comes out.

Cool. I agree that decoupling the head and torso and controlling direction with torso really, really reduces dizziness.

This looks great! Awesome work.

blazespinnaker wrote:Great work!

Does the hydra have an accelerometer?

I've almost finished my Android app (should release an APK today if not then tomorrow) which acts as a directional pedometer and you also mount to your upper torso.

It measures when you take steps (walking / jogging in place) by checking force spikes in the Y coordinate, as well as how you're leaning in the X/Z coords for strafing / walking backwards / crouching.

You can disable having to walk by jogging in place and instead by just leaning if you so prefer (Added this features as I have a sore tendon at the moment )

If the hydra has an accelerometer we can query than we can do the same thing with that.

In theory, they should be able to add this to the Oculus Rift as well (prior art, no go patenting it now ORV).

Sweet! I want that!

And how do you hook it up to the PC?

Java UDP server on the PC injects keystrokes.

More info on this thread: http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=17762

HydraDeck has body position here, which is interesting. That'd be possible to do with the smartphone as well. Not sure yet how to inject that into games like TF2 though.

blazespinnaker wrote:Does the hydra have an accelerometer?

No it's a different technology, this is magnetic positional and translational tracking.

Proper 6DOF.

zalo wrote:Are you sure there's no nausea whatsoever after applying your technique, or is there still some residual nausea from the latency, refresh rate, IPD, focus, etc.?

Luckily, I don't suffer from any kind of motion sickness whatsoever, so I can't be 100% sure. I'd be really interested to hear from anyone who does.

There's instructions in the ReadMe.txt on how to setup IPD and tweak the positioning, which may help also.

blazespinnaker wrote:I've almost finished my Android app (should release an APK today if not then tomorrow) which acts as a directional pedometer and you also mount to your upper torso.

It measures when you take steps (walking / jogging in place) by checking force spikes in the Y coordinate, as well as how you're leaning in the X/Z coords for strafing / walking backwards / crouching.

I tried the exact same thing! So instead of using the joystick to move forwards/backwards, you have to walk on the spot. But I could never get the movement detection quite right, it would often pickup any small movements like turning in place or bending down to look at something as a step. My motion detection algorithm wasn't great, would like to investigate this further.

It's also rather tiring while testing, so I wonder if some artificial view bobbing might be a substitute?

I had an idea for this hydra setup that I posted in the oculus dev forum thread. maybe people here would be interested

Inscothen wrote:You could have the attached left controller to your chest but still have the player be able to use the buttons for actions and the stick for moving.


The physical movement of the controller as you use it would hardly impact your avatars movement i think.

Also not being able to see your avatar's left hand move as you do would be weird. Maybe have a dummy hand in game on your chest like holding a coat closed. That way when using the left controller you'd at least get back some of that connection you have with your character.

With this setup, with the right stick you could control how open the hand is making grabbing things more like how we do naturally but you thumb would control it instead.

up on the stick is all the way open or could be used to poke or push virtual objects like buttons/switches

stick not being used is open, ready to grab

down is closed hand used for holding objects or making a fist to punch an enemy.

you could make it so if holding an object and moving stick to right or left allow using the object actions in hand, like using a lighter/torch to illuminate dark spaces

if holding down on the stick(to hold object) letting go would be like dropping it. or if you track the controllers 3d movement, you could make your character throw(when you stop holding down). it'd be kinda like how we throw in real life.

Awsome! First time that i had the feeling being there.
Walking on the spot was helping the experience, too.

Thank you so much for that!

Teddy0 wrote: I tried the exact same thing! So instead of using the joystick to move forwards/backwards, you have to walk on the spot. But I could never get the movement detection quite right, it would often pickup any small movements like turning in place or bending down to look at something as a step. My motion detection algorithm wasn't great, would like to investigate this further.

It's also rather tiring while testing, so I wonder if some artificial view bobbing might be a substitute?

Hah, it's fun while testing! That's the whole point!

I play a lot of TF2 with it. It's a blast running around, even if I am walking/jogging on the spot..

What do you mean by 'artificial view bobbing', btw?

Hey that is really great work man!
I can't try it because I'm still waiting for the Rift but it looks awesome.

There's only one part that would be great. If you could add arm and hand to the virtual Hydra.
Lot of demos are missing your own "virtual arms & hands".

Can we use the scripts in your demo only or in any other games with Rift and Hydra support too ?

This looks fantastic! Do you have to enter all your relevant geometry into the INIs? neck to eye height, hydra to neck base, etc? What are you setting when you "look at the hydra base station and press start?" I'm trying to perfect hydra positional tracking in Minecraft, and we're having calibration issues.

Wow amazing job ! Thanks for releasing this.

OutatimeTV wrote:There's only one part that would be great. If you could add arm and hand to the virtual Hydra.
Lot of demos are missing your own "virtual arms & hands".

It's really hard to do the Inverse Kinematics to make the arm bend in the right direction, and when it goes wrong it breaks immersion. Floating hands/wands isn't ideal, but it's much easier to work with

OutatimeTV wrote:Can we use the scripts in your demo only or in any other games with Rift and Hydra support too ?

I'll quite happily share all my code with anyone who wants it, just PM me and I'll send you a link

mabrowning wrote:This looks fantastic! Do you have to enter all your relevant geometry into the INIs? neck to eye height, hydra to neck base, etc? What are you setting when you "look at the hydra base station and press start?"

The main values in the INIs is the height of the Hydra Base from the floor (e.g. desk height, default to 75cm). The chest Hydra wand knows how high it is above the base station, and moves the players eyes up accordingly. The reason I did this is so my tall friends don't feel like their knees are sinking through the floor

When you do the calibration on the chest Hydra, it records the X & Y position of the wand, and the Yaw rotation. This position is the "center" of the player and the Yaw rotation is used as the forwards direction. Then any movements they make or any turning they do is mapped 1:1 from real world space to virtual space.

It also records the Yaw rotation of the Oculus and uses that as the forwards direction. I did this because I find there's quite a bit of drift in the rift.

Teddy0 wrote:

OutatimeTV wrote:There's only one part that would be great. If you could add arm and hand to the virtual Hydra.
Lot of demos are missing your own "virtual arms & hands".

It's really hard to do the Inverse Kinematics to make the arm bend in the right direction, and when it goes wrong it breaks immersion. Floating hands/wands isn't ideal, but it's much easier to work with

This makes me want a rayman VR game

Why have the tracker on the chest, vs attaching to the Rift? I did so here:

http://www.youtube.com/watch?v=ko5OdLE2Msw

(Fairly inelegantly, but it worked.)

eshan wrote:Why have the tracker on the chest, vs attaching to the Rift? I did so here:

http://www.youtube.com/watch?v=ko5OdLE2Msw

(Fairly inelegantly, but it worked.)

For some people, I think it would be a weight issue.

eshan wrote:Why have the tracker on the chest, vs attaching to the Rift? I did so here:

http://www.youtube.com/watch?v=ko5OdLE2Msw

(Fairly inelegantly, but it worked.)

Fairly?

I sure as hell won't attach an Hydra controller to my Rift, let alone duct-tape it!

eshan wrote:Why have the tracker on the chest, vs attaching to the Rift?

A few reasons, firstly to separate your bodies rotation from your head so you can walk in one direction and look in another.

The second reason is I found the accuracy of the Hydra degrades as you move away from the base station, it gets a bit of wobble. By still using the Oculus's rotation to do neck modeling and head movement, you can avoid this.

A few people have asked me for the source code to this, so if it helps you any here it is;

https://mega.co.nz/#!0Rg3XAgS!a6GTjAgSN ... DTFuh4y50o

Hello again,

I've put up a new (and maybe final?) build of this demo, which you can download here;
https://mega.co.nz/#!hNhmlQgQ!I4HcvCMkI ... 2DKwsTmbD4



Here's my changelist;
-Fixed Hydra's losing position when you go below the base (e.g. when you duck)
-Added new calibration start up process, which is more robust
-Added HMD calibration to the start button of right Hydra. (hopefully this is only temporary, as UDK is still using v0.1 Oculus SDK)
-Changed from just offsetting camera position to moving the player position when you walk around. This means you can strafe IRL to avoid obstacles as you walk! Also fixes problems
-Added a safety barrier sphere, which will become visible when you get too far from the base. This tells you when you're about to run out of cable!
-Added a warning when you've turned too far, so you don't get tangled in the cables
-Fixed rotation of Hydra wand going wonky (gimble lock)
-Tweaked jetpack physics a tiny bit

I've also moved a bunch of the Orbs into harder to find places, now that you can duck down. Happy hunting!

Teddy0 wrote:-Added a safety barrier sphere, which will become visible when you get too far from the base. This tells you when you're about to run out of cable!
-Added a warning when you've turned too far, so you don't get tangled in the cables

Genius!

It certainly helps to prevent accidents, but it highlights just how small a playing area you have with all the cables. Only about 2m x 1m area, which isn't enough really.

Can't wait for the wireless Hydra and Oculus to come out, so we can have more space. I imagine in the future, people will have an empty room in their house just to play VR in!

Thank you for that creation. I have just tried the new version. Did not used the first because I just got my Rift.

It's fun to walk around (and I do walk on place) the scenery and collect orbs.
I still get motion sick in this after a while (like in other Rift demos too, when I walk with game controller) but it's better with your demo.

I just want to ask for two things please. The safety barrier sphere is good but I would like to turn it on / off with a button or key. Sometimes it's annoying and breaks the immersion.

I think the sound when touching the orbs is too aggressive. I would like to hear a soft sound / chime.

Thank you.

Thanks! I'm guessing there's still some motion sickness that happens when the Oculus gets out of calibration due to sensor drift. You can fix this by looking forwards and pressing the start button on the right Hydra. If you look down at your feet, there's an arrow which shows the direction your body is facing, you can use this to tell if it's out of sync.

For the saftey barrier, you can change the size of it in HydraDeck\UDKGame\Config\DefaultGame.ini
HydraSafetyRadiusCM=140

I'd be really interested to know what value works for you. (Note; this is only for the sphere when you've moved away from the base, there's also a plane that runs across the base to let you know when you're about to walk into your desk that you can't tweak)

Thank you. I will edit the value.
I will be on holiday for two weeks and I will try it after that.

I have seen the black arrow and it's quite often that the Hydras are out of sync. You are right. That is causing the sickness, moving not in a straight line.
I have often corrected it with the Start button.

But all in all it's great fun. Does it save the number of orbs collected when exiting the demo ?
I had found 8 out of the first 10 and do I have to start over again ?

The Oculus drift is quite bad, because UDK is only running on the v0.1 Oculus SDK. Hopefully once they've updated it to v0.2 it'll be a bit better.

I didn't make it save the number of Orbs, otherwise once you get them all there's nothing to do the next time!

It's much easier to find them the second time around however

I've fixed a handful of bugs in this demo, I think this one will be the final release.



v1.1 - https://mega.co.nz/#!AchXVIDK!MoYP3jENn ... KK1vfEPc_I
-Simplified calibration procedure
-Fixed bug in calibration that reversed positions sometimes
-Fixed virtual Hydra wand and safety barriers jittering when you walk
-Reduced visible distance on safety barriers from 32 cm to 16 cm

Now that I've got a good control scheme, I'm going to try making a new demo with some guns and gameplay

Hey Teddy! I wanted to give you a suggestion for your demo (I mentioned this on your youtube video).


I found that walking in place (marching) removed some of the il feelings and I spent about 40 minutes in the first time. I found myself swinging my arms while I walked (while holding down the forward walk key while my arms swang).


Suggestion: can you add a toggle to the demo, so when you press and hold 'forward' on the controller it waits for the hydra to swing while walking. faster/further the swing, faster the forward movement. if you stop swinging, your player stops walking. this only works while you hold down forward walk.

thanks btw! I have found your demo to be the most enjoyable so far.

garagebattle wrote:Suggestion: can you add a toggle to the demo, so when you press and hold 'forward' on the controller it waits for the hydra to swing while walking. faster/further the swing, faster the forward movement. if you stop swinging, your player stops walking. this only works while you hold down forward walk.

thanks btw! I have found your demo to be the most enjoyable so far.

Thanks

I started to implement something similar to footstep detection, but could never get it right. It's quite hard to tell the difference between someone walking on the spot and someone bending down or turning in place.For now, I think people can just choose to walk on the spot if it helps them.

Now that I've got a good controls scheme, I'm now working on my next demo which involves guns and ducking behind things. Watch this space!

This was the first demo that really truly tricked my brain into feeling like I was in another world. The ability to stand and walk in the direction you're facing while being able to look around at the same time, combined with positional tracking is just really really awesome. I loved just going up to the gate and peering side to side, up and down. Leaning over ledges is amazing. Is there a way to recalibrate the right Hydra? A couple times it just got waaay off. Anyway, thanks for this! I can't wait for your new demo.

It's probably the Oculus that's getting out of calibration. If you look straight down, there's and arrow showing which way your body is facing. If that doesn't match up, then the Oculus sensor has drifted off.

To recalibrate it, just stand still and look straight ahead and press start on the right Hydra.

it seems that you have developed a control scheme for the Oculus Rift using the Razer Hydra, allowing players to move in virtual reality by attaching one Hydra wand to their chest and using the other Hydra joystick to control movement. The camera is controlled by the player's head and body, requiring them to physically turn around to change direction. You have created a demo in UDK's Citadel level where players can explore and collect golden orbs, and eventually unlock a feature to fly with "jetpack hands". You have also shared download links for the demo and the source code for interested users to access and customize. Additionally, you've provided a warning to be cautious of the physical environment while using this control scheme.