Half-Life 2 VR mod with head tracking (download now)

Post by **adoral84** » Mon Jan 21, 2013 1:47 pm

While we're all sitting around holding our breaths waiting for our shipment notifications from Oculus I thought I'd share my Half-Life 2 VR mod I've been working on. If you have a Hillcrest tracker and an HMD you should check it out. Assuming you're using a hillcrest tracker and have something that takes care of the warping and SBS rendering this should work great with a DIY rift (I'll be jealous if it does though as I've been doing everything on my ski-goggle modded HMZ).

[youtube-hd]http://www.youtube.com/watch?v=5H2rL-lBxRs[/youtube-hd]

(if the video below is blocked in your location b/c of the background music click here to watch it on vimeo: https://vimeo.com/59342255)
[youtube-hd]http://www.youtube.com/watch?v=RHD3Lwll7q8[/youtube-hd]

- 3 DOF true headtracking (not mouse emulation) + a head/neck model for some basic translation
- Transparent HUD that fades in when relevant information changes (taking damage, reloading etc)
- You can lower your gun ("use" key can toggle it as well as many of the actions like sprinting, picking up items, etc) because constantly having a gun in your face (when you aren't fighting) is an immersion killer... This also hides the crosshairs so there's effectively no HUD
- Removed all head kicks from both user weapons and enemy attacks (like getting hit in the face with a CP baton)
- Full support for custom tracker orientation
- support for gamepad / mouse turning in addition to the tracker yaw
- an installer (**when selecting the installation directory, point it to you "Steam" folder)
- independent 360 degree weapon tracking with a second tracker
- custom sensor fusion for hillcrest (the out of the box one is awful) and support for 250hz firmware

Bear in mind that this is still a really early version. Any feedback is helpful, if you have any issues I'll see what I can do. Oh and start from chapter one or use the console to load any of the maps to jump to a specific spot since I know we've all played the game like 200 times.

Personally I'm a big fan of the gamepad over mouse + keyboard mostly because analog movement is an absolute must for a good experience with head tracking.

*You'll need the Source 2007 SDK installed for this mod to start.

Here's the latest version:
https://s3.amazonaws.com/HLVR/halflife-vr-0.9.2.exe

Post by **cybereality** » Mon Jan 21, 2013 2:10 pm

Cool man, thanks for releasing this.

Post by **mahler** » Mon Jan 21, 2013 4:40 pm

adoral84 wrote: - Transparent HUD that fades in when relevant information changes (taking damage, reloading etc)
- You can lower your gun ("use" key can toggle it as well as many of the actions like sprinting, picking up items, etc) because constantly having a gun in your face (when you aren't fighting) is an immersion killer... This also hides the crosshairs so there's effectively no HUD

Some really interesting choices, which I think might end up being copied by multiple early VR games.

Post by **adoral84** » Mon Jan 21, 2013 6:25 pm

mahler wrote:Some really interesting choices, which I think might end up being copied by multiple early VR games.

I think the basic unobtrusive HUD and view models are going to be really important and generally widely adopted, I think what'll be most interesting is seeing what new and existing gameplay elements work best for VR. After giving HL2 and the two episodes a playthough there are parts that I loved on the PC that were my least favorite and vice versa. I was never really a fan of the ant-lion tunnels in ep2, but squeezing through the organic tunnels running from the queen (?) antlion was a much more unique experience than say shooting at more civil protection on a city street. I also really enjoyed the vehicle sections because driving with the control and freelooking/shooting with your head just feels awesome. Environmental effects like smoke, particles and lighting are also way more impactful, while at least the shooting segments are a bit tiring.

Post by **adoral84** » Mon Jan 21, 2013 6:31 pm

I just updated the installer, you can now play through HL2 and both episodes (plus the installer is around 5% the size of the original version). Apparently I can't do a direct CS:S mod but I do plan on converting a couple of maps from counterstrike for nostalgia's sake. I'm sure valve will eventually release their own support anyhow.

Post by **Fredz** » Tue Jan 22, 2013 8:07 pm

Very nice, thanks for doing this.

Post by **adoral84** » Tue Jan 22, 2013 10:39 pm

This evening I rewrote the tracking code to use the raw sensor data and do my own sensor fusion (based on an open-source algorithm) and honestly I'm pretty stunned at the difference. There is absolutely none of the signature hillcrest jolt on the roll (or any other axis), and the tracking is noticably more responsive. I haven't had time to work in all the configurable axis adjustments but I should be able to wrap that up tomorrow and release this.

This version will also be able to take advantage of the 250hz firmware upgrade that hillcrest did for Carmack if you have that as well, I haven't even upgraded yet and the results are already so much better but I'll give this a shot.

Post by **WiredEarp** » Tue Jan 22, 2013 11:16 pm

Wow, this sounds great adoral84. It would be awesome if you could move the code for the tracker into a DLL or something, that way the community could then swap out the DLL with a community written one to support different trackers, like Hydras or TrackIR. I believe its quite easy to add 6DOF to support those (according to another user here who has done some HL2 stuff), so not only would it be vastly expanding the audience for your mod, it would also add immersion (I think 6DOF movement adds HEAPS to immersion, you can look around and under things).

I really like the HUD fade and lowering gun ideas as well.

Post by **bobjwatts** » Tue Jan 22, 2013 11:28 pm

Awesome stuff adoral84! I think the HUD and gun ideas are fantastic.

Would this work with freePIE?

Post by **3dvison** » Wed Jan 23, 2013 1:40 am

Everyone seems to have the Hillcrest, any way to make this work on the YEI 3-Space Sensor ?

Post by **adoral84** » Wed Jan 23, 2013 11:30 am

I definitely want to support multiple trackers, probably by doing exactly what WiredEarp mentioned. I'm also intending to do 6dof using TrackIR (only going to try to solve for seated 6dof for now not 360) and independent weapon tracking as well. I have a rough draft of the separate weapon tracking with a hillcrest tracker but I imagine it won't be too common for people to have a second one so again having the ability to drop in custom trackers (wiimotes, ps move, YEI, etc) will be even more critical here.

Post by **WiredEarp** » Wed Jan 23, 2013 9:28 pm

That sounds great! By independent weapon tracking, do you mean gaining hip tracking, by separating out the gun tracking from the current way it is? Or are you talking about gaining a floating gun, that you can move through 6DOF? That would be a BIG immersion booster.

Post by **adoral84** » Wed Jan 23, 2013 9:46 pm

WiredEarp wrote:That sounds great! By independent weapon tracking, do you mean gaining hip tracking, by separating out the gun tracking from the current way it is? Or are you talking about gaining a floating gun, that you can move through 6DOF? That would be a BIG immersion booster.

I'm assuming you're referring by hip tracking to something like ARMA style head free look? That could be interesting, I was planning on throwing in an option to have your movements relative to your body rather than your head for seated play, I dont think that'd be very hard.

I'm currently working on floating gun like you mentioned although I'm only focusing on 3dof initially. This should be good enough for a first pass as you tend to keep the butt of a gun on or near your shoulder and kind of pivot from that point. Unfortunately most of the gun models aren't perfect for being anything other than a decoration for the corner of the screen but there are a few standouts. I have the shotgun worked out well enough where you can tilt your head because of the neck model and actually shoot down the iron sights

I still have to chase down a bunch of stuff like correcting tracers and targeting lasers and working out some sort of crosshair setup but the core functionality is there.

Post by **adoral84** » Wed Jan 23, 2013 10:32 pm

Here's an version with the newer much improved tracking code in it, and now there's only one command "vr_calibrate" which is bound to the L key that works exactly like TrackIRs recenter so try to hold your head as level as possible when you run it.

One caveat, I'm having an issue with the freespace api throwing up on exit so when you exit the game it may lock up and need to be killed from the task manager.

-- removed, this was an old version, check the updated original post for the latest --

Post by **3dvison** » Thu Jan 24, 2013 12:48 am

adoral84,
I must try your HL-2 Mod as soon as my Diy Rift is finished(Very Soon), So I just ordered a Hillcrest.

Post by **brantlew** » Thu Jan 24, 2013 11:44 pm

Just found this. Glad you finally released. I have a similar mod that integrates generically with any tracker (and also supports 6DOF) via a FreePIE interface. I would be happy to give you the code if you want to enable this type of generalized tracker support.

Post by **3dvison** » Fri Jan 25, 2013 4:18 am

brantlew wrote:Just found this. Glad you finally released. I have a similar mod that integrates generically with any tracker (and also supports 6DOF) via a FreePIE interface. I would be happy to give you the code if you want to enable this type of generalized tracker support.

brantlew, are you saying I did not need to buy that Hillcrest after all ? Where were you yesterday...LOL

Post by **PatimPatam** » Fri Jan 25, 2013 5:33 am

Thx a lot adoral, this is brilliant! It would be awesome in combination with brantlew's work as well.

After you had some time for polishing.. could you try and make a similar VR mod for Portal 2? Pretty please???

I believe [Portal 2 + VR mod + stereo/warp driver + Hydra aiming + Rift + custom 6DOF head-tracking] could be the ultimate VR experience/demo, at least until the first proper games developed with VR in mind start being released (probably along with the Rift consumer version).

Post by **adoral84** » Fri Jan 25, 2013 5:47 pm

PatimPatam wrote: I believe [Portal 2 + VR mod + stereo/warp driver + Hydra aiming + Rift + custom 6DOF head-tracking] could be the ultimate VR experience/demo, at least until the first proper games developed with VR in mind start being released (probably along with the Rift consumer version).

I totally agree, but from everything I've read I don't think it'll be possible because AFAIK valve never open sourced the engine beyond the 2007, so you can only create full on recoded mods for games up through that. I'm not even sure if the original portal is moddable in this way, but if it is I'll definitely work on that. I'd also really like to work on L4D2 but it may be the same issue.

Post by **adoral84** » Sat Jan 26, 2013 8:05 pm

So I have the separate head and weapon tracking all wrapped up, I'll try to post a video of it tomorrow. It's definitely got a really interesting feeling to it, oddly I feel like the latency is even more pronounced on the sharp weapon movements. Trackers with magnetometers would be ideal though, the yaw drift can get distracting depending on how rapidly you're turning but it's really easy to recenter. I just ordered a top shot elite to mount the tracker on, I read somewhere that my wireless xbox360 receiver will pick it up and all the buttons and axis map the same.

Probably the coolest effect is using the flashlight in dark areas now that it tracks with the gun and not just dead center on the screen. I'd love to get this integrated into one of the Slender mods, basically a wiimote + nunchuck for movement and flashlight control. It'd be hard for nay-sayers to disregard VR after a public pants-crapping

If anyone has 2 hillcrest trackers pm me and I'll send you an installer, the next thing up is supporting other trackers (ideally through FreePIE)

Post by **bobjwatts** » Sat Jan 26, 2013 8:39 pm

Nice one adoral84 - Sounds impressive, can't wait to try it out when FreePIE is up and running,

This will be the first demo with seperate gun and look as far as I know. So freaking cool.

Post by **3dvison** » Sat Jan 26, 2013 10:01 pm

adoral84,
I have a Hillcrest and a YEI space sensor.
Would it be possible to use the Hillcrest as the head tracker and the YEI hand held in mouse mode to control the gun and flashlight ?

Post by **adoral84** » Sun Jan 27, 2013 12:22 am

Here's a quick video of the mod for people who don't have the parts to run it yet. The caves and dark hallway at the end do a pretty good job of showing the separate head and gun tracking.

[youtube-hd]http://www.youtube.com/watch?v=BtbU9Cx7c0o[/youtube-hd]

3dvison wrote:adoral84,
I have a Hillcrest and a YEI space sensor.
Would it be possible to use the Hillcrest as the head tracker and the YEI hand held in mouse mode to control the gun and flashlight ?

Mouse emulation mode won't give you what you're looking for there because I'm using mouse input to allow the user to turn the character in addition to actual head turning so you can stay seated at your desk the same way the gamepad right stick works in all the oculus demos (and in this mod). I do want to support the YEI tracker for real tracking though, it seems to be the best one on the market IMO, unfortunately I don't have one which make it somewhat difficult.

I think the next thing for this mod will be getting a FreePIE general tracker api integrated like Brantlew mentioned so everyone can add their own device plugins.

Post by **PatimPatam** » Sun Jan 27, 2013 3:35 pm

adoral84 wrote:
PatimPatam wrote: I believe [Portal 2 + VR mod + stereo/warp driver + Hydra aiming + Rift + custom 6DOF head-tracking] could be the ultimate VR experience/demo, at least until the first proper games developed with VR in mind start being released (probably along with the Rift consumer version).
I totally agree, but from everything I've read I don't think it'll be possible because AFAIK valve never open sourced the engine beyond the 2007, so you can only create full on recoded mods for games up through that. I'm not even sure if the original portal is moddable in this way, but if it is I'll definitely work on that. I'd also really like to work on L4D2 but it may be the same issue.

Oh that's a shame, would be nice if you could find a way for Portal.. Still separate head and weapon tracking on HL2 looks awesome!! Well done sir!

Post by **bobjwatts** » Sun Jan 27, 2013 4:23 pm

That video looks great adoral84, outstanding work!

Post by **djdevin** » Sun Jan 27, 2013 4:59 pm

adoral84 wrote:Here's an version with the newer much improved tracking code in it, and now there's only one command "vr_calibrate" which is bound to the L key that works exactly like TrackIRs recenter so try to hold your head as level as possible when you run it.

One caveat, I'm having an issue with the freespace api throwing up on exit so when you exit the game it may lock up and need to be killed from the task manager.

https://s3.amazonaws.com/HLVR/halflife-vr-0.7.1.exe

How do you set the axes then? Mine's mounted sideways and the vr_head* variables don't work anymore.

Post by **adoral84** » Sun Jan 27, 2013 5:11 pm

djdevin wrote:How do you set the axes then? Mine's mounted sideways and the vr_head* variables don't work anymore.

Just hold your head level and use vr_calibrate on the console (or press L). If your device is mounted in a way that the mouse emulation works correctly it should reorient in the game correctly. If you have your tracker mounted in a way that doesn't work with mouse emulation (up goes down etc) you can recalibrate the actual tracker's orientation using the Freespace Studio software.

Post by **djdevin** » Sun Jan 27, 2013 5:32 pm

Aha perfect. Thanks!

Post by **adoral84** » Sun Jan 27, 2013 8:21 pm

Just ordered a razer hydra to dabble with, anyone have any thoughts on the ideal way to integrate it w/ something like this? I was thinking about tearing it down and using a tracker for the head and one for the weapon and mounting the base above my head (I've already got all the wires running up to a hook on the ceiling). I'd love to get some 6dof support going on this.

Post by **TheHolyChicken** » Mon Jan 28, 2013 11:52 am

Looks superb. I can't wait to try this out when I get my Rift

Post by **djdevin** » Mon Jan 28, 2013 12:44 pm

Played around last night with it - pretty great!! Probably the best VR demo I have at the moment to show off since you can really play through the game. Aiming is controlled by the head with this demo but it's still fun. The tracking smoothing seems really improved as well.

I'm still having weird roll issues but I'm positive it's from TriDef 3D in SBS mode. With TriDef off it rolls fine, with it on, the screen skews in addition to rolling - really weird. Just wanted to note in case anyone else has the same issue.

Working setup:
- Hillcrest tracker mounted sideways on DIY Rift
- Bluetooth Wiimote/Nunchuk, GlovePIE through PPJoy for movement (forward, back, strafe, action buttons)
- 15 feet of HDMI/USB/headphone cables so I can move around

I'd definitely buy another tracker to use for separate gun movements but FreePIE integration would indeed be awesomer.

Post by **3dvison** » Mon Jan 28, 2013 3:42 pm

djdevin wrote: Working setup:
- Hillcrest tracker mounted sideways on DIY Rift

On my Diy Rift, the Hillcreast will be mounted with the usb connector facing the rear of the HMD. When I test the Hillcrest in mouse mode, Up mves up, down moves down, left moves left, right moves right...

Will mounting the Hillcrest in this position work with this Mod or is it setup for a sideways mounting of the Hillcrest?

Post by **djdevin** » Mon Jan 28, 2013 3:47 pm

3dvison wrote:
djdevin wrote: Working setup:
- Hillcrest tracker mounted sideways on DIY Rift
On my Diy Rift, the Hillcreast will be mounted with the usb connector facing the rear of the HMD. When I test the Hillcrest in mouse mode, Up mves up, down moves down, left moves left, right moves right...

Will mounting the Hillcrest in this position work with this Mod or is it setup for a sideways mounting of the Hillcrest?

Yep that's the default position and it works out of the box.

If you need to reposition it you can recalibrate it with the Hillcrest motion studio.

It asks you to place the sensor down, record that, then angle the sensor up 90 degrees and record that. After that it knows the direction your unit is facing.

Just for anyone who needs it: http://hillcrestlabs.com/products/tools/

Post by **adoral84** » Mon Jan 28, 2013 3:54 pm

3dvison wrote: On my Diy Rift, the Hillcreast will be mounted with the usb connector facing the rear of the HMD. When I test the Hillcrest in mouse mode, Up mves up, down moves down, left moves left, right moves right...

Will mounting the Hillcrest in this position work with this Mod or is it setup for a sideways mounting of the Hillcrest?

Yeah that should work fine, my weapon tracker is mounted the same way.

Post by **3dvison** » Mon Jan 28, 2013 3:59 pm

GREAT..Thanks

Post by **djdevin** » Mon Jan 28, 2013 9:59 pm

Played some more with it but noticed that any movements out of center drift back towards it. If I look up/down/roll it will drift back. Yaw is pretty much spot on. Probably from the smoothing but it's really inducing some dizziness since my vision is moving without my head.

Post by **adoral84** » Mon Jan 28, 2013 10:16 pm

Hmm.. that's really odd. The sensor fusion does take a few seconds of input before it has enough data to properly track, during which the pitch/roll/yaw will slowly drift toward the target angle. That should only take a few seconds and is usually only an issue when the device is sitting perfectly still during the menus and level load (freespace returns timeouts when the device is totally still). Is that consistent or only when you first start it up?

Post by **djdevin** » Tue Jan 29, 2013 12:15 pm

It's consistent, my tracker is mounted sideways (but recalibrated with Motion Studio) so maybe that has something to do with it. I've tried other sensor fusion and it doesn't happen.

At work now but now I'm wondering if it's some setting in HL2 that's making it drift? Autoaim or mouse assist or something stupid I might have checked off.

I'll also be trying Vireio when I get home

Post by **adoral84** » Tue Jan 29, 2013 1:35 pm

That's really odd, I've tried several tracking positions w/ multiple trackers and played waay too much

and never seen any sort of drift on pitch or roll. If you send me your tracker's physical orientation (on the side with the usb coming out the back and the top of the board to the right etc) I'll try to reproduce what you're seeing though.

Mouse inputs only affect yaw and autoaim never really affects the viewport, only the aiming vector and crosshair so I don't think it's either of those things. It may be the sensor fusion averaging across an origin where an axis wraps but I was under the impression that quaternion based calculations didn't suffer from that.

Was this something you saw during your first run?

Post by **djdevin** » Wed Jan 30, 2013 8:15 am

Haha I believe you, it still could be something I did. I don't think I saw it before (before you had the installer) and I had it mounted the same way. Not 100% sure but I can do some more testing.

It's mounted flat on top of the goggles with the USB coming out of the right side, so yaw is the same, but roll and pitch are swapped.