Hydra Absolute Position in FPS

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Hydra Absolute Position in FPS

Post by StreetRat »

I could probably ask this in the sixense forums, but figured since the hydra users here are more tinkerers and probably trying to prepare for the rift, has anyone got fps games working with the hydra in an absolute 1:1 type mode?

I know the hydra has wand mode, but that results in just spinning around due to the way fps games work.
Ive tried the relative wand mode, but never really got that to work.
Mouse mode with ratcheting is ok, but annoying.
Anything i missed?

I have got a way to play fps, (hl2, me2 so far) using the hydra in a 1:1 type way, with the added advantage of no weapon recoil, but was hoping there was something better as my way could be considered cheating.

Basically, i use the windows ReadProcessMemory api to read the current pitch and yaw from the game so i know where the games facing, then get the hydra pitch and yaw from the api and find the difference.
From here theres 2 ways, either set the hydra values directly into the game, which is pushing it just over the cheating line, or the second way, the way im experimenting with, is send a mouse move command that the game expects. This way my app does exactly what the hydra software does, and since im only reading the display values, not changing them, i dont consider it cheating. But it is an advantage as there is no weapon recoil.
The problem with this approach is you need to find the rotation values for each game manually, and in some games, every session.

So i was wondering, those that have a hydra, have you got full 1:1 tracking in fps, and if so how?
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Hydra Absolute Position in FPS

Post by cybereality »

Interesting. I haven't looked into this much, but I have the Hydra and agree the racketing mode leaves something to be desired.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

So, we are talking about hacking in absolute positioning into games? Sounds good what you have done with the memory hacking. Not sure why the recoil effect on the in game gun wouldn't work though?

When I played L4D2, I just use a specific mode on the Hydra that let me turn 360 degrees and output that as mousemove. I did have to ratchet occasionally if I looked up beyond limits or down etc, or just due to eventual drift. Direct memory accessing is the right way to go. Interesting what you are saying about the mousemove thing, thats a nice idea to avoid cheat detection etc. Personally for my gaming needs however i'd probably just push the Hydra values direct to the game, if you dont want to play online against regular players that should be fine.

What tools do you use to find the rotation values? I failed when I was trying to get the rotation values for Commanche 4 (which, if I could get the rotation values, would be a PERFECT VR arcade heli sim), but I failed miserably. I know the values must be there, but I couldn't even find ones that related to numbers on the HUD, so either I was looking for the wrong data type or they are encrypted or something :(
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

I have done the push the values directly to the game, and this has the advantage of allowing roll. It does go weird with games that dont really allow it, rolling around in weird ways, but it is doable. And any 3d type game has the possibility of roll as part of the 3d environment.

Might at some point look into freepie and including that, that way you could use your phone or a head tracker for absolute position and just shoot where you look. Would be interesting, but i probably wont get that far. I have a bad habit of abandoning half finished projects.

The recoil disappearing is a good thing, but happens because any change between the hydra values and the ingame values gets updated per frame, so as soon as the recoil starts, my program starts sending mouse commands to counteract it. Both on the way up and down.

I think left for dead 2 like some of the other Valve games are getting native hydra support, the more the better, but this way should work for everything else.

Some games do encrypt there values, or add a multiplier so the display of 100 becomes 1000 in memory. Others use different data types, in memory 100 integer is different to 100.0 float is different to 100.0 decimal. And for progress bars or health bars, instead of using 100 health, they use a 0 - 1 value so 0.5 is 50 % health. That way the value displayed is just a display value and not really used.

As for finding the values, i use a program called CheatEngine, something like half life 2 took about 30 min, mass effect 2 took about 3 hours looking though the same values (it does weird things with its rotation).
Then theres the problem of constantly changing values and finding the base value that links to all the offsets. This is the hardest part. And each update patch or change in the game means re-looking for them.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

. I have a bad habit of abandoning half finished projects.
I know how that goes...

Thanks for the tips re the multiplier and the using the 0-1 value. I might investigate that. It would probably help if I knew exactly how the engine worked for Comanche so I could figure out if it was likely to be a float or int. I've tried different data types etc. I think I was using CheatEngine as well when I tried. Really I dont need any of the other values apart from the head pan yaw/tilt, which is what mouselook currently does.

I'm surprised there havent been patches like this previously for TrackIR, to enable absolute positioning in old games. Mouse emulation is never going to cut it (well, unless its the special type you are talking about which sounds like a good concept).
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

For Comanche 4 (just purchased the game), the values ive just found are
c4.exe+89F410 or c9f410 - 2 bytes - left / right
c4.exe+89F412 or c9f412 - 2 bytes - up / down

This game uses the values 0 - 65535 for one rotation and some weird byte shifting calculations to determine position
Time to see how well my program holds up so far.

Trackir and absolute position dont work well. Its possible to add it, but you cant look 360 degrees, or anything over 90 without some type of racheting

---
Edit: just added Comanche 4, and using the hydra to control the view works beautifully. Then i added basic TrackIR support. It truly does something to the older games
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Hydra Absolute Position in FPS

Post by brantlew »

@StreetRat: Interesting stuff about the CheatEngine. It sounds like a needle-in-a-haystack problem trying to find the exact memory locations where they are keeping a particular variable. And it seems like it would be hard to do universally. A game that allocated a lot of global memory right up front would be easiest, but it seems like you would be screwed if the program dynamically allocated memory on the heap. Your value would jump all over the place. I wouldn't have even tried to find it (assuming that it would be intractable) but I'm glad there are some tenacious people out there that will dig for it.
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

It is a needle-in-a-haystack type thing, theres a few times ive had 200 addresses to search though, and changing the wrong one causes the game to crash, so ive had to start again.
The dynamic addresses are a real problem, most games nowadays do use that approach, but the game needs to know how to access the address, so its possible to follow it to its base static address then use offsets from there. That way you have the dynamic address regardless of what it is or how often it changes. Dosent always work, but most of the time it does.
Cheat engine has a few tools to help in that regard that make life a bit easier, but the biggest thing is its time consuming.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Hydra Absolute Position in FPS

Post by brantlew »

Yeah I can envision some really nasty scenarios. Like a multi-threaded app where the polling function launched on a separate thread and then used a local stack variable to allocate a struct on the heap. What a mess trying to track that?
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

Well the idea is plausible, and works well both from the hydra and trackir, but its not overly practical.
Ive tried 3 games, and had 3 different ways of representing the angles, and constantly changing addresses dont help.
For a small per program hack, its not too bad but making it work with everything isnt really possible.

And those values for Comanche 4 are wrong. They worked for the first few times i tried, then stopped, maybe mission based.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

For Comanche 4 (just purchased the game), the values ive just found are
c4.exe+89F410 or c9f410 - 2 bytes - left / right
c4.exe+89F412 or c9f412 - 2 bytes - up / down
This game uses the values 0 - 65535 for one rotation and some weird byte shifting calculations to determine position
Time to see how well my program holds up so far.
Trackir and absolute position dont work well. Its possible to add it, but you cant look 360 degrees, or anything over 90 without some type of racheting
WOW, YOU LEGEND STREETRAT!!!!

I am so stoked to hear this. I'll give it a try tonight and see how I get on! How did you manage to figure out the variables? I tried for several hours :( I'm sure I tried to look for 2 byte values as well. One thing that sucked was when you change something and it crashes etc, and having to switch in and out (its a full screen game only) was a PITA. Do you think it could be possible to remotely run CheatEngine, such as logging into the PC the game is running on via RDP, and running CheatEngine from there? That would make life easier when hunting for these positions.

Oh, I just noticed this :(
And those values for Comanche 4 are wrong. They worked for the first few times i tried, then stopped, maybe mission based.
I guess they move around dynamically... might have to do pattern matching or similar to find where they might move to? Also, you mentioned the position, can you also change head position? Currently, its just 2DOF, but if we could move head location as well, it would be cool to have 5-6DOF!

Re TrackIR and absolute positioning, its still quite useful for those that dont have HMD's. For example, playing C4 I need to use the mouse to look around, and its not absolute. If its absolute and paired to trackIR, I can look left, right, etc, no issues. You can't turn past 45 degrees, but you can apply scaling in the TrackIR software to allow you to turn 45 degree and have your virtual head turn 90. I imagine that there will be lots of TrackIR users would would appreciate patchs for older games to give absolute positioning - its really the fact that absolute gives you an accurate 'center' that is most important.

Perhaps we could make a sticky post or similar that would list the variables or the patches/hacks that have been written for common games - I can see that something like that could be a useful resource.
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

Not being able to alt tab is a real pain.
One thing ive done is use short cut keys. Under settings Cheat engine allows you to set some up.
I have
Ctrl+8 = new unknown scan
Ctrl+9 = changed value
Ctrl+0 = unchanged value
Ctrl+- = decreased value
Ctrl++ = increased value

From there set it as a 2 byte search, start the game move the mouse so its just off center and start a new unknown scan
This finds every 2 byte address
Then move the mouse to the left and select increased value do this a few times, then right and decreased value and leave the mouse still and a few has not changed. Make sure you do the has not changed after a few increases and decreases otherwise it really slows the searches down for a while. Do not go over half way though.
Youll probably have 100 - 200 scans left to go though, from there its a matter of trial and error to see which ones work.
I usually put all remaining addresses in the lower part and freeze them 10 at a time and see if the mouse moves, if it does unfreeze them, delete them and try the next 10
If nothing works, try move left decrease move right increase (opposite of before)
C4 rotation is from 0 - 65535 so when you have a few left, move the mouse from left to right across the center and see which values jump from 65000 ish to 1000 ish. There should be about 5 or so.
Unfortunately it dosent allow roll, it gets reset to 0, and probably wont allow movement
Head tracking with the TrackIR worked a lot better than i though, even using mouse mode positioning instead of direct memory update, the angles relative to the cockpit so you can turn your head as you turn.

On top of that, the TrackIR values i was getting were read out of the TrackIR program rather than the SDK. I read the ones used after any scaling so any settings changed worked as a proper TrackIR game. Only problem being, the TrackIR software had to be open in the background, not minimized. Minimizing it stops the program getting the values which means i cant read them.

Ill try tidy up the code and upload it tomorrow, its semi usable if you know what your doing.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Thanks StreetRat, that is some good info. I'll be keen to test out your code if/when you release it. I tried using a program that is supposed to allow you to make full screen apps run in a window to make it easier to trace the values, unfortunately it didn't seem to work for C4 :(. Might try the RDP route next and if that doesn't work I'll use the hotkeys idea of yours. How come you directly got the values from TrackIR instead of going the SDK route?

For anyone else who is interested, Comanche 4 is a really fun (arcady) game that would probably be PERFECT for a sit down VR system (apart from the lack of 6DOF head movement in the cockpit). It has bright, clear graphics, and big objects etc (I believe its designed to run mainly in 1024x768 but its been a while, sorry, can't remember the exact res it maxes out at). I think it would be a perfect match for the Rift. I'll ask CyberReality in his driver thread if he could perhaps look at supporting it!
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

Ive uploaded the code to http://paulsprograms.com/hydra/HydraEmu.rar
its 8 meg.
Both the source and a compiled exe included.
Only works with mouse mode at the moment, you can change it in the source file if you recompile it (need the sixense SDK), but i had some ideas that might make it more robust so thats the next step, and probably add direct update mode too.

If you just want to run the exe copy the database.txt file from the HydraEmu folder to the debug folder with the exe in.
The program will run the last item in the text file so if you want to run something else youll have to add it to the end.
The files not the easiest to understand, but ive added some comments.

The reason im not using the TrackIR SDK is, from what ive heard you have to build a commercial application and sign a NDA to get access to it, or something extreme like that.
Theres the OptiTrack SDK which supports the TrackIR, but ive never got it working with the Track Clip Pro properly, and even then your not actually using the official software so you cant change profiles or rotation speeds easily.
Reading them strait out of the official software gets rid of both problems, both the Vector Clip and Track Clip Pro work and any profile changes work well too.
But its dependent on the correct software version.

C4 works at 1280x1024
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Thanks StreetRat. I've grabbed it now and took a quick look, seems good. I like the way you have stored the addresses in a flat file instead of hard coding them.
Haven't had a chance to fully look at the code yet however. I tried running the EXE but it didn't seem to find C4 running - should I run the exe first, or C4 first, and did you design it using the Steam or standalone version?

I do have Sixense SDK installed, so will have a closer look/compile/trace tonight hopefully. Good info re TrackIR, that does sound like the best solution if the SDK is that much work to use!
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

The program is designed to wait for the game to load, so it will sit there waiting until you exit the program or the game loads.

Normally since the values change, youll have the game open anyway, would have found the values, updated the text file then run the program.
One thing i havent got in the program yet is a refresh. So itll only read the values from the textfile once on load and thats it. so if you have the program open and change values in the textfile, youll have to close the program down and reopen it.
As for the game version, my program looks for the name, so that should be the same between the regular version and the steam version. From there, since you have to find the memory locations yourself, it dosent matter which is running.

if you can find a base address (like with mass effect 2) then that will only work with that specific version, which happens to be the latest steam version in the file. So any updates or the non steam version (if there is one) will be different.
I need to take that into account somewhere in the textfile as well. Although i guess a basic comment should suffice.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Thanks StreetRat, that explains a few things. Been busy trying to resistor mod my Hydra, hope to have a chance tonight to have a play.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Hey Streetrat, I've managed to have a play with CheatEngine today and sussed out a few things. I had real issues switching back and forth (it seemed my mouse was drifting while i was switching out/switched out, which made it hard to trust what i was doing) and I always ended up with 1000's of entries to go through. However, I thought a bit more about my RDP idea and did some googling and found some handy stuff.

Basically, you need to run a patched termsrv.dll, to allow rdp connections to run in the background, not take over the screen, possibly disable UAC or use another workaround that is provided, and turn on 'fast user switching' on the client machine. You can then RDP in from your main machine as a different admin user (so you may have to create a new user on the client), and then 'runas' CheatEngine as the main client user (the one you are going to run the game under). This means both CheatEngine and the game will run under the same credentials. Its pretty easy to get set up really, now I just RDP to my client, run cheat engine, and run the game on the client. Once I had this set up, I was able to very quickly get the yaw and pitch variables, just doing 'decreased' and moving the mouse a bit to the right repeatedly, I managed to get it down to about 12 variables, so I then just looked for the ones jumping from 65000>1000 etc at the center and found them. They seem to be right next to the variable(s?) used to store the ammo etc. Way nicer than switching in and out all the time!

http://digiex.net/guides-reviews/guides ... v-dll.html
is the main guide, and
http://forum.cheatengine.org/viewtopic. ... 6&start=15
has some info about setting up the CheatEngine shortcut.

Now I have working variables and an easy method to find them quickly, I'll take another look at your code, get it going, and then see if I can figure out a way to pattern match the memory location or something, so we can patch the variables easily even if they change address.
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

I had the same problem when i alt tabbed too, thats why i switched to the shortcut keys, i seem to use them more now than alt tabbing, even in games that allow it. Lot less messing around too.

I have been playing around with a new version made in VB.net with a graphical interface, but i got sidetracked. I tried in vc++ but have no idea how to create/use forms in vc++.
Started looking for the rotation values in Crysis, but got carried away and played all 3 instead.
The values i did find are odd and i havent figured them out completely, or how to get them working within the current code.

I also found the base values for C4 and played a few levels with a few restarts and what i have seem to work well.
the new entry should look something like:

gamename:c4.exe:Comanche, V1.0.1.20
angleType:y2p2r2
angleInvert:0:0:0
memType:static
address:0x0058FB40
offsetCount:1
offsetValues:0x190

Nice to know there next to the ammo, i never even looked for that.

ive been playing around with CE and memory locations for years, but only just found out the relevance of the 0 - 65535 and how it works for games like C4. CE reports the values as an integer 0 - 65535, but the value is actually an unsigned integer from -32767 to 32767. Makes sense when you think about it that way
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

I can totally see how the keys could help a lot now.
How do the static addresses in your format work with the offset values/count etc? That address doesn't seem to work for me (did a scan around there as well). I have a method that gives me the 0c00cxxxxx yaw/pitch address easily, but its a little kludgy so I want to find something better... so if you have a non moving address that would be sweet.
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

If those values arnt working then you can try find them yourself.
With the value you have found in cheat engine, right click on the first value and go to pointer scan

Let it run with the default items, dont put the values too high, save the scan details. Play the next few missions or reload, find the new values then go back to the pointer scan page and go to the menu Pointer scanner and rescan memory, put in the new address. This will find all the old locations that match the new address.
Find one that dosent change as you change map/reload and use that. Double click on it will add it to the list.
From there add the address + the offsets to the text file
stinkvis
One Eyed Hopeful
Posts: 28
Joined: Sun Oct 21, 2012 1:27 pm

Re: Hydra Absolute Position in FPS

Post by stinkvis »

StreetRat wrote: Mouse mode with ratcheting is ok, but annoying.
Anything i missed?
The ratcheting mode is terribly unintuitive and I can't understand why they decided to use this as the default mode. Hybrid mode is a lot easier to use and works really well with most games. The only downside is that it's kind of awkward to use in menus as it's really easy to let the cursor drift off to the sides. Other than that I highly recommend it over ratcheting.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Thanks again StreetRat, those were some more good CheatEngine tips.

I have a good static now that seems to work for me repeatedly (0x49FFFE). I've nearly finished a small util that gets the position from TrackIR using your TrackIR routines, and pushes them to the roll/yaw location. Works fine, but I need to scale the responses to those possible for the Comanche view. I'll up the code once I've finished it (it will be a util dedicated to C4 only).
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Well, been a while! Sort of got distracted by life (and quite a few video games, discovered the Helix mod and have been playing some new and old games in great 3D). Anyway, with the holidays I scraped a bit of time and tidied up my code, so I've upped my source etc here (and an exe for those who dont want the source). It works well for me in enabling TrackIR in V1.0.1.20 of Comanche.

I've started on a new one that will do FarCry3 as well (when you are in the vehicles)...
You do not have the required permissions to view the files attached to this post.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Hydra Absolute Position in FPS

Post by brantlew »

This seems like the perfect thread to post this.

[youtube]http://www.youtube.com/watch?v=WYJrN7lNNcc[/youtube]

I am using a semi-generalized method for this. I wrote a custom HL2 mod to support true 6DOF tracking without memory hacks. I also have a generic API for transmitting orientation values inter-process. The HL2 mod and the FreePIE process both link to this API so that FreePIE can send orientation data to the HL2 mod. The video shows the Hydra being used to generate the motion, but you can easily swap that out for FreeTrack, etc. Or you can combine methods like using the Sparkfun IMU for orientation and the Hydra for position.

@WiredEarp: You should consider standardizing your interface into these games that you are memory hacking. That way you could allow a variety of tracking devices to control it - maybe even from something like FreePIE. My interface uses named pipes so it's trivial to hook into and it's super efficient, but I have also been thinking about doing a VRPN variant. That would standardize it even more (at the small expense of adding the overhead of UDP communication).
User avatar
PatimPatam
Binocular Vision CONFIRMED!
Posts: 214
Joined: Thu Jun 28, 2012 1:31 pm
Location: Barcelona

Re: Hydra Absolute Position in FPS

Post by PatimPatam »

Awesome work brantlew!

This could be really useful for a tracking project i'm working on (i'll post about it this weekend, hopefully).
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

That looks great brantlew! Perhaps HL2 could end up being a better/easier testbed for VR gaming than Unity, since you already have game assets in there. In the 'up/down' movement you show, it seems infinite, is this correct, or is it simply performing a 'crouch' on the down part? If its infinite, that will add heaps to the immersion!

As you suggested, I've already settled on a standardized input for different games, pretty simple really, its just using floating point euler angles to represent pitch/yaw. The gamehack class itself then translates these to the correct range for the game (after clamping etc to avoid invalid ranges). Also I eventually plan to have a drop down list of different 'input' systems, so users can swap from TrackIR to Hydra etc. To support multiple games, I thought it would be easiest if it does this automatically - so my intention it to make it simple list all the executables running, and hack the first supported executable automatically.

Since its just a C++ class, I also thought it could be a easy fit with freePIE. That way, users could just send commands to the gamehack using FreePIE, and it will automatically translate it to whatever game is running. Since its using euler angles externally, all that is required for other sensors would be to translate the pitch/yaw into euler angles and send them to the gamehack.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Hydra Absolute Position in FPS

Post by brantlew »

@WiredEarp: Every since I saw https://developer.valvesoftware.com/wiki/Head_Tracking I figured that HL2 would make a perfect VR test bed. As you suggested the motion is true 6 DOF, not just crouch/jump commands. I'm not even doing bounds tests so you can swing right below the floor if the drop the controller far enough. I'm handling the Euler angles from FreePIE just as you mentioned, so you are welcome to the code if you want it.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

That is awesome. I was going to use Unity for my testbed, I might change that now! Really, The infinite movement is a super important component of VR. I guess if you are modding it, you can give yourself independent head motion as well?
If thats the case, I'll probably use HL2 as well for my testing. I want to create a simple testbed similar to the old Dactyl Nightmare, with lots of big, geometric obstacles etc. I think these will be better on the Rift that something with detailed textures...

Yep, I will have to get into FreePIE and do some stuff. I was going to write a Hydra adaptor for it at one point (if this hasn't been done by someone already, I can't imagine it would be hard once you have the SDK running, in naive form its just initializing, polling, closing). I'm going to have to look at the TrackIR path that CV & CR have been following as well, as I dont want to depend on StreetRats TrackIR hacks which are really just a stopgap thing.

Are you planning on trying to miniaturise your Hydra, or intending to use it as is as a gun or something?
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

Thats cool brantlew.
Love to get a hands on with the mod.

WiredEarp: My trackir hack is just that since i believe theres a bug within the optitrack drivers when using the track clip pro.

I have some basic working trackir code for use in python and javascript here:
python: http://code.google.com/p/py-trackir/
javascript: http://code.google.com/p/js-trackir/

Again, projects i started playing with, and then left, these are mostly finished though. They work great with the hat clip.
If you guys have anything that works with the track clip pro, id be rather interested in the code to get it to work.

Anyway, one thing i thought of the other day was to try use one of the directx matrices as a reference point. Since there used in almost all 3d games, it might be a workable location for all 3d directx games. Unfortunately, it would probably only allow mouse updates rather than direct updates, but might still work as a base item, with more advanced features in the more specific apps.
User avatar
bobjwatts
Cross Eyed!
Posts: 125
Joined: Mon Oct 01, 2012 1:46 am
Location: Melbourne, Australia

Re: Hydra Absolute Position in FPS

Post by bobjwatts »

brantlew wrote:This seems like the perfect thread to post this.


I am using a semi-generalized method for this. I wrote a custom HL2 mod to support true 6DOF tracking without memory hacks. I also have a generic API for transmitting orientation values inter-process. The HL2 mod and the FreePIE process both link to this API so that FreePIE can send orientation data to the HL2 mod. The video shows the Hydra being used to generate the motion, but you can easily swap that out for FreeTrack, etc. Or you can combine methods like using the Sparkfun IMU for orientation and the Hydra for position.

@WiredEarp: You should consider standardizing your interface into these games that you are memory hacking. That way you could allow a variety of tracking devices to control it - maybe even from something like FreePIE. My interface uses named pipes so it's trivial to hook into and it's super efficient, but I have also been thinking about doing a VRPN variant. That would standardize it even more (at the small expense of adding the overhead of UDP communication).

Cool brantlew! Would this mod work with iphone to freePie? If so would you be open to posting the mod?
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Hydra Absolute Position in FPS

Post by brantlew »

@WiredEarp: Yeah, I've been needing a VR testbed for a while. I considered using Unity as well, but having assets and a real game experience offers a more compelling demonstration. Eventually I want to add a custom motion interface so I can pipe in actual player coordinates instead of just putting in WASD commands. So you could map true player motion and have a 1:1 VR experience with head positioning and player walking. Also, native Rift stereo warping would make a nice addition to the mod.

The Hydra is just for testing right now. I beat you to it and just finished FreePIE integration last week. http://www.mtbs3d.com/phpBB/viewtopic.php?f=139&t=16051 But I don't intend to use the Hydra for anything. I'm just clearing the way for testing other tracking methods - in particular some camera based optical flow methods.

@bobjwatts: Yeah, you can control the mod with an iPhone. Anything that FreePIE currently supports.

This is my first game mod so it's a bit of a mess. I should clean it up a little before I release it, but I'll give it out in a week or two.
User avatar
Chriky
Binocular Vision CONFIRMED!
Posts: 228
Joined: Fri Jan 27, 2012 11:24 am

Re: Hydra Absolute Position in FPS

Post by Chriky »

Can't believe I missed this - great work!
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

@ StreetRat: That is an interesting idea re the using DirectX matrices. If that was possible, combined with your simulated mouse movement idea, you might have a way to make a generic mod.
However, I dont think it will work. You will be changing the view, but not the _games_ view. IE, you might be able to make it look around (isn't this what CyberReality is doing with his driver?), but the game wont know that you were really wanting to do the equivalent of 'aim up', so it won't work for aiming guns in FPS's etc (not that aiming with the head is a good thing). For example, you may pan the view, but the crosshair will probably stay pointing in the same direction.

I rewrote my code for my mod the other day to be more object oriented for future expansion (wanted to add FarCry 3 and decided I may as well do things properly to save maintaining it later on). If you want the Far Cry 3 memory location for pan/view, let me know, used your tricks to find a static pointer that works.

I think CyberVillain and CyberRascal have been working on TrackIR for FreePIE. Not sure if they could provide any light on the trackClip thing. If you could get a copy of the Freetrack 2.1 source (think this was the last version with TrackIR), I believe that supported the trackclip, so might be of use/interest. However, I think these have all been purged from the internet, probably due to legal threats from NP (this is just a guess). I used the TrackClip for ages, but I've given up on it, since I've discovered it just doesn't seem to work as well at extreme angles as the hat clip.
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

I think your right. After talking to CyberReality about his drivers, and a few days of playing around, it seems not all games use the standard directx commands so theres not way to make a global mod. Cyber did give me some pointers that i have yet to follow up.

Whats worse is some games might not be hackable. Ive spent many hours within Crysis and still come up with nothing.
Ive actually started looking around at aimbots to see how they do it for some ideas.

I also downloaded the latest FreePie exe, still need to get around to testing it out, too much to do, and jumping from one project to another means nothing gets done.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Hydra Absolute Position in FPS

Post by cybereality »

Yeah, I had a lot of problems with Crysis. Never could get it working with my driver.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

Which Crysis are you experimenting with StreetRat? Seems wierd not to be able to find them, perhaps they are encrypted? Do you know what type of variables this info is stored in, ie, floats or integers or whatever? I'd assume floats.

Yep, I think eventually I'll make mine into a plugin for FreePIE, that way I can leverage all of brantlew CR and CV's work at the input side, and FreePIE gets new outputs that can be used to control games...
StreetRat
Two Eyed Hopeful
Posts: 65
Joined: Sun Oct 24, 2010 11:11 pm

Re: Hydra Absolute Position in FPS

Post by StreetRat »

Ive tried both Crysis and Crysis 2.
From what ive learned, the game seems to respond directly to the mouse input, so i can find and freeze the directInput value that moves the screen, but that only allows for relative movement based on the input.
Ive yet to find a variable that tells me which way is north or south, actually i think i did once, but upon freezing it the whole thing crashed and i havent found something since.
Havent give up yet
Ive tried looking for floats, as well as 1/2/4 bytes, and nothing stands out too much.
Problem is if it is stored in a matrix, finding the values between -1 and 1 (usually 0.948271 etc) can get annoying when you have 200 or so.
Game also crashes when adding a debugger, so cant go backwards though the code to figure it out either.

Fortunately Crysis 2 (cant remember about Crysis 1) comes with a split screen option in the 3d settings, but it wont have the distortion. Hopefully Crysis 3 will be a little better (although doubtful)
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: Hydra Absolute Position in FPS

Post by MSat »

For games that dynamically allocate variable addresses every new process instance, might it be possible to eliminate the randomness by running those processes in a sort of "strict" virtual machine - one that persistently maintains all initialization variables and OS API call returns (don't know if that's the correct terminology) upon game startup?
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Hydra Absolute Position in FPS

Post by WiredEarp »

MSat, do you mean running in a VM and just saving and restoring that original game state from then on?

Not sure, but I think running in a VM like that would be too slow, and it would also require other people to have the same saved state file...?
Post Reply

Return to “General VR/AR Discussion”