[Solved] Vireio Shared Memory Tracker + Android

[moukrea]

Hi guys, I'm kinda stuck, I've tried many solutions/scripts browsing your forums, but nothing got to work for me!

So here's my issue:
I want to play Vireio Perception (+ VR Boost) enhanced games on my Colocross Headset + Galaxy S5 (I'm currently trying to build my DIY Rift-like with 2K lcd and DIY sensors) to be able to get Pitch, Yaw and Roll working (and possibly positional tracking using Freetrack + Camera + IR Leds, but that'll be for later)

So far I've tried the Freetrack option of Vireio, Opentrack receiving the sensors data from Trinus VR (my phone) and output Freetrack. Vireio doesn't sees it (too bad, it works with any TrackIR/Freetrack enhanced games!)

So I still have the Shared Memory Tracker option, which uses FreePie.

I'm totally new to scripting this kind of things, so I wonder... How can FreePie get my phones sensors data (I tried Freepie Redirect & FreepieIO in Trinus, with many scripts found here and there... not working) and output it as Shared Memory Tracker to make Vireio recognize the headtracking.

I already have the vireioSMT.dll in my FreePie plugin directory

Any leads guys? I'm totally stuck with this, and honestly after playing TrackIR/Freetrack enhanced games, I don't want to play games through Mous Emulation anymore, it's just so wrong!

Thanks!

Edit: This issue is solved, with this code, I can play Left4Dead 1 & 2 on 3 DOF using my Phone (Android), Trinus VR, OpenTrack, FreePie and Vireio Perception:

1. Trinus VR (Sensor Output: OpenTrack Redirect), Sensors on defaut port (5555).

2. OpenTrack on UDP sender (Port 5556), Protocol freetrack 2.0 Enhanced.

3. FreePie with this script:

Code: Select all

def update():
	# We define here names of TrackIR imputs to make the script easier
	global Yaw #trackIR.yaw will be named Yaw, we have to precise it in the next lines (Line 11 to 16)
	global Pitch
	global Roll
	global Xaxis
	global Yaxis
	global Zaxis
	
	#For exemple "Yaw = trackIR.yaw", mean the imput "trackIR.yaw" will be named Yaw in our script
	Yaw = -trackIR.yaw / 55.0
	Pitch = trackIR.pitch / 55.0
	Roll = trackIR.roll / 120.0
	Xaxis = trackIR.x / 5.0 #Positional Tracking
	Yaxis = trackIR.y / 5.0 #Positional Tracking
	Zaxis = trackIR.z / 5.0 #Positional Tracking
	# "/ 50.0" mean the imput received from trackIR will be 50 times smaller when sent to Vireio's Shared Memory Tracker, change this values according to your experience in-game (I use this with Left 4 Dead)
	
	#Assigning TrackIR imputs to Vireio's Shared Memory Tracker imputs
	vireioSMT.yaw  = Yaw - centerYaw
	vireioSMT.pitch = Pitch - centerPitch
	vireioSMT.roll = Roll - centerRoll
	vireioSMT.x = Xaxis - centerXaxis
	vireioSMT.y = Yaxis - centerYaxis
	vireioSMT.z = Zaxis - centerZaxis
	
	#Setting all vallues to 0 when starting, make sure your tracking device is in the right direction before starting the script
if	starting:
    centerYaw = 0 #We define the center of Yaw to 0, same for all the "center" lines
    centerPitch = 0
    centerRoll = 0
    centerXaxis = 0
    centerYaxis = 0
    centerZaxis = 0 
    Yaw = 0
    Pitch = 0
    Roll = 0
    Xaxis = 0
    Yaxis = 0
    Zaxis = 0
    trackIR.update += update #Thanks to this line, FreePie will start updating Vireio's Shared Memory Tracker (receiving imput from TrackIR)

	# Assigning an imput to re-center in case of drifting, here "K", can be changed up to your tastes
if	keyboard.getPressed(Key.K):
	centerYaw = Yaw #Freepie will set to "Yaw" the same value as "centerYaw" (0) meaning that Yaw will be back to its initial position, same for other lines bellow
	centerPitch = Pitch
	centerRoll = Roll
	centerXaxis = Xaxis
	centerYaxis = Yaxis
	centerZaxis = Zaxis

#The lines bellow are only for test purpose within FreePie, delete them once you're done to save some CPU power
diagnostics.watch(trackIR.yaw)
diagnostics.watch(trackIR.pitch)
diagnostics.watch(trackIR.roll)
diagnostics.watch(trackIR.x)
diagnostics.watch(trackIR.y)
diagnostics.watch(trackIR.z)
diagnostics.watch(vireioSMT.yaw)
diagnostics.watch(vireioSMT.pitch)
diagnostics.watch(vireioSMT.roll)
diagnostics.watch(vireioSMT.x)
diagnostics.watch(vireioSMT.y)
diagnostics.watch(vireioSMT.z)

Note: The X, Y & Z related lines are for positional tracking, hasn't been tested yet as I don't own the cameras to do so, it should work with some games enhanced by VR Boost.

4. Vireio Perception in Side-by-Side, Shared Memory Tracker

5. Launched Trinus VR Server (I don't use the FreePie APK, due to difficulties to get the right network configuration playing with the WiFi Hotspot of Trinus), Started OpenTrack, Launched FreePie script and... The Game (Left 4 Dead)

[CyberVillain]

You can use FreePIE directly and the android plugin, then use the SMT plugin to forward that to Vireio

[moukrea]

Thanks, indeed I don't know how to script this at all, any ideas? I guess with this method I have to use the Freepie APK, but then, how to set FreePie itself?

[CyberVillain]

Isnt Opentrack also using the FreePIE apk? If not my bad.

[moukrea]

Well you probably can, but as I use Trinus VR to stream the game, I get as well the sensors data from it, that I redirect to OpenTrack within Trinus VR

[konstantin_lozev]

I am not sure of the way headtracking is implemented in Vireio. Does it allow you to rotate directly the in-game stereo camera rig instead of having to pass through mouse emulation?
If you want to have mouse emulation or FreeTrack emulation directly into your game, you will have to disable the sensor tracking in TrinusVR, install the FreePIE apk on your phone, calibrate the gyro and possibly the magnetometer on your phone, load one of the mouse/FreeTrack emulating scripts and start both PC and phone FreePIE apps.
Again, what is the advantage of routing the input through Vireio?

[moukrea]

The advantage is that Vireio (+ VR Boost) allows tracking of Yaw, Pitch and Roll in a few games rather than just just Yaw and Pitch provided by Mouse emulation, which ends up with a far better immersion, it seems nothing but O realized after playing TrackIR Enhanced games in VR (I tested only ArmA 2, DayZ and DiRT: Rally) it makes a huge difference! I didn't realize I tilt my head that much in real life/trackIR before I switched back to Mouse emulation...

I think I found a way though, using Vireio Perception 2.1.6 and the FreePie APK, just gotta try it out and I'll get back to you guys

[CyberVillain]

Games like Arma are made for headtracking, games that are not will not work if you ask me. You do not want to aim the gun with your head?

[moukrea]

I know I will still have a gunface, the only reason I want this to work is that Vireio allows to rotate on Roll (on top of Pitch and Yaw), which is much better than classic mouse emulation to me...

Like you can see around 0:30/0:45 in this video:

https://www.youtube.com/watch?v=C67VsZVF7x8

You don't get that with mouse emulation ^^

So I tried this code

Code: Select all

def update():
    global yaw
    yaw = android[0].yaw
    global pitch
    pitch = -android[0].roll
    global roll
    roll = android[0].pitch

if starting:
    centerYaw = 0
    centerPitch = 0
    centerRoll = 0
   
    yaw = 0
    pitch = 0
    roll = 0
    android[0].update += update

vireioSMT.yaw  = yaw - centerYaw
vireioSMT.pitch = pitch - centerPitch
vireioSMT.roll = roll - centerRoll

if keyboard.getKeyDown(Key.LeftControl) and keyboard.getPressed(Key.C):
    centerYaw = yaw
    centerPitch = pitch
    centerRoll = roll
    
diagnostics.watch(android[0].yaw)
diagnostics.watch(android[0].pitch)
diagnostics.watch(android[0].roll)

The last three lines are just for test purpose...

It works! Vireio Perception doesn't tell me "HMD NOT INISIALIZED" or whatever, it just starts tracking right away. BUT! Oh god the values are getting crazy, in game it's going nuts, rotating constantly in every directions Instant motion sickness

I wonder how can we filter this, any ideas ?

[moukrea]

I dunno if anyone is interested but I got it to work, still have a bit of tweaking to do in FreePie but it's decent enough for test purpose...

I ditched the Freepie APK, I want to use Trinus VR wirelessly, and to do it without massive latency, the best is to create an hotspot wifi within Trinus, it mess around with network and became really hard to get FreePie to get the data the APK is supposed to send. So, here's what I did:

1. Trinus VR (Sensor Output: OpenTrack Redirect), Sensors on defaut port (5555).

2. OpenTrack on UDP sender (Port 5556), Protocol freetrack 2.0 Enhanced.

3. FreePie with this script:

Code: Select all

def update():
	vireioSMT.yaw = -trackIR.yaw / 50.0
	vireioSMT.pitch = trackIR.pitch / 50.0
	vireioSMT.roll = trackIR.roll / 50.0
	vireioSMT.x = trackIR.x / 5.0
	vireioSMT.y = trackIR.y / 5.0
	vireioSMT.z = trackIR.z / 5.0
	
if starting:
    trackIR.update += update

4. Vireio Perception in Side-by-Side, Shared Memory Tracker

5. Launched Trinus VR Server, Started OpenTrack, Launched FreePie script and... The Game (Left 4 Dead)

It works great, indead the camera tend to jumps sometimes (no ideas why), and when rotating 45° in real life, it rotates more like 50-60° in the game, with a little bit of tweaking that should be corrected

And so, yes, it's a much better experience than Mouse Emulation, the head tracking is now on 3 DOF, like in real life! Still the gun face (not real life) and positional tracking to get, just a full headtracking is already nice

[konstantin_lozev]

Games like Arma are made for headtracking, games that are not will not work if you ask me. You do not want to aim the gun with your head?

CV, I have a solution for that with the wiimote dynamic swapping script. I have hit a wall with the switching off of the nunchuck when the motion+ is enabled, but I am adding now support for a dance pad instead, so that you won't need a numchuck.

[konstantin_lozev]

The advantage is that Vireio (+ VR Boost) allows tracking of Yaw, Pitch and Roll in a few games rather than just just Yaw and Pitch provided by Mouse emulation, which ends up with a far better immersion, it seems nothing but O realized after playing TrackIR Enhanced games in VR (I tested only ArmA 2, DayZ and DiRT: Rally) it makes a huge difference! I didn't realize I tilt my head that much in real life/trackIR before I switched back to Mouse emulation...

I think I found a way though, using Vireio Perception 2.1.6 and the FreePie APK, just gotta try it out and I'll get back to you guys

Sorry to bump again, so you are saying that some games that have no FreeTrack native support, with Vireio + VR Boost (which games?) would rotate the camera directly instead of needing to emulate a mouse? I was not aware. That would be very useful in games that don't support raw mouse input and would solve the issue of different mouse sensitivity in ADS mode. Could you please point me to more details?

[moukrea]

Well, you actually still have what's called a "gunface" but with mouse emulation, the headtracking work only on 2 DOF (Yaw & Pitch), Vireio uses shaders (? not sure if it's the shader doing that) allowing the headtracking on 3 DOF (adding Roll to it)!

Possibly on 6 DOF if you got something to get positional tracking (Yaw, Pitch, Roll, X, Y, Z), indeed the gun still stuck on your face it just make your mouvements feel much more natural than the robotic X + Y axis from mouse emulation.

Yes it doesn't requires Mouse Emulation, you can force Mouse Emulation within the in-game menu of Vireio but instead it uses its own camera system, I don't know how it work exactly, just noticed that it mess around with the mouse behavious if you don't play with a gamepad... I guess if you want to play with Keyboard+Mouse, you gotta use Mouse Emulation for Yaw & Pitch, letting only Roll to Vireio

While it doesn't trully turn the game into full VR experience that could be brought buy TrackIR or real VR games... It's still a huge step up (specially to play our old favorites games in something close to a real VR gameplay)

It works on Left 4 Dead 1 & 2, didn't test it yet with other games but I bet it works with Skyrim and many of the other games listed in the Vireio manual: https://docs.google.com/document/d/1TYp ... 6rgQw/edit#

Like you can see in this short video (with an old release of Vireio):

https://www.youtube.com/watch?v=9IGpGR-TGt0

[konstantin_lozev]

Well, you actually still have what's called a "gunface" but with mouse emulation, the headtracking work only on 2 DOF (Yaw & Pitch), Vireio uses shaders (? not sure if it's the shader doing that) allowing the headtracking on 3 DOF (adding Roll to it)!

Possibly on 6 DOF if you got something to get positional tracking (Yaw, Pitch, Roll, X, Y, Z), indeed the gun still stuck on your face it just make your mouvements feel much more natural than the robotic X + Y axis from mouse emulation.

Yes it doesn't requires Mouse Emulation, you can force Mouse Emulation within the in-game menu of Vireio but instead it uses its own camera system, I don't know how it work exactly, just noticed that it mess around with the mouse behavious if you don't play with a gamepad... I guess if you want to play with Keyboard+Mouse, you gotta use Mouse Emulation for Yaw & Pitch, letting only Roll to Vireio

While it doesn't trully turn the game into full VR experience that could be brought buy TrackIR or real VR games... It's still a huge step up (specially to play our old favorites games in something close to a real VR gameplay)

It works on Left 4 Dead 1 & 2, didn't test it yet with other games but I bet it works with Skyrim and many of the other games listed in the Vireio manual: https://docs.google.com/document/d/1TYp ... 6rgQw/edit#

Like you can see in this short video (with an old release of Vireio):

Can you not use OpenTrack with the printed Aruco marker for positional tracking? https://github.com/opentrack/opentrack/ ... co-tracker

[moukrea]

Mhhh I don't know that, at the time I don't have webcam, nor printer to try it out but I guess it might work! Though I think using Trinus VR, there's a similar solution embedded within the soft, I think OpenTrack can get it and convert it into a TrackIR output!

Talking about that... I wanna get a PS3 Eye, remove its IR filter, put some IR leds around my headset... not only in front, but all around to get 360° tracking in Freetrack. So far this is possible, but I'd like to mix sensors output with Freetrack (and IR leds) to get a more accurate head tracking+positional tracking.... That's the way the Rift is working (internal sensors+camera), I don't know if there's a way of doing it in a DIY solution... Maybe Freepie can mix the output?

[konstantin_lozev]

After reading your post, I gave Vireio and the SMT dll for FreePIE a try and came up with that script. It works quite well, actually in Borderlands. I added just now the roll part, so have not tested it.
I think you should set TrinusVR for no sensor output and rely fully on the FreePIE to VireioSMT output

Code: Select all

import time
def update(): #assigned to the Android update
	global t_prev_A
	t_delta_A=time.time()-t_prev_A #deltatime for the android update
	t_prev_A = time.time()
	preX = filters.deadband(android[0].raw.gx*t_delta_A,deadbandX) #android polling with conversion to degrees and low pass filter
	global x
	x +=preX*multiply #results of the android polling adjusted with senstitivity
	preY = android[0].googleRoll #android polling in radians - same as VireioSMT 
	y = -preY*multiply-y_offset #results of the android polling adjusted with senstitivity
	preZ = android[0].googlePitch #android polling  - same as VireioSMT 
	z = -preZ*multiply-y_offset #results of the android polling adjusted with senstitivity
	diagnostics.watch(x)
	diagnostics.watch(y)
	diagnostics.watch(y)
	vireioSMT.yaw  = x
	vireioSMT.pitch = y
	vireioSMT.roll = z
#	if keyboard.getPressed(Key.RightArrow):
#		vireioSMT.x += 10
if starting:
	android[0].update += update
	multiply = 1
	t_delta_A=time.time()
	t_prev_A=time.time()
	y_offset=math.pi*0.5
	deadbandX = 0
	x=0

[LewisFletcher123]

def update():
# We define here names of TrackIR imputs to make the script easier
global Yaw #trackIR.yaw will be named Yaw, we have to precise it in the next lines (Line 11 to 16)
global Pitch
global Roll
global Xaxis
global Yaxis
global Zaxis

#For exemple "Yaw = trackIR.yaw", mean the imput "trackIR.yaw" will be named Yaw in our script
Yaw = -trackIR.yaw / 55.0
Pitch = trackIR.pitch / 55.0
Roll = trackIR.roll / 120.0
Xaxis = trackIR.x / 5.0 #Positional Tracking
Yaxis = trackIR.y / 5.0 #Positional Tracking
Zaxis = trackIR.z / 5.0 #Positional Tracking
# "/ 50.0" mean the imput received from trackIR will be 50 times smaller when sent to Vireio's Shared Memory Tracker, change this values according to your experience in-game (I use this with Left 4 Dead)

#Assigning TrackIR imputs to Vireio's Shared Memory Tracker imputs
vireioSMT.yaw = Yaw - centerYaw
vireioSMT.pitch = Pitch - centerPitch
vireioSMT.roll = Roll - centerRoll
vireioSMT.x = Xaxis - centerXaxis
vireioSMT.y = Yaxis - centerYaxis
vireioSMT.z = Zaxis - centerZaxis

#Setting all vallues to 0 when starting, make sure your tracking device is in the right direction before starting the script
if starting:
centerYaw = 0 #We define the center of Yaw to 0, same for all the "center" lines
centerPitch = 0
centerRoll = 0
centerXaxis = 0
centerYaxis = 0
centerZaxis = 0
Yaw = 0
Pitch = 0
Roll = 0
Xaxis = 0
Yaxis = 0
Zaxis = 0
trackIR.update += update #Thanks to this line, FreePie will start updating Vireio's Shared Memory Tracker (receiving imput from TrackIR)

# Assigning an imput to re-center in case of drifting, here "K", can be changed up to your tastes
if keyboard.getPressed(Key.K):
centerYaw = Yaw #Freepie will set to "Yaw" the same value as "centerYaw" (0) meaning that Yaw will be back to its initial position, same for other lines bellow

#The lines bellow are only for test purpose within FreePie, delete them once you're done to save some CPU power
diagnostics.watch(trackIR.yaw)
diagnostics.watch(trackIR.pitch)
diagnostics.watch(trackIR.roll)
diagnostics.watch(trackIR.x)
diagnostics.watch(trackIR.y)
diagnostics.watch(trackIR.z)
diagnostics.watch(vireioSMT.yaw)
diagnostics.watch(vireioSMT.pitch)
diagnostics.watch(vireioSMT.roll)
diagnostics.watch(vireioSMT.x)
diagnostics.watch(vireioSMT.y)
diagnostics.watch(vireioSMT.z)
This script allows the user to play games with Vireio Perception and achieve 3 degrees of freedom (DOF)