Eye Tracking - Tobii EyeX/SteelSeries Sentry

[eyeBo7]

I'm using my eye tracker to control the freelook camera in Elite Dangerous. Actually works better than I thought it would:

https://www.youtube.com/watch?v=DstqBQp0A9Q

What do you think?

The implementation is simple. I'm mapping eye tracking screen coordinates to a vJoy 2-axes joystick and then bind it to the camera in the game.
Here's a link to the SDK: http://developer.tobii.com/

[cybereality]

Wow!

[eyeBo7]

Take On Helicopters

https://www.youtube.com/watch?v=xFhSPl-e0tI

[CyberVillain]

Nice! How about a FreePIE plugin?

[eyeBo7]

https://www.youtube.com/watch?v=RqayFa_nSXs

[BFQ]

eyeBo7, could please tell me (a code noob) how to make my Eyetracker work like "take on helicopters"?

As I understand I need FreePIE (tobii version of FreePie?) with some sort of a Tobii script. Where can I find this script? Do I need to install something else then FreePie?

Thank you in advance

[eyeBo7]

eyeBo7, could please tell me (a code noob) how to make my Eyetracker work like "take on helicopters"?

As I understand I need FreePIE (tobii version of FreePie?) with some sort of a Tobii script. Where can I find this script? Do I need to install something else then FreePie?

Thank you in advance

Hi! You can use FreePIE 1.9.

And then you map gaze data to a joystick or freetrack in the way you want and bind a joystick/freetrack to the camera in the game.

If you want more info, pm me your skype or email.

You can use this script as inspiration:

Code: Select all

import time
import math

class Util(object):
	HalfPi = math.pi / 2.0

	@staticmethod
	def IsEven(x):
		return x % 2 == 0

	@staticmethod
	def Clamp(value, minValue, maxValue):
		return min(max(value, minValue), maxValue)

	@staticmethod
	def SensitivityTransform(x):
		a = 2
		b = 1.2
		
		t = min(math.floor(b * x), 1)
		return ((1 - t) * (((b * x) ** a) / b)) + (t * (1 - ((((b / (b - 1)) * (1 - x)) ** a) / (b / (b - 1)))))

	@staticmethod
	def SensitivityTransformPowerOf(x, power):
		# y = 1 - cos((x ** 3) * HalfPi)

		if Util.IsEven(power) and x < 0:
			y = -(x ** power)
		else:
			y = (x ** power)
		return y

class Vector2(object):
	def __init__(self, x, y):
		self.X = x
		self.Y = y

	@property
	def Magnitude(self):
		return math.sqrt((self.X ** 2 + self.Y ** 2))

	def Scale(self, scaleFactor):
		self.X = self.X * scaleFactor
		self.Y = self.Y * scaleFactor

	def Normalize(self):
		self.X = self.X / self.Magnitude()
		self.Y = self.Y / self.Magnitude()

#######################################################################################################################

if starting:
	yaw = 0
	pitch = 0
	freeTrackDeltaRange = math.pi
	freeTrackDeltaVector = Vector2(0, 0)
	updateTimeStamp = time.time()

	# Game specific settings
	# Elite
	sensitivity = 0.5
	minYaw = -0.6 * freeTrackDeltaRange
	maxYaw = 0.6 * freeTrackDeltaRange
	minPitch = -0.2 * freeTrackDeltaRange
	maxPitch = 0.3 * freeTrackDeltaRange

deltaTime = time.time() - updateTimeStamp
rotationalSpeed = (freeTrackDeltaRange * deltaTime) * sensitivity
freeTrackDeltaVector = Vector2(tobiiEyeX.normalizedCenterDeltaX, tobiiEyeX.normalizedCenterDeltaY)
freeTrackDeltaVector.Scale(Util.SensitivityTransformPowerOf(min(freeTrackDeltaVector.Magnitude, 1), 1) * rotationalSpeed)

yaw -= freeTrackDeltaVector.X
yaw = Util.Clamp(yaw, minYaw, maxYaw)
pitch -= freeTrackDeltaVector.Y
pitch = Util.Clamp(pitch, minPitch, maxPitch)

freeTrack.yaw = yaw
freeTrack.pitch = pitch

updateTimeStamp = time.time()


#diagnostics.watch(freeTrackDeltaVector.X)
#diagnostics.watch(freeTrackDeltaVector.Y)
#diagnostics.watch(freeTrack.yaw)
#diagnostics.watch(freeTrack.pitch)

[RedRiot]

Hey there! a noob here aswell. was trying to reach you on youtube as i couldnt figure out how to register on this site before now. i basicly whant to do the same as you did in the Elite Dangerous video. set up my Steelseries Sentry to use as free cam in Dirt Rally and other games like it. i couldent figure out how to get this to work with FreePie and Vjoy sofar. as i dont know how to code it. and how they interact together and with the game. i get that the EyeX has to feed the Gaze info trough FreePie to Vjoy and that i have to bind the Vjoy Ax's to the ingame setting. how to do this im still fubling with it should work as i do see the oculus working in that game. dont they basickly do the same, just different methods? so yeah. any help would be imensly helpfull and apprichiated! struggeling to find toturials and forums to read up on this. this is the only one sofar that does Exactly what im thinking of doing

[Z3R0 SQUAD]

Yo, just wanted to say that this is crazy. I've always wanted eyetracking in games, but never knew how to make this happen; and now we have this!

To the point, I want to buy the steelseries Sentry next week, but I need to make sure that, if I did, I would have a solid foundation to learn all that is needed to make this work. For us noobs that are willing and dedicated to putting the time in and get it to work, is there any tutorial or explanation for what the lines of code do? Or even what the important lines are so we have solid a foundation to both learn and work from?

All I really want to understand is the basic implementation, I can figure out all the in-depth programming and stuff later

Thanks for any help you can give, and thanks for showing us the possibilities of eyetracking with Sentry!

[CyberVillain]

Hi and welcome, I'm sure if the hardware Wil be well received there will be tons of FreePIE scripts written by the community. Right now you are pretty alone though

[Z3R0 SQUAD]

Well, I guess it's up to me then. No programming knowledge, no tutorial, no problelm

[tepidbread]

Hello. I just wrote this simple script in free pie with the help of some example code that I could find online and was curious about some potential features that I could add.

#####################################################################
def update():
if (enabled):
mouse.deltaX = test.getY(tobiiEyeX.normalizedCenterDeltaX) * multiply
mouse.deltaY = test.getY(tobiiEyeX.normalizedCenterDeltaY) * multiply
if starting:
enabled = False
multiply = 30
tobiiEyeX.update += update

toggle = keyboard.getPressed(Key.ScrollLock)

if toggle:
enabled = not enabled

######################################################################

Forgive me if my code was poorly implemented or dumb. I am relatively new to free pie and coding in general. Last semester I learned C and Python.

Anyways I am currently controlling the view via a mouse movement that is generated via the eyex. So I am using mouse look in many games. I have not looked very much at the eyex SDK so my knowledge on the subject at hand is very limited and I think others may know more about this than me. My current solution seems very hacked together so I was wondering if there were other solutions.

I have three primary questions:

1. Could I somehow make the eyex created its own separated control axis for use in the game? ie. it would act like any other gamepad or joystick.

2. I thought this would just be a cool idea. Would it be possible to map the controls in such a way that your cursor in game moves exactly where you look (similar to the gta V implementation) but using free pie. I think the gta mod is separate plugin.

3. Would it be possible to remove the need for free pie entirely?

For those of you that are curious here is my control curve.

Control Curve.PNG

Edit:

I thought that I added comments to my code. I was wrong.

The code works by using the deviation of your eyes from the center of the screen and applying a control curve and multiplyer. it uses the afformentioned output information to generate a mouse movement.

you have to use scroll lock to toggle it on and off again... which kind of sucks because it can be annoying.

[CyberVillain]

1) If the game supports gamepads/joysticks you can use vJoy
2) If the joystick control ingame is absolute you could do it, otherwise you need to know the rotation of the camera and for that you need to inject into the game
3) Why would you want that?