It is currently Tue Oct 21, 2014 4:20 am



Reply to topic  [ 15 posts ] 
 Wii Hammer - Wii Remote 6 DOF Motion Controller 
Author Message
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
Hi,

I have built and successfully tested a six degree of freedom all optical motion controller using two Wii Remotes.

The Wii remotes are equipped with their own sensorbars which I call the "HammerHead". The converted Wii Remote with attachment is called the 'Wii Hammer".

I built this proof of concept device because I am not really impressed with the slow progress out there in Nintendo and gaming land. Hopefully there are some others who can take this concept further and shake it up a bit.

I originally proposed this idea for headtrackers. It may also be useful as a 6 DOF gun tracker for a First Person Shooter.

Quote:
I previously wrote - Two IR cameras can be used to create 6 degree of freedom tracking. It just needs both cameras to recognise each other face-to-face, one placed on the persons head and the other one on the display, and everything else can be calculated from there.


Quote:
Crim3 wrote - Could you explain a bit more how this would work, please?


Okay Mr Crim3 - you are a worthy modder who may appreciate this ......

Image


Fri Sep 04, 2009 6:52 am
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
Some other views of the Wii Hammer - the Wii Remote 6 DOF Optical Motion Controller.

Three Quarter View
Image


Plan View
Image


Side View
Image


Front View
Image


Fri Sep 04, 2009 7:01 am
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
Internal Views of Wii HammerHead sensorbar attachment


Internal View No 1

Image



Internal View No 2

Image


Last edited by budda on Fri Sep 04, 2009 7:33 am, edited 1 time in total.



Fri Sep 04, 2009 7:04 am
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
This is a pictorial representation of yaw and pitch calculations for the Wii Hammer 6 DOF Optical Motion Controller.

Please note, the angles shown for the yaw and pitch examples are exaggerated to illustrate the concept.

In practice the angles which can be seen by the Wii Remotes are limited to their respective fields of view.




YAW calculation example

Image





PITCH calculation example

Image


Last edited by budda on Fri Sep 04, 2009 7:46 am, edited 1 time in total.



Fri Sep 04, 2009 7:08 am
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
GlovePie script with working proof of concept code for trackir 6DOF emulation will be posted soon.


// ***START OF GLOVEPIE PROGRAM***

// Proof of Concept - Prototype Wii Remote 6 Degree of Freedom All Optical Tracking
// Requires Two Wii Remotes each with a Wii HammerHead sensorbar attachment
// Programmed using GLOVEPIE v0.30

// Fake TrackIR tested on Battle of Britain 2 - Wings of Victory - with patch v2.08 - Full 6 DOF cockpit model

// Concept and hardware by M.A.V. Brisbane Australia
// Public Release v1, 14 September 2009
// Thanks to all third parties including Carl Kenner (GlovePie), Natural Point Inc (TrackIR), Nintendo (Wii Remote), Shockwave (BOB2).

// Thanks to "Meant to be Seen" MTBS for originally hosting this post at MTBS Forums
// viewtopic.php?f=6&t=4251


pie.framerate = 100Hz

var.screenwidth = screen.Width
var.screenheight = screen.Height

// Wii camera centre pixel coordinates
var.wiicameraxcentre = 512
var.wiicameraycentre = 384

// Wii camera nominal half field of view x & y axis
var.wiicameraxhalfFOV = 20
var.wiicamerayhalfFOV = 15

// Wii camera nominal angular resolution
var.wiicamangletopixel = 20/512

// Wii Hammerhead sensor bar LED separation in centimetres for both Wii remotes
var.S_ledbaseline = 6


//CONNECTION ORDER for TWO Wii Remote controllers
// First wiimote connected is mobile wiimote1 followed by stationary wiimote2

debug = ' working '

// Check for and read in all IR point coordinates at once

if wiimote1.dot1vis AND wiimote1.dot2vis AND wiimote2.dot1vis AND wiimote2.dot2vis

var.w1dx1 = wiimote1.dot1x
var.w1dy1 = wiimote1.dot1y

var.w1dx2 = wiimote1.dot2x
var.w1dy2 = wiimote1.dot2y

var.w2dx1 = wiimote2.dot1x
var.w2dy1 = wiimote2.dot1y

var.w2dx2 = wiimote2.dot2x
var.w2dy2 = wiimote2.dot2y

// debug = 'W1 LED 1 ( ' + var.w1dx1 + ' ,' + var.w1dy1 + ' ) ' + ' W1 LED 2 ( ' + var.w1dx2 + ' ,' + var.w1dy2 + ' ) ' + ' W2 LED 1 ( ' + var.w2dx1 + ' ,' + var.w2dy1 + ' ) ' + ' W2 LED 2 ( ' + var.w2dx2 + ' ,' + var.w2dy2 + ' ) '

// CODE FOR MOBILE WIIMOTE - W1
// Mobile Wii Remote wiimote1 looking at sensorbar2 on stationary wiimote2

// Mobile wiimote IR pixel coordinates looking at stationary sensorbar w2 leds (0,0) bottom left corner
// w2 IR camera centre position located at middle of w2 sensorbar leds

var.M_saveragex = (var.w1dx1 + var.w1dx2)/2
var.M_saveragey = (var.w1dy1 + var.w1dy2)/2

// for rotation calculations

//ordinary coordinates for w1 transformations (-512, -384) bottom left corner
var.M_sanglex = ( var.M_saveragex/512 - 1 ) * var.wiicameraxhalfFOV // x positive to right of vertical
var.M_sangley = ( var.M_saveragey/384 - 1 ) * var.wiicamerayhalfFOV // y positive up from horizontal


// Twist gradient for artificial horizon as seen by mobile w1 camera (0,0) bottom left corner
var.twistgrad = ( var.w1dy2 - var.w1dy1)/( var.w1dx2 - var.w1dx1 )

// NOTE: roll angle between +90 to -90 with atan function
// The actual roll angle for the mobile w1 remote is opposite to the perceived artificial horizon roll angle
var.M_roll = 0 - atan( var.twistgrad )


// for translation calculations
// stationary sensorbar2 led pixel separation distance as seen by mobile remote w1
var.M_Sledbaselinepixeldist = sqrt( ( var.w1dx1 - var.w1dx2)^2 + (var.w1dy1 - var.w1dy2)^2 )

// End mobile w1 code


// CODE FOR STATIONARY WII REMOTE - W2
// Stationary Wii Remote wiimote2 looking at sensorbar1 on mobile wiimote1 (0,0) bottom left corner

var.S_maveragex = (var.w2dx1 + var.w2dx2)/2
var.S_maveragey = (var.w2dy1 + var.w2dy2)/2

// for rotation calculations
// absolute x,y angle coordinates of mobile w1 camera centre as seen from stationary w2 (-512, -384) bottom left corner

var.S_manglex = ( var.S_maveragex/512 - 1 ) * var.wiicameraxhalfFOV
var.S_mangley = ( var.S_maveragey/384 - 1 ) * var.wiicamerayhalfFOV


// for translation calculations
// for w2 stationary camera centre axis to w1 mobile camera centre included angle as seen from stationary w2

var.S_angleYP = sqrt(var.S_manglex^2 + var.S_mangley^2)

// End stationary w2 code


//********************************************************************************************************************
// STATIONARY WII REMOTE w2 - YAW AND PITCH COMPONENT CALCULATIONS - WITHOUT ANY OFFSETS AND WITH SENSE CORRECTION

var.S_yaw = 0 - var.S_manglex

var.S_pitch = var.S_mangley

var.S_roll = 0


//********************************************************************************************************************
// MOBILE WII REMOTE w1 - YAW AND PITCH COMPONENT CALCULATIONS WITH ROLL RECTIFICATION

// METHOD: Rotate the angle coordinate vector about the origin so the s2 artificial horizon is effectively level
// Roll rectification rotation of coordinates is in the same sense as the actual mobile camera roll rotation


var.M_Sxref = var.M_sanglex * cos( var.M_roll ) - var.M_sangley * sin( var.M_roll )

var.M_Syref = var.M_sanglex * sin( var.M_roll ) + var.M_sangley * cos( var.M_roll )


//*********************************************************************************************************************

// MOBILE WII REMOTE w1 - YAW AND PITCH COMPONENTS AFTER ROLL RECTIFICATION AND SENSE CORRECTION

var.M_yawangle = var.M_Sxref // rectified yaw angle component

var.M_pitchangle = 0 - var.M_Syref // rectified pitch angle component


//*********************************************************************************************************************

// ANGLE ROTATION SECTION

// NET REAL YAW, PITCH & ROLL ANGLES as detected by the Mobile Remote looking at the Stationary Remote
// One to One direct motion output
// Angles in DEGREES
// Based on Right Hand Coordinate System
// PHYSICAL VIEW basis - Motion measured relative to the Stationary Wii Remote Axes Orientation


var.yawout = var.M_yawangle + var.S_yaw // yaw positive is left of vertical

var.pitchout = var.M_pitchangle - var.S_pitch // pitch positive is up from horizontal

var.rollout = var.M_roll // roll positive is anticlockwise from horizontal

// debug = 'Mobile Controller Output Angles (yaw, pitch, roll) ( ' + var.yawout + ', ' + var.pitchout + ', ' + var.rollout + ' )'


//*********************************************************************************************************************

// OUTPUTTING ANGLES FOR APPLICATIONS

// CUSTOM Angle Profile settings can be adapted here for particular applications/features
// Examples of features include Smoothing, Deadband, Inversion, Mouseview, Sensitivity Bias and TrackIR emulation
// TrueView has not been implemented
// Note: ROLL may require inversion for applications based on a Left Hand Coordinate system

//Custom Angle settings for Battle of Britain 2
//Game settings, Options - Advanced - Game : check TRACKIR & VECTOR EXPANSION & VE Z AXIS MODE ( Z Axis or FOV mode as required)


var.interimy = 0 - 6 * var.yawout
var.interimp = 6 * var.pitchout
var.interimr = 0 - var.rollout


FakeTrackIR.yaw = Smooth( var.interimy , 5)

FakeTrackIR.pitch = Smooth( var.interimp , 5)

FakeTrackIR.roll = Smooth( var.interimr, 5)


//*********************************************************************************************************************

// TRANSLATIONS SECTION


var.M_Sobliquesemibaselinedist = var.S_ledbaseline/2*cos( var.S_yaw )

var.M_Sbaselinehalfangle = var.wiicamangletopixel * var.M_Sledbaselinepixeldist /2

var.twistaxisdistance = var.M_Sobliquesemibaselinedist/tan( var.M_Sbaselinehalfangle)

var.zdistance = var.twistaxisdistance * cos( var.S_angleYP )


// REAL TRANSLATIONS
// NET REAL X, Y & Z translations as detected by the Mobile Remote looking at the Stationary Remote
// One to One direct motion output
// Translations in CENTIMETRES
// Based on Right Hand Coordinate System
// PHYSICAL VIEW basis - Motion is measured relative from the centre point of the Stationary Wii Remote Camera


var.xtranslation = var.zdistance * tan( var.S_yaw ) // X positive is directed right of vertical

var.ytranslation = var.zdistance * tan( var.S_pitch ) // Y positive is directed up from horizontal

var.ztranslation = var.zdistance // Z positive is directed backwards on the horizontal

//debug = ' Mobile Controller Output Translations (x,y,z) (' + var.xtranslation + ', ' + var.ytranslation + ', ' + var.ztranslation + ' )'

//*********************************************************************************************************************

// OUTPUTTING TRANSLATIONS FOR APPLICATIONS

// CUSTOM Translation Profile settings can be adapted here for particular applications/features
// Examples of features include Smoothing, Deadband, Inversion, Sensitivity Bias, Zooming and TrackIR emulation
// TrueView has not been implemented
// NOTE: TrackIR and FakeTrackIR Translation outputs need to be in METRES (refer GlovePie Help Manual)
// NOTE: Z-axis motion may require inversion for applications based on a Left Hand Coordinate system

//Custom Translation settings for Battle of Britain 2
//Game settings, Options - Advanced - Game : check TRACKIR & VECTOR EXPANSION & VE Z AXIS MODE ( Z Axis or FOV mode as required)

FakeTrackIR.x = Smooth(var.xtranslation/100, 5)

FakeTrackIR.y = Smooth(var.ytranslation/100, 5)


// FOV MODE - zooming for external view
// FakeTrackIR.z = Smooth( (0.6 - var.ztranslation/100 ) * 1.5 , 5)

// Z AXIS MODE - zooming for cockpit instruments - internal view
FakeTrackIR.z = Smooth( (0.6 - var.ztranslation/100 ) * 1.5 , 5)


// end translation section

else // LED coordinate not detected

// debug = 'W1 LED 1 ( ' + var.w1dx1 + ' ,' + var.w1dy1 + ' ) ' + 'W1 LED 2 ( ' + var.w1dx2 + ' ,' + var.w1dy2 + ' ) ' + 'W2 LED 1 ( ' + var.w2dx1 + ' ,' + var.w2dy1 + ' ) ' + 'W2 LED 2 ( ' + var.w2dx2 + ' ,' + var.w2dy2 + ' ) ' + wiimote1.dot1vis + ' ' + wiimote1.dot2vis + ' ' + wiimote2.dot1vis + ' ' + wiimote2.dot2vis


end // stop

//***END OF GLOVEPIE PROGRAM***


Last edited by budda on Wed Sep 16, 2009 4:11 am, edited 3 times in total.



Fri Sep 04, 2009 7:12 am
Profile
3D Angel Eyes (Moderator)
User avatar

Joined: Sat Apr 12, 2008 8:18 pm
Posts: 10927
WOW! :woot

_________________
check my blog - cybereality.com


Fri Sep 04, 2009 11:38 pm
Profile
Certif-Eyed!
User avatar

Joined: Fri May 11, 2007 10:13 am
Posts: 521
cybereality wrote:
WOW! :woot


You say what I think...


Thats... Awsome :shutter

Not I have a intetion to get my wiimotes back working on my pc...

_________________
Image


Sat Sep 05, 2009 2:08 am
Profile
One Eyed Hopeful

Joined: Tue Feb 12, 2008 8:37 am
Posts: 5
Great work budda, many thanks.
I have few questions:

1) What brand of Ir LED you are using? Their tips look flat not spherical. Did you cut them off (for better Ir light emission) or that's how they where when you bought them.
2) Does your algorithm take (maybe you do and I didn't understand your script well, just asking :) ) into consideration focal length of Wiimote Ir camera, or we can omit that? I've been reading some texts on the internet about similar works (6DOF with wiimote projects) and some people add calculations to correct images according to focal length before using IRs positions.
3) do you have same algorithm/code in C# or C? That would make so easier for me to play with your script.

Thanks


Sat Sep 19, 2009 3:24 am
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
Hi,

Thanks for your interest.

To answer your questions:



1. The LED was filed back mainly for appearance.

I suggest using a regular 5mm IR LED ( 940nm ) with the rounded lens intact for improved performance.

In this case a 40 degree field of view for the IR LED emmission is desirable.

This concentrates the IR light in the useful 40 degree detection range of the Wii Remote, and makes the sensorbar work more effectively with distance.

I also suggest to minimise the height of the exposed IR lens to reduced unnecessary back reflections scattered from the enclosure case.



2. The GlovePie script takes no particular account of the optics of the IR camera.

Such a small focal length camera has a very big depth of field, so practically all image points are essentially at the same level of focus. (Note some out-of-focus may be tolerated or desired for improved blob-tracking and manufacturability).

As a first order approximation, the program assumes a linear correspondence of the viewing angles and the coordinates detected by the camera.



3. This program was developed using GlovePie because much of the hard work for interfacing has been done already, and it was relatively easy to prototype this application for a non-programmer like myself.

So no, there has been no other coding in other computer languages.

In any case, further development of this concept will need to be performed with proper code, as GlovePie does not have the programming flexibility to fully exploit the possibilities.

Also GlovePie v0.30 apparently has some bugs (which I have not encountered so far) when combining camera functions with button functions. This could be a showstopper, and hopefully Carl Kenner or others may come out with GlovePie v0.31 .... with all the fixes.



4. As a further note, I have seen other Wii 6 DOF motion control concepts described on the internet.

It really astounds me that such a meal has been made of this. There are quite a few University level papers written about this using advanced mathematics.

This concept is fairly simple mathematically and is elegantly implemented. There is only one "IF" statement and some high-school level trigonometry.

There is also much scope to improve the basic concept both in hardware and software, with new features and capabilities. There is still plenty of meat left on the bone.

Thanks.


Sun Sep 20, 2009 2:47 am
Profile
One Eyed Hopeful

Joined: Tue Feb 12, 2008 8:37 am
Posts: 5
I agree that your script seems to be the most elegantly applied one I've seen so far.


Sun Sep 20, 2009 10:40 am
Profile
One Eyed Hopeful

Joined: Tue Feb 12, 2008 8:37 am
Posts: 5
Innovation and creativity. This situation is very similar to what we see in Hollywood. Many big budget movie studios spend millions and millions of dollars for special effects but because their stories lack creativity people don't want to see their movies and they don't make revenues. PS3 is nothing but high performance new gen console but Wii came with wiimote which uses effectively its sensors and has a great IR Cam. Also Nintendo helps a lot to developers for how to use its technologies effectively, on the other hand programming PS3 is very difficult because its processor is different from what developers used to program on. Hence wii people didn't see much special effects but experienced new ways of playing games. A girl once told me "if you are not rich you can't have fun" I disagree and numbers in this example I believe prove I'm right.


Mon Oct 12, 2009 3:04 am
Profile
Certif-Eyed!

Joined: Sat Sep 22, 2007 3:11 am
Posts: 639
Location: Spain
Not only a great project, but also so well explained. Thank you so much.
I've been missing this post till now. Mtbs is becoming too big to fully browse the forum with the little free time I'm having lately. Now I'm afraid I won't be able to calmly study the script in quite a time.

_________________
Zalman Trimon ZM-M220W / Acer H5360 with Another Eye2000 shutters / nVIDIA GTX285


Wed Oct 14, 2009 6:17 am
Profile
One Eyed Hopeful

Joined: Tue Jun 14, 2011 1:04 pm
Posts: 6
Budda,

I wanted to check in with you and see if there has been any progression regarding the hammer since you first developed it. What have you found to be the best usage of the device so far, whether it be a platform, game, etc.? Does the device require a game or server with high availability to function properly? I would be very interested to know where this awesome device fits best in the gaming world so I can transfer the information to a few IT staffing agencies. Have you been making any progress?


Last edited by quakelore on Wed Jan 18, 2012 6:35 pm, edited 2 times in total.



Wed Jun 15, 2011 12:00 pm
Profile
Binocular Vision CONFIRMED!

Joined: Thu May 22, 2008 9:13 pm
Posts: 301
Hi,

Thanks for your ongoing interest.

I have continued work on this project with some hardware improvements and some software improvements.

Its still the same basic concept and hardware, which could work with any platform or PC application.

I am sure there could be commercial success with this idea given some tender loving care.

As I noted previously, this device could be used as a head tracker, or gun aiming device for a first person shooter.

The essential elements are the IR LEDS and IR Camera, which could be configured in various ways to acheive the same results.

For instance, I would like to see a wireless version of the Wii Camera (without the Wii Remote body) attached to some glasses.

In principle this should be easy to do, but no-one has apparently done it.

Another idea would be to mount a miniature fish-eye lens on the Wii Camera to get wide angle tracking. And so on.

There seems to be a fad at the moment with the Microsoft Kinect and other camera based tracking systems whereby the user does not have to wear anything.

Given the over 65 million Wii Remotes in US households, there's no shortage of cheap parts to make Wii Hammers in the future.

With the newly announced Wii U also being compatible with Wii Remotes, the use and sale of Wii Remotes is sure to continue.

Interestingly, the Wii U hand controller tablet seems to have a camera and sensorbar mounted at the top of the touchscreen display. This is a similar to the IR LED and IR Camera configuration as exists with the Wii Hammerhead when fitted to a Wii Remote. Mmmm.

I wonder what Nintendo are up to, particularly when they previously blocked the native head tracking feature in Stephen Spielbergs Bloom Box game on Wii.

Sadly, for those interested in progress, the Wii Hammer project is dormant but not done. I don't intent to leave it unfinished when there is easy potential to be extracted from it. When something new comes up, I will announce it. However, don't wait for me ..... make your own. Ha.

Surely, the are plenty of college level students looking to do interesting projects of this kind on a budget.

Thanks.


Sat Jun 18, 2011 3:23 am
Profile
One Eyed Hopeful

Joined: Tue Feb 12, 2008 8:37 am
Posts: 5
Hi budda
It's great that you still work on this. I think that's a great project.
Do you have any C# experience or did you try to pass that algoritm to C# ?
Thanks


Mon Jun 20, 2011 4:54 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 15 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group
Designed by STSoftware.