Going to college, need a project.

Post by **bobv5** » Sun Sep 02, 2012 7:14 pm

I'm only working part time at the moment, so am also going to college. As part of my course I need a project to work on. I have a few ideas, but I'm wondering if any of you guys have any better ones.

1: VR gun recoil- In theory I have already done this. In reality it has not been tested, and will likely have stupid software bugs that will take ages to fix by myself. EDIT-If any of you software guys are intrested I can post details of the hardware.

2: Ambilight clone - I think even if consumer HMD's get really awesome, people will be using normal screens for quite a few years. Current Ambilight clone systems have a big problem, in that they require PC side driver support, and simply don't work with non PC systems. I have already done this, but the PC driver software is a pain in the anus. I would like to make something that connects via a splitter cable and is invisible to the source.

3: Motion tracking- Seems to be mosty a software issue. I suck at software.

4: Z80 computer- Will help me learn machine code, but I'm not sure that is usefull now.

5: Sumo robot- Robot's are awesome.

Post by **brantlew** » Sun Sep 02, 2012 7:30 pm

How about a telepresence robot? There are a number of projects like that on the web, but you could take it to another level if you integrated an HMD and head-tracker control mechanism. Even just a table sitting robot that had remote neck control via head-tracker. If you really wanted to get crazy you could merge the GoPro camera project with a robotic neck and the Rift and really simulate being in a different location.

Post by **Zaptruder** » Sun Sep 02, 2012 7:43 pm

How about a motion controlled telepresence robot with a VR force feedback gun that uses that ambilight system for muzzle flashes?

Post by **bobv5** » Sun Sep 02, 2012 7:51 pm

That sounds like a software project. I suck at software!

The course I am doing is electronic engineering. While that subject now includes some progaming, as microcontrollers need firmware, I think your sugestion would be more suited to a programming course. A robotic neck would be fairly easy to make, but the software to drive it would be far beyond my skills. If you want to write the code I will add it to the list...

Post by **cybereality** » Sun Sep 02, 2012 8:10 pm

If you want to do the ambilight thing I can help with the software. Basically I could expose the image data from the DirectX backbuffer and you could process that yourself to do what you need. So it would still require some software support on your end, but almost any reasonably complex project will anyhow.

Another idea that would be cool: cyber-skates. Basically roller-stakes that have servos on them that spin in the opposite direction you are moving in as to cancel out the movement. Would be very cool if a system like this could work, you could potentially make a small room into an infinite space if they worked well.

Post by **bobv5** » Sun Sep 02, 2012 8:37 pm

Cheers Cyber, but other guys have already made software to send the color data down a serial port. My idea was to make an ambilight that works from vga or dvi and does not need drivers.

If you could send some sort of haptic data, that would be a really good project.....

As for skates, this is a college project not a PHD project. But I will think about it. I could probably build the skates in a few months. But the software would be a massive task.

Post by **android78** » Sun Sep 02, 2012 9:20 pm

The telepresence robot seems like a good idea and can be a purely electrical/engineering project if you use open source for the software for the head tracking:
http://www.rcgroups.com/forums/showthre ... 559&pp=100
There are good video transmitters/receivers here (the TX5823 and RX5808 have been used with reasonable success for medium range):
http://www.foxtechfpv.com/fpv-equipment ... 2&sort=20a
I'm not sure your budget and what you already have on hand, but if you could do stereo FPV compatible with the rift, then I'm sure there would be a few here who would help out with advice.

Post by **zalo** » Sun Sep 02, 2012 11:00 pm

Well, if brant gets to say Telepresence robot, then I get to say Omni-directional treadmill!

This is purely an engineering problem and if you come up with a good enough solution, you could do a kickstarter and sell me one.

Post by **android78** » Sun Sep 02, 2012 11:26 pm

I would prefer cyber-skates (omni skates or whatever you want to call them) over treadmill and I think they would be easier (and cheaper) in a lot of respects, but I think bob could be right in that it could be a bit more then a semester project.
If he did go down the path of cyber skates, I've lots of drawings that I would share with him... I'm sure cyber has too though. lol

Post by **Chriky** » Mon Sep 03, 2012 3:07 am

The way I see it, the "big three" technologies for "good enough" VR (within rounding distance of sci-fi stuff) are;
- High FOV/Resolution HMD
- Accurate 6DOF positioning
- Small/cheap omni-directional treadmill

These are what the DIY VR community has been wanting for a long time.

Well, Palmer has pretty much struck the first one off the list, the second one is pretty software heavy, so I would go with the third one.

You don't need to actually make the thing, you just need to demonstrate something that is feasible and people will jump in to help. A perfect project would be some kind of hardware set up with basic software demonstrating what data you can get hold of. Just like a screen that shows that you can work out when the feet are down, what direction the user is facing etc.

The "6DOF headtracking ideas" thread was pretty successful, but there hasn't been one for ODTs.

Post by **brantlew** » Mon Sep 03, 2012 7:20 am

Correction - there hasn't been one in a good while. This was one of the best threads on MTBS since I've been coming here. Dozens of ideas for omni-treads...

http://www.mtbs3d.com/phpBB/viewtopic.p ... irectional

Post by **Chriky** » Mon Sep 03, 2012 7:58 am

Ah yeah I remember that thread. I hadn't found the forums when it started but I remember reading the whole thing at one point, it was still getting posts until quite recently.

Post by **FingerFlinger** » Mon Sep 03, 2012 8:39 am

I'd probably put haptic feedback at #4 on Chriky's list, and I've had an idea for a haptic glove kicking around in my head for a few days. Run fishing line to each finger tip and route them to a gauntlet on the user's wrist. When the user flexes their hand, the gauntlet can let out line, and when they relax it, springs in the gauntlet will retract the line. When the user's fingers encounter a virtual object, control a clutch on each finger individually to approximate the shape of the object. Obviously, making a reliable and slim mechanism is the tough part. I've got a few thoughts, but nothing that seems great. For a school project, just exposing the device control would be enough, no real need to interface it with an entire simulation.

Post by **android78** » Mon Sep 03, 2012 3:50 pm

One other idea I had was to use a TENS machine for feedback. Putting electrodes reasonable close together can get really specific targeting for when you get shot in a game from a certain direction. I imagine that 8 pairs of electrodes could give a great sense of realism. Not sure how you can get this kind of information from the game, but I think I read someone was able to get direction information by sensing red flashes on certain parts of the screen.
Even if you were to just have it as a kind of kick in the palm of your hand when you pull the trigger, it could be good.

If you want to try out the kind of stimulation I mean, then these are pretty cheap:
http://www.ebay.com.au/itm/4pads-Electr ... 8294wt_906

Post by **pierreye** » Mon Sep 03, 2012 6:09 pm

I would be interested in a good home security robot.

Post by **cybereality** » Mon Sep 03, 2012 7:01 pm

pierreye wrote:I would be interested in a good home security robot.

I've got just the thing...

Post by **pierreye** » Mon Sep 03, 2012 9:08 pm

That's a good one. Seriously, the project could be just a simple robot with sound/movement sensor, able to patrol the house and call/sms for intrusion alert. Also able to control the robot through WIFI would be great.

Post by **Namielus** » Mon Sep 03, 2012 9:26 pm

Ironically one of the other kickstarter projects named Oculus is a telepresence robot.

http://www.kickstarter.com/projects/289 ... t?ref=live

Note the similar logo too

Post by **bobv5** » Mon Sep 03, 2012 9:37 pm

So, it is decided. I must build ED-209.

One of you guys can fly over to switch it on though.

Post by **bobv5** » Tue Sep 04, 2012 8:50 pm

Can one of you guys poking about in games to make 3D drivers, have a look what could be exposed to drive haptics devices? It seems to be the only useful area that I have the skills to help with, but don't want to spend ages working on it if the PC side software is going to be non-viable.

Post by **cybereality** » Tue Sep 04, 2012 8:54 pm

Yes, I would love for haptics to be a part of my driver (or anyone else's for that matter). I take it that its possible, only because Novint and TN Games have 3rd party drivers for games they clearly did not have source code for. But I am not 100% sure what they are doing. With DirectX you may be able to "detect" when certain textures or shaders are being used, and this may indicate you are under attack. But I don't know how specific that would be (ie enough to shake a rumble pack, but probably not precise enough to power a vest). They may even be hacking the games themselves, which seems more promising but I haven't too much experience with this. So I do think its possible, I'm just not sure how easy or feasible it would be. Hope that helps.

Post by **android78** » Tue Sep 04, 2012 9:04 pm

cybereality wrote:Yes, I would love for haptics to be a part of my driver (or anyone else's for that matter). I take it that its possible, only because Novint and TN Games have 3rd party drivers for games they clearly did not have source code for. But I am not 100% sure what they are doing. With DirectX you may be able to "detect" when certain textures or shaders are being used, and this may indicate you are under attack. But I don't know how specific that would be (ie enough to shake a rumble pack, but probably not precise enough to power a vest). They may even be hacking the games themselves, which seems more promising but I haven't too much experience with this. So I do think its possible, I'm just not sure how easy or feasible it would be. Hope that helps.

I wonder if they would use video or audio though. I'm not entirely sure how the audio is handled, but it would require fairly accurate source positional data.

Post by **bobv5** » Tue Sep 04, 2012 9:41 pm

Would it be possible to make a software that is the opposite of the old cheat cartridges, that read memory values instead of writing to them? I'm guessing not due to antivirus/drm/memory randomisation stuff.

I'm tempted to burn my PC's, buy a video accelerator for my C64, and go back to the stone age.

Post by **cybereality** » Tue Sep 04, 2012 9:58 pm

Yes, you can read memory values directly. This is how some software (like CheatEngine) works. It would be enough to tell, for example, that your life was going down. But to detect the exact location of a hit on your body, seems a little harder.

Post by **bobv5** » Tue Sep 04, 2012 10:14 pm

It has to be stored somewhere though. I'm not asking you to program it for me, but it would be good if you could have a look next time you play with that stuff. What would I need to do to look at that stuff myself?

As much as I say I suck at programming, I'm good enough that I modified the Linux kernel to run on an old laptop somebody gave me. I had to add a couple of cycles wait in the keyboard driver. Then X stopped crashing. The problem was caused by a bug in the bios of the dinosaur. Sorry if that sounds lame, but I was well pleased with myself

Windows.. Doesn't work that way.....

Post by **cybereality** » Wed Sep 05, 2012 6:43 pm

Yeah, you are right. At some point that data is in the system somewhere. So its not an impossible task.

Post by **bobv5** » Thu Sep 06, 2012 6:19 pm

Been having a look arround, seems like I was asking for more than I thought I was in my earlier posts.

Cheat engine looks like it might do the job. The problem is, how to find which of those millions of addresses is the data in.

Another option might be something like a source SDK mod.

For something like a rumble pack, I am already doing that directly in hardware from the audio. Better than nothing but really not ideal.

Post by **pierreye** » Thu Sep 06, 2012 8:26 pm

Most fps game use flashing red on the edge of the screen to indicate the direction of the hit. If you can analyse the frame where the red flash happen, you can derive the info to the haptic suit on which motor to activate. I'm thinking if FRAPS can capture the video realtime, then it is possible to tap into the video and look for visual indicator.

Post by **cybereality** » Thu Sep 06, 2012 8:40 pm

pierreye wrote:Most fps game use flashing red on the edge of the screen to indicate the direction of the hit. If you can analyse the frame where the red flash happen, you can derive the info to the haptic suit on which motor to activate. I'm thinking if FRAPS can capture the video realtime, then it is possible to tap into the video and look for visual indicator.

Yes, I had this same notion, it could probably work for certain titles. Maybe something as simple as checking if the combined pixels from the red channel are past a certain threshold (should be easy to program). I certainly would like to explore this more with my driver, so it will probably be after the first release (just focusing on the core Rift support right now).

Post by **android78** » Thu Sep 06, 2012 9:03 pm

@cyber - I'm not really up on how the graphics are processed, but are you unable to see when a certain texture is being applied in post processing? I would think that the red flashes would be fairly simple overlay textures applied after the scene is rendered, similar to other HUD elements.

Post by **cybereality** » Thu Sep 06, 2012 9:25 pm

Yes, finding if a certain texture or effect is being used is totally possible. However it would have to be done on a per game basis. I think I did mention this earlier.

Post by **bobv5** » Tue Sep 25, 2012 4:20 pm

Pretty much commited to doing a haptic system now, unless somebody has a great idea no one has mentioned. The plan is to refine the haptic vest I already made (based arround tactile transducers), and complete the haptic gun I have designed and etched PCB's for, but not yet assembled (it attempts to recover gunshots from game audio). As long as it is properly documented this will get me a good grade.

As I already have a design that works in the simulator, and shouldn't need too much debuging (I hope), I will use the remaning time to investigate including serial input for software to drive my hardware directly, and more advanced forms of haptic output.

Any thoughts?

Post by **cybereality** » Tue Sep 25, 2012 5:19 pm

Sounds cool. Can you elaborate on what kinds of effects you can simulate with this vest of yours? I thought the 3rd Space vest was cool, except it was only really good for piercing type effects (like getting shot). I would have liked if there was also more of a rumble effect in the whole vest for more subtle response (like the engine of a race car for example, or a helicopter flying nearby). Can your vest also do this type of stuff?

Post by **AugmentedR** » Tue Sep 25, 2012 9:42 pm

wrong thread

Post by **bobv5** » Fri Sep 28, 2012 5:27 pm

Fatctual:
It is very similar to Aura Interactor. It produces effects like those you mentioned, engines, gunshots, explosions, wind, wind when falling, sea, etc. It can't (yet) do piercing effects. I have done experiments with directional effects (not built well enough for regular use), using surround sound decoding, and that seems like it will add a lot, so that will be part of the refinements I mentioned. In it's current form it is better for vehicle games, as gunshots can be felt even when they miss. A filter to eliminate distant or quiet noises is already part of the gun circuit, so could easily be added to the vest circuit.

Subjective:
It really does add to games, some people say that Aura Interactor gets annoying after a while, but from my experiments I would guess that they have it turned up far too high. You want it high enough to feel it, but low enough to forget about it and concentrate on the game. The most impressive effects are the ones you don't really notice untill you think about it. With my vest thing, a racing game, a PS1 pad hacked for PC, a crappy HMD, and all the problems hacked together gear brings with it, was the first time I experienced "true" immersion. It was enough to fool my brain, just for a few seconds, that it was dealing with some kind of machine, not a computer game. The wind in part of Mass Effect 2 was also really good.

Post by **cybereality** » Fri Sep 28, 2012 6:03 pm

Hmm.. I think you just convinced me to get a Aura Interactor...

Actually, nevermind. Just watched this: http://www.youtube.com/watch?v=oFTsFlxDLBk , Arrgg!

Post by **bobv5** » Fri Sep 28, 2012 6:46 pm

Cyber, if that link is the stupid jazz guy, I'm going to find a way to make haptic feedback hurt you. A lot.

EDIT: OK, it was a proper video. I admit it didn't look good. Like I said I think he had the power set far too high. It was also reproducing frequencies that are far too high. I have the schematic for the box on my screen, if you can use a soldering iron it should be easy to improve, I can give you simple instructions but I accept no responsibiliy if something goes wrong!

Post by **cybereality** » Fri Sep 28, 2012 7:16 pm

No joke this time.

Anyway, I think I'll pass on the Interactor for now. I know I will get a bug sometime and order it, but for the time being I have more than enough stuff on my hands.

Post by **bobv5** » Fri Oct 12, 2012 4:55 pm

Been looking at that schematic again, it is a really clever circuit, it should be able to give far better performance than it does in that video. It has 2 important controls, one is the overall force of the effects. The other is the threshold at which it activates. What this means is that you can set it to only respond to sounds above a certain level, and ignore people speaking etc.

It also synthesises lower frequencies that would not have been present in the 16 bit consoles. That isn't needed in modern games, but can easily be disabled. (Cut one wire)

The sound coming from that unit in the video would not be so bad if the system was configured corectly, but would still be present. My system does the same to a far lesser degree. With headphones, it is hidden.

Post by **android78** » Sun Oct 14, 2012 5:35 pm

bobv5 wrote:Been looking at that schematic again, it is a really clever circuit, it should be able to give far better performance than it does in that video. It has 2 important controls, one is the overall force of the effects. The other is the threshold at which it activates. What this means is that you can set it to only respond to sounds above a certain level, and ignore people speaking etc.

It also synthesises lower frequencies that would not have been present in the 16 bit consoles. That isn't needed in modern games, but can easily be disabled. (Cut one wire)

The sound coming from that unit in the video would not be so bad if the system was configured corectly, but would still be present. My system does the same to a far lesser degree. With headphones, it is hidden.

Looking at the video, I'm not sure that he understood how to set it up properly (effect knob, whatever that does). That being said, I have to agree with him that it would only work if there were games programmed for it with a secondary audio output, just for when getting hit. If I was going to try it with the fighting game though, I think that turning the music off would help a lot.