New here with my micro 3D Cam(motion capture or AR)

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply

Do you think it's a good idea?

yes
2
100%
no
0
No votes
maybe yes
0
No votes
maybe no
0
No votes
 
Total votes: 2

User avatar
davyfirst
One Eyed Hopeful
Posts: 6
Joined: Tue Jun 17, 2014 1:37 am

New here with my micro 3D Cam(motion capture or AR)

Post by davyfirst »

Hi everyone, I'm new here, my name is davy(friends call me champion, because I have a "1st" in my username, due to the Yahoo registration in 1998, "davy" was occupied, so the system added a "1st". :lol: ).
I've been interested in VR from 2007. Lately I've got an idea in motion capure and go apply for a PCT patent. When building the prototype I still have some uncertain factors.
Heard about that there are many top VR researchers in this forum. So I come here to post my idea and the mini sensor I made. Hopes to get helps, suggestions or even criticisms. Thanks haha :)
I name it the “X-immerse”.

1st: The idea---3D cam to capture the small Infra-red LED beacons located on the gloves, rings, and guns. Along with it is the micro vibration motor for force feedback. In the glove and ring , I think I can make the module very small and light so you`ll feel nothing. The force feedback and gun-controller button signals links the PC with a Bluetooth Hub.
When ur finger touchs the virtual pannel or other interaction occurs, the corresponding module vibrates. (the glove or the rings can handle most of the virtual interactions,I think)

The gun-controller is available in game,a pistol or rifle in FPS, a sword or dagger in ACT, maybe. The 3d cam work in 90hz/120hz(different accuracy)with 120 degree FOV,to capture the quick movement of the glove or controller. I also have an exciting idea that when the controller hung on girdle, it sleeps until you touches it, when it wakes the same side glove go to sleep(coordinated by the Bluetooth Hub). In the game,you can fire the virtual weapon with one hand and open door, cast spell with the other hand.


Image
Figure in patent file.



2nd:The 3d cam 90hz/120hz,120Fov. Made in my friend's factory. It's so cheap (we knew each other through a golden retriever community, ha, thanks to my dog).


Image
Image
Image



3rd:the glove.It's cheap. the IR LED and vibration motor have not been assembled yet. I think thumb and index finger can do most of the job, and it's easy to track. If you want to give the finger, you might need a "contral VR" LOL. But in the future I think I can add some curve sensor onto the glove to make it happen.(I dont like the idea, because I love simple things)

Image
Image
The leather is too thick.


4th:The gun. The gun have stick to control movements, buttons to do action, and have stronger vibration. It's also cheap (things in china are so cheap ,and I have a bunch of friends supports my idea to make it even cheaper. )I'm worry about the quality because I have no quality control exp, I might need someone more experienced to do this.
Image
I draw this on my noteII when the first time I have the idea.



5th:Algorithm. The most important part. Friends and I are working hard to improve the accuracy and reduce the latency with tools like opencv. About a month later there will be a video to show the performance. I will also sent some prototype to my MIT and NUS friends to help improve the Algorithm performance.

6th:AR. It's not our priority. The sensor in 3d cam can support 1280*800 resolution. I will try to put some simple AR function into it.

When everything ready, I want to sent some free samples in this forum to those who interested in our invention. About 100pcs, that's the max amount I can afford. hopes to get more precise suggestions. (I think the cost of production is about 50-60 dollars, I will try to make it cheaper)

I was encouraged by Palmer lucky, priovr, control vr, omni and all the VR pioneers. Hopes to provide a cheap and easy way to get into the fantasy world. “To make it happen” is my motto.

In the coming few months I will updates our progress here.

There are so many misunderstandings in this new area, i’m not certain that whether I have any. So thanks for your suggestion and thanks for reading the whole thing. muhahha.
By the way, we can talk something else here, I love to chat , haha^_^
(ps:ignore my pool grammar :lol:
LOVE D&D(Dad and Dog, Dungeons & Dragons) L.L.B
Email:davy4job@163.com

Image
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by zalo »

Looks awesome, keep us updated!

Think about getting higher field of view camera lenses if you can. You'll want to be able to track your hand/gun just outside of what you can see, so it looks nice as it comes into your field of view.
User avatar
davyfirst
One Eyed Hopeful
Posts: 6
Joined: Tue Jun 17, 2014 1:37 am

Re: New here with my micro 3D Cam(motion capture or AR)

Post by davyfirst »

zalo wrote:Looks awesome, keep us updated!

Think about getting higher field of view camera lenses if you can. You'll want to be able to track your hand/gun just outside of what you can see, so it looks nice as it comes into your field of view.
HAHA, thanks!
Great idea. I was thinking that if the hand or gun out of the FOV, i will ignore them, make them disappear. Just like you said, it‘s not so consistent.
I think there are 2 solutions:
1, Higher FOV lenses. To maintain the same accuracy, I need better sensors provides higher resolution. I will try this solution in my 2nd or 3rd genaration 3d cam. At the same time, higher FOV means higher optical distortion, algorithm has its limite,I will do experiment. Or I might need to design a new lens, this is out of my capability. I will seek friends' help to solve this problem.
2, IMU unit. To add a simple IMU unit into the golve. The movement outside my view need low accuracy, so a cheep tiny IMU unit might solve this.

The suggestion will be put in my upgrade plan, thanks again zalo,so nice to know you. :lol:
LOVE D&D(Dad and Dog, Dungeons & Dragons) L.L.B
Email:davy4job@163.com

Image
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by zalo »

I've seen cameras go up to 140 degrees Field of View and still be able to counter optical distortion in algorithm, so that might be helpful.

Distortion can actually be good for Field of View because your resolution ends up concentrated around the edges so tracking feels consistent even at the edges (this is from personal experience).

An IMU is also a good idea. If you have an Oculus Rift, I made a demo that shows what IMU positional tracking looks like:
http://www.mtbs3d.com/phpbb/viewtopic.php?f=140&t=18749

Sony's PS Move has it where (if it loses sight of the glowing ball) it will revert to IMU positional tracking for a second or two before not moving anymore. So it stays accurate for a couple seconds.

Good luck with the project.
User avatar
davyfirst
One Eyed Hopeful
Posts: 6
Joined: Tue Jun 17, 2014 1:37 am

Re: New here with my micro 3D Cam(motion capture or AR)

Post by davyfirst »

zalo wrote:I've seen cameras go up to 140 degrees Field of View and still be able to counter optical distortion in algorithm, so that might be helpful.

Distortion can actually be good for Field of View because your resolution ends up concentrated around the edges so tracking feels consistent even at the edges (this is from personal experience).

An IMU is also a good idea. If you have an Oculus Rift, I made a demo that shows what IMU positional tracking looks like:
http://www.mtbs3d.com/phpbb/viewtopic.php?f=140&t=18749

Sony's PS Move has it where (if it loses sight of the glowing ball) it will revert to IMU positional tracking for a second or two before not moving anymore. So it stays accurate for a couple seconds.

Good luck with the project.
This morning I called my friend to make a new 150 FOV and higher resolution 3d cam for me. I'll do the experiment.
I have to put aside the IMU idea, the coordinate between 2 kinds of sensor system is more complex than I thought. I will try it later. And the SONY PS idea would help.
I like your demo, I have other plans for my immersive system, I might need it in the future.

Thank you very much. When I finish it, I hope to send you a set of free sample. I hope I can perfect it by then. Even if I failed, it still would be a perfect mrico 3D CAM, :P .
LOVE D&D(Dad and Dog, Dungeons & Dragons) L.L.B
Email:davy4job@163.com

Image
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by MSat »

Good stuff, davy

Instead of using standard camera modules which require a fair amount of bandwidth and external processing for blob tracking, perhaps you might want to look into the Pixart Multi-Object Tracking Sensor (http://www.pixart.com.tw/product_data.a ... ify2_id=33) which is essentially the same as the camera module used in the Wii Remote. It runs up to 200Hz, and since it performs blob tracking on-chip, you don't have to send entire frames to the computer for processing, just coordinate data of said blobs. They should be pretty cheap too, so you can probably use more than two of them for greater FoV.
User avatar
davyfirst
One Eyed Hopeful
Posts: 6
Joined: Tue Jun 17, 2014 1:37 am

Re: New here with my micro 3D Cam(motion capture or AR)

Post by davyfirst »

MSat wrote:Good stuff, davy

Instead of using standard camera modules which require a fair amount of bandwidth and external processing for blob tracking, perhaps you might want to look into the Pixart Multi-Object Tracking Sensor (http://www.pixart.com.tw/product_data.a ... ify2_id=33) which is essentially the same as the camera module used in the Wii Remote. It runs up to 200Hz, and since it performs blob tracking on-chip, you don't have to send entire frames to the computer for processing, just coordinate data of said blobs. They should be pretty cheap too, so you can probably use more than two of them for greater FoV.
WOW! This is a very important message, I was thinking that I might build a blob tracking DSP module myself to reduce the latency. But thanks to your information, I get a simple solution.
I have left a message on the PIXART‘s website, hopes to get their further information. I dont know whether their cmos module can do the AR job as well. Anyway I plan to cut the AR function someday.

Thanx brother Msat!
LOVE D&D(Dad and Dog, Dungeons & Dragons) L.L.B
Email:davy4job@163.com

Image
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by MSat »

Glad you found it of interest :)
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by zalo »

I wasn't terribly impressed with the resolution when I was using it for point tracking.
It says 1024x768 cursor res, but I believe it's doing 8x sub-pixel interpolation on a 128x96 sensor.

You'll probably want something more custom to take advantage of the resolution of your cameras.

This was the best I could get from a single Wii Remote:
https://www.youtube.com/watch?v=IAXQ5CKzo54
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by MSat »

Indeed, it's not actually a 1024x768 sensor, so it does do interpolation, but it might be sufficient for hand/object tracking with minimal processing requirements.
User avatar
davyfirst
One Eyed Hopeful
Posts: 6
Joined: Tue Jun 17, 2014 1:37 am

Re: New here with my micro 3D Cam(motion capture or AR)

Post by davyfirst »

I have called all of the Pixart offices in China, but no one answer!! There will be two versions of my 3d cam, one is pro version with high accuracy, the other is quick capture with low accuracy. I might need to do a lot of experiments to try different sensors.

Thanx bro. :D
LOVE D&D(Dad and Dog, Dungeons & Dragons) L.L.B
Email:davy4job@163.com

Image
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New here with my micro 3D Cam(motion capture or AR)

Post by MSat »

Cool! Keep us updated!
Post Reply

Return to “General VR/AR Discussion”