Building a Photogrammetry scanner. Namielus needs help

[Namielus]

I just bought 40 dslr's.
The plan is to set up a photogrammetry face scanner.
40 should be enough for the face only.

These cameras can be triggered with a 2.5 stereo jack, where you have ground+half press and ground+full press.
All the cameras can be connected to a computer and triggered with USB, but there is a delay.
So the USB will only be used for file transfers, and the triggering will be done with a manual trigger system or a usb relay.

What I need to do next is figure out how to mount the cameras in the cheapest and most optimal way.
I cannot afford getting a tripod for each camera, I might be better off building something out of PVC or similar.

I need 40 2.5mm stereo jacks, 40 camera mounts and a shitload of help to set this up correctly.

If I get this done, I promise to do everything I can to take these babies on tour to vr-meetups and
have as many of us able to scan themselves and have their own avatar as possible.

In any case, everyone on this forum is welcome to come over to visit.

[GeraldT]

wow ... you have no idea how jealous this makes me!!
I think that using GoPros would have been the easier way to do it, but hell that is certainly gonna be a very interesting setup.

Please keep us posted.

[Namielus]

It seems dslrs are the standard cams for this purpose, and it has to do with the remote socket triggering and low shutter lag when triggered with a remote socket release.

I hope so !!! Gopros would be so much cheaper. But the image quality and distortion is also worse.

[Oneironaut]

I just bought 40 dslr's.

Like a BOSS!

About time someone did something like this.

[Pingles]

I used to have a lot of time but no money.

Right now I have plenty of money but no time.

Thank goodness I don't have plenty of time AND plenty of money or I'd be getting into all sorts of trouble like you are.

Good luck with your crazy project.

My advice: Figure out the spacing you need first. You might be able to get away with just buying a bunch of simple coatracks and nailing arms with mounts onto each.

[GeraldT]

It seems dslrs are the standard cams for this purpose, and it has to do with the remote socket triggering and low shutter lag when triggered with a remote socket release.

I hope so !!! Gopros would be so much cheaper. But the image quality and distortion is also worse.

Yeah, but you can do some cool stuff with them and mounting 40 would be much easier (plus they come with a fixed lense). But your setup will be higher quality I agree.

[Marulu]

If I get this done, I promise to do everything I can to take these babies on tour to vr-meetups and
have as many of us able to scan themselves and have their own avatar as possible.

As a fellow European I really hope you are going to be attending Gamescom with this setup.
I am going to attend Gamescom on the press only day.

[Namielus]

The problem would be time, I need time to set this up and time to finish my kickstarter.

The reason I got these cams right now is that there was a one time offer that expired this weekend and they are going out of production.
I would buy 40 more if I had the funds.

[GeraldT]

so how is this setup gonna be build up?

one ring 360° and a few added to make shots from below and above?

[Namielus]

My understanding is that its best to have the cameras set in pairs.
All cameras facing the persons face.


I am afraid 40 cams is too low an amount for capturing the entire body unless I can rotate the rig.

[GeraldT]

wow ... so just the face and not the full head?
what software do you plan to use for this? makes me curious, is there some place to read up on that?

[Namielus]

Breeze Systems Software to remote control the cameras
Agisoft Photoscan for the image analysis/photogrammetry

[GeraldT]

I read a bit into the Agisoft Photoscan and it is quite interesting. I did something similar for Panorama 360° photography and this is like the stitching process of that with 3D data generation. Sounds really cool, but I think 40 DSLR for a 3D facial reconstruction might be a bit much, at least you could do 360 with such a setup.
Then you might want to check on the lenses - I don't know what the bundle lenses are, but I guess using portrait lenses would give you the best results. And you need good lighting too.

The best result could be achieved with a light tent where you cut out holes for the lenses. Would make the stitching easier and you have a good way of creating even light.

[Namielus]

Believe me I did my research.
Infinite Realities are using 120 dslr cameras for full body scans I belive.
As for even light I will probably make a tent. But I need to figure out how far I can go with 40 cams.

Face, head, torso or full body.


Check out this footage of 3D-scans done with 70 cams
http://www.youtube.com/watch?v=5ULCDPKGF10

[cybereality]

Sounds Awesome! Good luck man.

[GeraldT]

Believe me I did my research.
Infinite Realities are using 120 dslr cameras for full body scans I belive.
As for even light I will probably make a tent. But I need to figure out how far I can go with 40 cams.

Face, head, torso or full body.


Check out this footage of 3D-scans done with 70 cams
http://www.youtube.com/watch?v=5ULCDPKGF10

I wasn't trying to imply otherwise, just wanting to help mate.

They are awesome, so if you can do full body with 70, I think you can do more than just the face with 40. At the very least you should do head with neck. For VR & gaming that would be the most interesting part anyway.
And you would have relatively easy to clean objects ... I wouldn't want to have to clean out all those not properly detected crinkles from clothes. And it would be a good base to start with, much easier to light than the entire torso.

[nateight]

Wow. This is exactly what I'd be doing right now if I had the funds, but as it stands I couldn't even afford a plane ticket to Norway to pitch in. Best I can do is offer my thoughts:

Even as an enthusiastic Omagles supporter, I strongly suspect a PVC cage for these cameras would be a source of constant frustration. The primary problem I expect you'll encounter with any kind of rig is that even a gnat's breath will jostle the cameras around subtly, and you'll get 160 mismatched seams. Perhaps this Agisoft PhotoScan thing can overcome this problem, but I'm always distrustful of software that claims it can perform feats of magic with zero user effort. The worst case scenario I can imagine is that even the motion of 40 shutters operating simultaneously would move things around enough that the very action of taking a calibration shot could slightly uncalibrate the rig; this seems unlikely, but PVC can get pretty wobbly over several feet and under sufficient weight. I also don't see how you could reliably attach tripod mounts to a PVC tube, hot glue clearly ain't enough. I think I would want to design something out of square tubing, build it in such a way that the rig broke apart into several welded and reinforced yet easily-transportable sections while still being sturdy when cinched together, and find a way to sneak the front lighting alongside the cameras. Photographic lighting has historically been extremely hot and power hungry stuff; has "true white" LED lighting finally started appearing in the photographic market, or are we still that eternal "five years" from production? This still leaves the method for attaching the cameras to the rig unsolved, but having steel to weld onto would make me hopeful about finding some suitable tripod mounts; some sort of cage mount built around each camera would be better (and probably cheaper, not accounting for the complex design and construction of 40 of them), and disassembling the cameras to remove any unnecessary parts might be ideal (but good luck with that).

A good place to start would be testing how well Agisoft PhotoScan deals with small camera misalignments. Find a good test object, image it with a circle or half-circle of five or ten cameras, momentarily jump up and down in the vicinity of the table this is all sitting on, and image the test object again. If the outputs of both are similarly error-free, try a third test in which you manually jostle each camera a minuscule amount. If all three of these tests return fairly good results, Omagles might be enough. If, however, Agisoft PhotoScan appears to need perfectly aligned cameras, you might have a problem. Can you get rid of the misalignment errors by running a calibration routine between the three jostle tests? Knowing how much tolerance you have to play with is always crucial to producing a well engineered object.

[gjpetch]

Super jealous!
Agisof Photoscan definitely doesn't need perfectly aligned cameras. Give PVC pipes a try, but I tend to think that biting the bullet and buying 10 or so C-stands will be less painful.
Good luck with it! If you're also wanting to capture diffuse colour then I recommend playing around with polarising filters on your lights and camera, to filter out all the specular. The downside is that it's cutting out some of the light, so it'll make it slightly harder to get good exposure.

[therage79]

Alas I cannot add to the technical conundrum under discussion, but just wanted to add my support for what you are doing here Namielus.

Is it wrong that I have pinned all my 'Ready Player One' fantasizations on the success of this development - No pressure!

I really think we are on the cusp of a workable realtime self-image avatar driven believable 3d environment...and that is sooo cool!

'Lykke til'

[virror]

How much harder would it be to make some kind of rotating rig?
Full body capture would be so cool, i would visit you in a heart beat if i could have myself scanned in : D

[Namielus]

If I am to make a rotating rig, I have to make it so that one camera cannot see the other.
Some kind of half dome, and that means I would have to rotate pretty far risking movement of the captured person.

The reason one camera cannot see the other when moving around a subject, is that it would involve a fixed point in the photos moving relatively to the subject and this would confuse the software.

I might be able to scan a whole person, just using 40 cams tho

[Infinite]

No offense. Really ...but what has this got to do with the Rift? There are lots of other forums out there where you can discuss this and ask questions. Here is a good place http://www.agisoft.ru/forum/index.php you can learn allot there.

Canon's IMHO really aren't the best choice for photogrammetry, especially D1100's and I've bought a few Canons in my time. I'm finding Nikon are much more powerful, faster more reliable sync at faster speeds.

FYI you can't trigger sync the cameras by software because you can't synchronize those DSLR's mechanically. There will always be a ms delay between each shutter.

[nateight]

rotating rig?

Generally speaking, isn't rotating the capture rig foolhardy when you can much more easily rotate the subject? If Agisof Photoscan actually can perform a bit of image stitching magic, wouldn't uniting a face model and a back-of-the-head model be a somewhat trivial operation? "Stand on the X and look here" is effectively the same task as "Stand on the X and look here, *click*, now turn around and look there". I'm not even sure full-body is necessary - if you can approximate skin tone from the face capture, you can measure height independently and choose from an array of body types to fill out the avatar. We're dealing with approximations regardless - wouldn't any software sophisticated enough to incorporate a full body model be plenty sophisticated to offer a range of body models that could obviate the need for complete scanning?

[Namielus]

No offense. Really ...but what has this got to do with the Rift? There are lots of other forums out there where you can discuss this and ask questions. Here is a good place http://www.agisoft.ru/forum/index.php you can learn allot there.

Canon's IMHO really aren't the best choice for photogrammetry, especially D1100's and I've bought a few Canons in my time. I'm finding Nikon are much more powerful, faster more reliable sync at faster speeds.

FYI you can't trigger sync the cameras by software because you can't synchronize those DSLR's mechanically. There will always be a ms delay between each shutter.

Im asking here because I know a lot of people and it will mainly be used to produce Oculus Rift content. But you are right, its not directly related.
Thanks for the link.

[Infinite]

No offense. Really ...but what has this got to do with the Rift? There are lots of other forums out there where you can discuss this and ask questions. Here is a good place http://www.agisoft.ru/forum/index.php you can learn allot there.

Canon's IMHO really aren't the best choice for photogrammetry, especially D1100's and I've bought a few Canons in my time. I'm finding Nikon are much more powerful, faster more reliable sync at faster speeds.

FYI you can't trigger sync the cameras by software because you can't synchronize those DSLR's mechanically. There will always be a ms delay between each shutter.

Im asking here because I know a lot of people and it will mainly be used to produce Oculus Rift content. But you are right, its not directly related.
Thanks for the link.

OK. Well if it's allowed to be discussed here I can offer some advice, you've started down the rabbit hole now. So there is no turning back! It will consume your life, believe me.

Look out for SmartShooter (trigger software, cheaper) and try to get someone to build you some opto-isolated trigger relays for 3.5mm ports. Also Pocketwizard II/III and Multimax are ideal for triggering remotely. Get some Paul C. Buff Einstein flash lights.

[Namielus]

Yes I plan to trigger using the remote socket release. I think its 2.5mm stereo plug for these cameras but the principle should be the same.

For testing, can I make a passive trigger circuit that just connects half press and full press to ground on those wires?
Or do I have to make a more advanced circuit right away?

[virror]

rotating rig?

Generally speaking, isn't rotating the capture rig foolhardy when you can much more easily rotate the subject? If Agisof Photoscan actually can perform a bit of image stitching magic, wouldn't uniting a face model and a back-of-the-head model be a somewhat trivial operation? "Stand on the X and look here" is effectively the same task as "Stand on the X and look here, *click*, now turn around and look there". I'm not even sure full-body is necessary - if you can approximate skin tone from the face capture, you can measure height independently and choose from an array of body types to fill out the avatar. We're dealing with approximations regardless - wouldn't any software sophisticated enough to incorporate a full body model be plenty sophisticated to offer a range of body models that could obviate the need for complete scanning?

That was a typo from my side, of course i meant to write rotating the subject. But, rotating the subject would have higher risk of injecting faults in the capture because of subject changing position slightly when turning?

[Namielus]

I think we can pretty much forget about rotating the subject when its a living creature.
Too much movement.

[Infinite]

Yes I plan to trigger using the remote socket release. I think its 2.5mm stereo plug for these cameras but the principle should be the same.

For testing, can I make a passive trigger circuit that just connects half press and full press to ground on those wires?
Or do I have to make a more advanced circuit right away?

I'm not sure then how you will trigger the flash lights with half press and full press. You need to sync by light, the best way using a delay signal on the lights. PW's are 3.5mm and yeah Canons are 2.5mm, something to bare in mind, depending how you trigger them. I mentioned opto-isolated, otherwise you will get feedback loops on the connections, say if you power off one camera or need to unplug one, they will all fire. Which is wasteful and adds to depreciation.

I think we can pretty much forget about rotating the subject when its a living creature.
Too much movement.

Yeah not recommended. Single shot is best to minimize subject movement. Even when capturing video at 24-48 frames per second, humans move an incredible amount!

[Marcel]

Cool project! I would love to experiment with something like this if I had the time!

I would build something with Aluminium 'Bosch' profiles. They are priced around 6 euro per meter and are incredibly stiff (you don't want a setup that wobbles leading to blurry pictures). You can buy corner connectors, plastic end caps, and other good stuff. So it would be pretty easy to build a nice construction that looks professional as well. I would either have it cut to length by a shop (or invest in mitre saw that can cut aluminium), if you cut it by hand it will take ages and look messy.

Here is an example (Dutch webstore) http://stappenmotor.nl/Steppermotor/alu ... %20cnc.htm
Search for "bosch profiles" on Google Images to get an idea how to construct things.

To attach ballheads/cameras to the profiles you will probably need to fabricate a small metal plate (attach ballhead to plate, attach plate to profile). The advantage of the profiles is that you can slide stuff up and down, so it is easy to make small adjustments.

[Infinite]

Something to make sure this thread stays on track to the Rift and also related to this thread.



Marmoset Toolbag's Engineer/Author Jeff Russell looks to be adding Oculus Rift support to Marmoset Toolbag 2.0 http://www.marmoset.co/

http://marmosetco.tumblr.com/post/50840184370]

This is great for character and environment artists who wish to view 3D scans, 3D characters, game objects or levels in a cheap, easy to use 3D viewer. Marmoset is incredibly powerful and not over bloated compared to UDK, CryEngine or Unity becuase it's just for viewing content. It also has advanced shaders, full screen effects and IBL (http://en.wikipedia.org/wiki/Image-based_lighting).

It would be very easy to be able to distribute content using Marmoset for other Rift users to enjoy

[Vamplifire]

hmm maybe paint them? Do a whole green screen and paint them to reflect the color

[Vamplifire]

also I posted this on a different forum and I know it’s not to scale of what you are doing but this is really interesting

http://www.youtube.com/watch?v=VoEBJfI8kDw

[Rickardo69]

This really is getting interesting and you certainly are determine Namielus
I cant help with any of the techical stuff but sure would like to contribute towards it
A friend of mine owns a small metal fabracting company , If you could get a drawing/scetch of the rig you have in mind i would be more than happy to pay him a small fee to
fabricate it and send it over to you mate along with 40 2.5mm stereo jacks too .

[Benjimoron]

This sounds super cool! I wonder how much detail you need though? Especially with multiplayer, there's only so much data you can transmit around without causing yourself problems.

Also, there's only 38 cameras in the picture, assuming you used one of the cameras to take the picture, where's the other one gone?!?

[Namielus]

Here is the design for the circuit board for optotriggers , 1 input 11 outputs and programmable delay wihin a few microseconds using arduino nano.
I can calibrate and sync the cameras fairly well and if I want to I can use it for a matrix style bullet time shoot in the feature.

For now tho, its going to do its job as a trigger for the photogrammetry part.

Thanks to my friends at hackheim for helping me design the triggering system.

[nanicoar]

I wonder if a fixed-focus lens would give you more resolution than a zoom lens, and if removing the anti-aliasing filter from the sensor would do the same again. With multiple exposures from different angles any patterns from aliasing should blend out.

I'm also very curious if DSLRs are really needed for this since you presumably control the lighting on the object... Capturing RAW is definitely nice, but if dynamic range is what you need then film is still unbeaten. - You would also get an approximate 24Mpx out of 35mm film, and 48Mpx out of medium format. For preview just one or two digital shots of the same exposure should suffice.


ED:
Forgot to say,

[Namielus]

I assume you mean a prime lens a.k.a fixed focal length? I agree on that. If my budget allows for it when this gets rolling, I will get better cameras and better lenses.
As for the dynamic range you would gain from film cameras, it would be a nightmare to develop 40+ films and it still wouldnt work as the digital files have a lot of embedded
exif information the software needs like exact timings, settings and not to mention distance to subject in the exif.

So analog film is out of the question unfortunately.

[nanicoar]

Ah, that's making more sense. - Development and scanning could probably be done with a second-hand automatic device, but like you say DoF and focal length would be lost unless recorded manually.

And prime lens is what I meant, yeah... Are you shooting portrait or landscape?

[geekmaster]

Canon cameras support Magic Lantern firmware, booted from the SD card. Magic Lantern adds many features, including in-camera Lua scripting, and a CHDK firmware add-on.

CHDK supports remote control of multiple cameras over a single shared (modified) USB cable, including synchronizing multiple cameras:

Multi-Camera Synchronization
... A useful application of CHDK is the ability to connect two or more cameras together via a common USB cable so that their operations can be synchronized. ... To achieve better synchronization when taking an actual photograph, CHDK has the ability to stop the picture taking process immediately before the image is actually captured. This allows CHDK on several cameras to focus, adjust exposure, setup the flash and then wait for a shared signal to complete the shot (i.e. sync). ... To more precisely tune the synchronization between cameras, CHDK allows a precise fixed delay to be added to each camera to adjust for differences in the time each camera takes to finish shooting a picture. The sync delay value is given in units of 0.1 mSec. ... (Developer's note : the USB remote code includes optional code that can be used to calibrate the sync delay units precisely for each camera.)

For a simple remote shutter release switch, it can be as simple as this:



The sync delay is a multiple of 100 microseconds (0.1 mSec) though, so this software-only method is not as precise as your hardware device.

This custom camera firmware does not harm your camera. You can always insert a "normal" SD card and revert to the standard Canon firmware.

I like the live remote view over USB, and the ability to instantly upload and remote-execute Lua scripts over the USB cable. Lua has fairly complete control over the camera, including forcing button presses, and so much more.

But for multiple camera synchronization, you just need to toggle the USB power wires. There is also support for synchronized zoom of multiple cameras. It may be worthy of investigation for this application.