2d pictures -> 3d app

Post by **h-bar** » Fri Sep 19, 2008 8:03 pm

Hi all,

I've been working on an application during my spare time this week that let's you transform any 2d image into a 3d image.

It's in a working state now, but needs some optimization. If people are interested I will make it available when it's matured a little more.

Here's the last image I 3dorized,

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1705,

which I got off a google image search,

http://appuvista.com/images/forest_wall ... ure-la.jpg.

h-bar

Post by **Okta** » Fri Sep 19, 2008 9:35 pm

Checked out the pic with my prisms, not bad theres a bit of depth there. I assume it would be difficult to have specific pop out featuares rather than general depths?

Post by **h-bar** » Fri Sep 19, 2008 11:58 pm

As far as I can tell, you can get both. It's really just a matter of adjustments and time spent manipulating the images. I typically only like a small amount of pop-out, so it's how that image is set up.

h-bar

Post by **yuriythebest** » Sat Sep 20, 2008 12:02 am

nice image! At first I thought it was a real forest and even sent you a pm saying what a nice photo it was. But yeah, I did think that the trees were too "seamless" somehow, without depth gaps do to speak. Still, you fooled me

Congratz!

Post by **yuriythebest** » Sat Sep 20, 2008 12:24 am

sorry for hijacking this thread a lil: I made a simple experiment on speeding up taking stereophotos using a monoscopic camera- you just put it in video mode and move it quickly from left to right, then using the video editing app of your choice you extract the 2 frames. This lessens the problem of having something change like someone enter the picture on the second shot because with this technique it takes maybe 2-3 seconds to take the image. The downside is reduced resolution

Post by **nubie** » Sat Sep 20, 2008 3:44 am

h-bar wrote:Hi all,

I've been working on an application during my spare time this week that let's you transform any 2d image into a 3d image. It works by loading your image and attaching it to a mesh. You can then manipulate the mesh by translating areas toward and away from the screen. With 3d turned on, this allows you to manipulate the depth of the image.

It's in a working state now, but needs some optimization. If people are interested I will make it available when it's matured a little more.

Very cool, I remember seeing something like this that worked on a train, and a pathway onto the street with a tall hedge on each side, I wonder if I can find that program, It looked very good.

yuriythebest1 wrote:sorry for hijacking this thread a lil: I made a simple experiment on speeding up taking stereophotos using a monoscopic camera- you just put it in video mode and move it quickly from left to right, then using the video editing app of your choice you extract the 2 frames. This lessens the problem of having something change like someone enter the picture on the second shot because with this technique it takes maybe 2-3 seconds to take the image. The downside is reduced resolution

2 things:

1. try putting the camera on something that rotates, that way you can set a pivot point and make it more like your eyes. Alternatively you can make it on a straight track and curve it or put a bump that makes it turn inwards at on end, then you can get a repeatable/tripod mountable effect. I think I just left the camera on regular picture mode and slid it left or right to do the picture when I last tried. Maybe something with a mirror could work?

(off-topic farther, I made a 3D-izer invention to make the world around you more 3D

, just build a short periscope and hold it horizontally and look through it with one of your eyes, then move it until you focus both eyes on an object, you will have to move it for different objects to mimic normal eye convergence, bit it can make very distant objects 'pop' in real life. The eye on the periscope will get a bit strained so be sure to take a break or align on a stationary platform with you sitting down to reduce the amount that eye will try to move and line up the images. This can also funk up your depth perception a bit, so don't try to drive right after.)

2. There are 1080p video cameras

Post by **h-bar** » Sat Sep 20, 2008 11:27 am

Been playing around with the app and made a few more pictures...

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1711
http://www.mtbs3d.com/gallery/displayim ... ?pos=-1712
http://www.mtbs3d.com/gallery/displayim ... ?pos=-1713

If your're capable of adjusting the depth of these pictures, set it so that the closest part of the image is flush with the screen (little to no pop-out).

h-bar

Post by **sharky** » Sat Sep 20, 2008 11:49 am

hi! absolutely CONGRATS! its by far the best 2D to 3D conversion i have seen! really amazing! i love it! any chance to become beta tester? would love to test it on my own pics..

cu

igor

ps: i love the one with the garden.. so much depth.. amazing.

Post by **yuriythebest** » Sat Sep 20, 2008 12:02 pm

wow that is some extremely awesome (#@@- congratz man!!!! The green field totally blew me away- the sky is wrong but that totally adds to the style- it's not a stereo3D screenshot from a game and it's not a stereophoto either- it's like something it between. Like some insanely good 3d game or an offline 3d render. Amazing stuff- nice work man keep it up can't wait to see more. What is the name of your program?

Post by **h-bar** » Sat Sep 20, 2008 7:25 pm

Thanks sharky and yuri. I don't have a name for it; I'll wait til it's more complete to think about that. I'll work on it more during the week in my spare time and hopefully have a version I can share with everyone later in the week. It needs to be optimized more so that the mesh can be finer, and I'd like to add some more tools to manipulate the image.

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1714 <-good for popout

h-bar

Post by **Likay** » Sun Sep 21, 2008 3:46 am

I have to agree! The pictures looks superior! Love the picture of the rests from a supernova! I guess it takes a lot of effort for a picture like the goblin one to make all details straight. Still looks awesome!
How much time does it take to make this result with a 2d-picture with your program?

Post by **yuriythebest** » Sun Sep 21, 2008 4:53 am

also can you post a screenshot from your program? does it work by you painting a grayscale height map? Does it have real time iz3d preview?

Post by **sharky** » Sun Sep 21, 2008 5:30 am

your proggy is simply awsome! i would also like to see a screenshot of the interface..

also will it be freeware or for sale?

is it like z-brush o create the 3D effect?

Post by **h-bar** » Sun Sep 21, 2008 9:06 am

Actually, the screenshots that you've seen are pictures of the program. I just move the cursor offscreen and screencap it. You can even see the cursor in http://www.mtbs3d.com/gallery/displayim ... ?pos=-1712; two triangles, bottom right. There's really not too much to it, just a textured mesh, mouse input and some math.

The process involves loading in an image of any kind. The max size is limited by the max texture your videocard can support; mine does something close to 5000x5000 (8800 gts). After that you turn on 3d with the iz3d panel, and manipulate the image. So, you're working in 3d with your glasses on. You can use the normal iz3d controls for making adjustments to the overall depth and position. And yes, for complex pictures the overall detail will be limited by how much time you want to spend with it. So far, it's been about 10-15 minutes per image. With some better tools though, it'll be even easier and the results should be more satisfying.

I think the only requirements so far are that you can enable stereo in windowed mode (although adding fullscreen should be simple) and that you can screencap to side-by-side, which the iz3d driver does.

And it'll be free; it's a really simple program. If anything, it's a small thank you to mtbs3d for maintaining interest in s3d.

h-bar

Post by **sharky** » Sun Sep 21, 2008 9:56 am

just pay attention that nobody steals teh idea from you.. its a very powerfull tool.. would be sick if somebody steals it and owns money trough YOUR idea.

can you export jpgs or do you have to screencap tehm?

Post by **LukePC1** » Sun Sep 21, 2008 11:22 am

You are doing nice art here. Looks fine crosseyed.

I tried a program where you can draw the depth with gray scales, but somehow I wasn't able to get somothe grades.
Your program has always smooth steps to the next depth level. I like that. Is it because of the mesh?

The problem with 3D pictures from a moving camera movie is, that it is not sharp/not in focus perfectly. You can get it much better with just making 2 pictures and non moving objects.

Post by **yuriythebest** » Sun Sep 21, 2008 12:36 pm

sharky wrote:just pay attention that nobody steals teh idea from you.. its a very powerfull tool.. would be sick if somebody steals it and owns money trough YOUR idea.

*makes a mental note to steal his idea*

Post by **Likay** » Sun Sep 21, 2008 1:12 pm

h-bar wrote:Actually, the screenshots that you've seen are pictures of the program. I just move the cursor offscreen and screencap it. You can even see the cursor in http://www.mtbs3d.com/gallery/displayim ... ?pos=-1712; two triangles, bottom right. There's really not too much to it, just a textured mesh, mouse input and some math.

The process involves loading in an image of any kind. The max size is limited by the max texture your videocard can support; mine does something close to 5000x5000 (8800 gts). After that you turn on 3d with the iz3d panel, and manipulate the image. So, you're working in 3d with your glasses on. You can use the normal iz3d controls for making adjustments to the overall depth and position. And yes, for complex pictures the overall detail will be limited by how much time you want to spend with it. So far, it's been about 10-15 minutes per image. With some better tools though, it'll be even easier and the results should be more satisfying.

I think the only requirements so far are that you can enable stereo in windowed mode (although adding fullscreen should be simple) and that you can screencap to side-by-side, which the iz3d driver does.

And it'll be free; it's a really simple program. If anything, it's a small thank you to mtbs3d for maintaining interest in s3d.

h-bar

Inserting noob sentence here: "Since the whole texture is loaded into the graphicmemory it sounds like the gpu does the graphic calculation too?" *curious

Post by **h-bar** » Sun Sep 21, 2008 1:55 pm

No export, just a screencap. It's really simple this way and I'm not sure there would be any advantage to an export vs. the amount of time I have to work on the program.

LukePC1, yes because of the mesh. It's really like you've painted your image on a piece of cloth and you move areas of the cloth in and out to give the illusion that the image was 3d to begin with.

Likay, nope not using the gpu to do transformations; it's done on the cpu. This is my first directx application and have only spent about a week learning so far, so it uses a simple implementation. The mesh and texture are sent to the graphics card, but only after you transform them on the cpu. So far this is sufficient, and it seems that it will still be sufficient when I scale the rest of the innards to where I want them to be.

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1721

h-bar

Post by **yuriythebest** » Sun Sep 21, 2008 1:58 pm

wow h-bar you keep surprising me! awesome nuke! I really like this new direction of stereo3d art.

Post by **LukePC1** » Sun Sep 21, 2008 2:09 pm

Hm... I think I've got an Idea for the interface:
Left and right mouse buttions are for increasing and decreasing depth (decreasing depth is popout then). The longer you hold down the mouse button, the more depth do you get. And maybe add some sort of painter, so that you can change speed and size of the effected area...
Not shure, if it could work or would help things...

Other Idea: draw some fugure (like a line or a circle) and move that whole part. the sorroundings will be aligned smothly.

That would make it easy to make easy depth for a horizon like on some pictures. It would make the forest picture fast and easy, too...

Just some Ideas, but maybe you do it better and faster already

Post by **Likay** » Sun Sep 21, 2008 2:14 pm

Ok. Now i get it.

Don't have any programming skills regarding anything introduced later than -95...

Ps: Love the nuke too! Kinda makes me feel warm. o.O

Post by **ignatius** » Mon Sep 22, 2008 12:55 am

These images are bloody incredible. When i found out you've been taking 2D images and converting them into 3D, I was even more blown away. Keep it up -h-bar!

I would love to check your app out if you ever decide to release it as a beta or officially, it's looks very impressive.

Looking forward to seeing more of your 3D images, so far the nuclear blast is by far my favorite.

Cheers,
Varinder

Post by **cirk2** » Mon Sep 22, 2008 10:11 am

Thats great, the pictures realy blow me away.

I also like the style how you make the deph... Have you tryed it with the nvidia-vista drivers?

Post by **Welder** » Mon Sep 22, 2008 12:58 pm

Wow!
Amazing work!

What 3D device are you using to view these images?

I tested these on the iZ3D, and they are almost all completely perfect!

The first one you posted of the forest, is probably the deepest image I have seen

Very nice work!

And, does your program render the 3D, or do you export and view it in a separate application.

Post by **LukePC1** » Mon Sep 22, 2008 1:30 pm

Welder wrote:Wow!
Amazing work!

What 3D device are you using to view these images?

I tested these on the iZ3D, and they are almost all completely perfect!

The first one you posted of the forest, is probably the deepest image I have seen Very nice work!

And, does your program render the 3D, or do you export and view it in a separate application.

He makes a screenshot with IZ3D driver!
If you had read everything you'd know it... And he wrote the progrem on one, too

Post by **Welder** » Mon Sep 22, 2008 1:44 pm

Woops! Sorry about that

I have been reading, but all over the course of the week, so I forgot

Anyway, That's awesome! And let us know at iZ3D If you want us to beta test your software! I'd love to be able to check it out at our office!

Post by **h-bar** » Mon Sep 22, 2008 5:22 pm

Thanks for the comments everyone. The program is in much better shape. The performance is good, and the manipulation tool is behaving much better. There's just a couple of more things that needs to be worked out.

A couple more images: The first one I tried ( with varying degrees of success ) to give depth to the smaller objects in the image. The second one was just a quick 5 minute manipulation that looked nice.

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1723
http://www.mtbs3d.com/gallery/displayim ... ?pos=-1724

h-bar

Post by **Welder** » Mon Sep 22, 2008 5:54 pm

wow!!

The Lion looks AMAZING! (One of the best 3D images I have ever seen)

The other is a bit more difficult for me to see, the trees and such stand out, but the houses seem a bit doubled.

I guess it helps to make 3D images on our monitor, to unlock it's potential even further!

Us at iZ3D LLC Can't wait to try your program!

Post by **nubie** » Mon Sep 22, 2008 7:32 pm

I would like to know if this is related to the several other projects doing this, like this one: http://www.youtube.com/watch?v=VuoljANz4EA

I know there was some software somewhere that does this, I just need to go looking. Pretty cool though, now if only there was a S3D GUI/desktop

(I did play with a terrible one that you played like a FPS, but it wasn't very good, this was maybe 4-6 years ago.)

Check out all the relevant links here:

http://www.google.com/search?q=3d+from+2d

I am not trying to downplay the OP awesome accomplishment, but I highly doubt that he can patent it.

I have seen the 3D from 2d projects a while ago, they are very neat I will grant you.

http://www.3d-conversion.com/examcr.html

http://www.jim3dlong.com/

http://www.techcrunch.com/2008/02/27/ma ... -3d-model/

It looks like stanford will do this for free

http://make3d.stanford.edu/

As will Fotowoosh: http://www.techcrunch.com/2007/04/15/fo ... -3d-image/

Post by **h-bar** » Mon Sep 22, 2008 9:29 pm

Thanks for digging up those links nubie, I remember coming across the stanford one awhile back. I think they've patented the automation of the process, which is really the novel part of what they do. Pasting an image to a mesh and then distorting the mesh isn't really anything new.

Although, looking at some of the example galleries in the links you provided, it doesn't seem like that's what those people are doing. One says it's done by pixel shifting, which I started with in some image-editing software, but is a real PITA since you can't see what you're doing in real-time. The other site ( which charges 400euros for a 1000x1000 image! ) looks like a careful compositing job; he might be pixel shifting too, to add smoother depth change.

This app lets you work in 3d, which is pretty fun, and produces 3d images from pictures you can find anywhere. It's also really easy to do. I first tried doing this in some 3d modelling software, but most everything uses OGL (if they use a graphics API at all). Consequently, the alternative was either wait for iz3d to work something out with zbrush, wait for OGL support, or figure out a way to do it myself. It turns out it wasn't that hard to do, except maybe a steep learning curve with directx. I'm not trying to patent this and I hope that it's not gonna be a problem making it available to everyone. The only thing I might do is set up a donation link.

It's almost done too. I'm adding some ways to maipulate at a finer detail level so features, like those houses in one of the recent pictures that was posted, look better.

h-bar

Post by **shonofear** » Tue Sep 23, 2008 12:28 am

G'day
well you really are raising the (h-)bar....

um, yea, is there a way to be able to use your program and the IZ3D 1.09 beta driver in Interlaced mode?
Cause it should/maybe/hopefully work with my Shutters/CRT...

I dont have a clue about Code at all,
but I have been looking for a real time live based image creator in true Stereo 3d
so congrats on achieving this

so atm its working with the IZ3d monitor and drivers yea?

Post by **sharky** » Tue Sep 23, 2008 5:08 am

im not sure but i think that this prog is a "simple" one screen aplication using direct-x so it should work with every driver that supports direct x and windowed apps. and that can make printscreens. maybe it works with nvidia drivers too? the program itself doesnt contain a 3D output afaik. the 3D is done by the drivers so teh proggy itself shouldn't be proprietary.. am i right?

Post by **h-bar** » Tue Sep 23, 2008 7:02 am

Sharky is right. Since it's just a simple directx window, any set-up you have that allows stereo for windowed directx should work. Of course, I can only vouch for the iz3d monitor and their latest drivers, but there is nothing fancy going on behind the scenes that should restrict it for other drivers/display.

Post by **LukePC1** » Tue Sep 23, 2008 12:20 pm

h-bar wrote:Sharky is right. Since it's just a simple directx window, any set-up you have that allows stereo for windowed directx should work. Of course, I can only vouch for the iz3d monitor and their latest drivers, but there is nothing fancy going on behind the scenes that should restrict it for other drivers/display.

would you implement a fullscreen function, too?
I try to do everything at full screen and it's necessary for NV drivers...
but maybe you want to push IZ3D. I could understand it

Post by **h-bar** » Tue Sep 23, 2008 1:16 pm

I'll take a stab at fullscreen tonight. I don't think it's a problem, just have to make sure it behaves well with how the program works.

Post by **shonofear** » Tue Sep 23, 2008 1:35 pm

Thats great,
Thanks for clarifying and hope it works with NV full screen

Post by **nubie** » Tue Sep 23, 2008 2:53 pm

h-bar wrote:Thanks for digging up those links nubie, I remember coming across the stanford one awhile back. I think they've patented the automation of the process, which is really the novel part of what they do. Pasting an image to a mesh and then distorting the mesh isn't really anything new.

. . .

This app lets you work in 3d, which is pretty fun, and produces 3d images from pictures you can find anywhere. It's also really easy to do. I first tried doing this in some 3d modelling software, but most everything uses OGL (if they use a graphics API at all). Consequently, the alternative was either wait for iz3d to work something out with zbrush, wait for OGL support, or figure out a way to do it myself. It turns out it wasn't that hard to do, except maybe a steep learning curve with directx. I'm not trying to patent this and I hope that it's not gonna be a problem making it available to everyone. The only thing I might do is set up a donation link.

I dunno, I think that pixel-shifting may be what some of them are doing, notably the one with a VRML web app to show you the "3D", it looks poor enough to be a plain pixel manipulation.

Cool app though, how does a mesh come out of "shadow" maps? I am sure I don't understand how it works.

Or does it generate a few "tries" and then you pick the best and adjust it by hand?

Post by **h-bar** » Tue Sep 23, 2008 10:42 pm

It's much simpler than that nubie. Think of it as having an image on a sheet of rubber; then all you do is stretch the rubber to give it depth.

http://www.mtbs3d.com/gallery/displayim ... ?pos=-1725
http://www.mtbs3d.com/gallery/displayim ... ?pos=-1726

All the tool work is done now. It just needs a little fine tuning. Now it's capable of zooming in and working in much greater detail with the image.

Post by **yuriythebest** » Tue Sep 23, 2008 11:24 pm

wow that waterfall i amazing!