Tutorial: Generating quick game geometry from simple photographs!

[Piggyson]

Wanted to share a quick pipeline process I found recently using Zbrush and Autodesk's Photofly. Using a digital camera, Zbrush and Photofly you can create quick highly detailed game geometry extreamly fast. The first images show some of the test examples I have created. The youtube links are to the HD versions of the tutorial Videos.

stump1.jpg (http://javascript%3Cb%3E%3C/b%3E:zb_insimg%28%27256116%27,%27stump1.jpg%27,1,0 %29)
stump2.jpg (http://javascript%3Cb%3E%3C/b%3E:zb_insimg%28%27256117%27,%27stump2.jpg%27,1,0 %29)
wall1.jpg (http://javascript%3Cb%3E%3C/b%3E:zb_insimg%28%27256118%27,%27wall1.jpg%27,1,0% 29)
headexample.jpg (http://javascript%3Cb%3E%3C/b%3E:zb_insimg%28%27256119%27,%27headexample.jpg%2 7,1,0%29)

HD Part1: http://www.youtube.com/watch?v=QyL8g1tlMMY
HD Part2: http://www.youtube.com/watch?v=vm9MJKWiyuU
HD Part3: http://www.youtube.com/watch?v=NPuuEIkGKUw
HD Part4: http://www.youtube.com/watch?v=cJdwefzdobU

Happy Zbrushing!!

-Joseph

[nerveink]

This is incredible. the possibilities are endless!!!!

:tu::tu::tu::tu::tu:

[Sparky3d]

:D thank you so much :D

[Clintus Maximus]

Wow that was really cool!

[Nancyan]

Unbelievably great tutorial! Thanks so much!!!!

[hendrik F]

truly awesome! thanks for sharing :tu:

[Maya4fun]

Great too see some new technique for production, thank you Joseph.

Emmanuel.

[Etcher]

Great stuff Joseph, thanks for sharing :cool: :tu: !

[mushmouth]

wow. great tutorial. I'm already trying my own modeling experiments....thanks!:D

[pabgo]

:tu:.

[jamesT]

wait, what :qu:

[machado]

WICKED... Thanx

[Altermind]

brilliant tutorial!.... now all we need are normal map generation tips!

[mix_mash]

Great tutorial BUT you should have explained more about the camera you used, the camera settings and camera techniques you used to take the photos otherwise it's hard to know if the technique can be done with a normal camera and settings or not. Also, an idea about how many photos should be taken for a good level of reconstruction would be good (at least a minimum amount).

Paul
(mix_mash)

[Nancyan]

mix_mash, your post wasn't directed to me, BUT...

I think Piggyson, did his bit here.

It took me exactly 30 seconds to go to the Photofly website and be directed to this youtube tutorial: http://www.youtube.com/watch?v=8YNrQA6eofI

Personally I think producing 4 videos in both regular and HD versions is enough for one free (and EXCELLENT) tutorial from any forum member here.

[Grfx]

amazing tutorial, would you mind explaining how you scaned the face, that would be realy nice. thnx

[Piggyson]

@mix_mash: The process can be done with any normal cameras. I didn't use any special settings I just took a camera and started walking in a circle taking pictures. As long as the camera produces EXIF data it will work.

A minimum of 2 pictures are required to generate the 3d effect (same process that was used in the old viewmasters back in the day, and the current 3D craze with TV's.) You just need to take enough pictures for the program to stitch together the subject you want to capture (if you have a camera that does panoramic mode that would help in making sure you don't miss any areas.)

@Grfx: The stump took longer to capture then the face. The face was captured in the shade so that the lighting was even. Then I just rotated around the subject exactly like the stump and took 11 pictures (tests I have done are never exact in the amount of pictures. For a head it's turned out to be 11-12 photos, it all depends on how many steps I take around the subject.)

After it is captured you should get a mesh that is decent (first raw model in the image) and a excellent diffuse texture. Applying the diffuse as a bump will help increase the quality of the raw model (I have been quickly dumping the diffuse to photoshop equalizing the channels then seperating the blue channel to use as a bump.) The rest of the process was just quick Zbrush sorcery using masking, retopology, projections, and simple sculpting. Creating a tutorial to cover the head process I use is on my list of to do's.

Another quick example of a head 12 pictures with a Canon 7D were used. (just the raw data from photofly)
headexample2.jpg (http://javascript%3Cb%3E%3C/b%3E:zb_insimg%28%27256298%27,%27headexample2.jpg% 27,1,0%29)



-Joseph

[killfrog58]

wow this is awesome! very useful thanks!

[joebount]

Really cool indeed !
Have you tried puiting the subject on a rotating plateform ? I have the feeling that would not really work with this kind of thechnic, am I wrong ?

[Sebcesoir]

Thats really great, its the a plan to make a stand alone version or its imposible due to calcualtion?
Standalone would be great ;)

[Maya4fun]

Agree with Sebastien.

Emmanuel.

[InDoe]

Joseph: thx a lot man, awsome technique, ty ty ty:D

I tried that, rotating plateform doesn't work :cry:, photo scene editor calculate the snaps in the same origin, so it comes out as a flat mesh

[Sparky3d]

I have tried this multiple times in the last 24hrs and I cant get it do work,

I have tried several different head references, and its just coming out flat and there is absolutely nothing wrong with the images

[Piggyson]

@Joebount: Yeah I too was hoping to be be able to just setup one camera and spin a subject while taking pictures. In all the tests I have tried the camera ends up picking something that is a constant in the scene and renders all the camera positions right on top of each other. Now you can manually set all the registration points across the images for the alignments but it's a big pain. Also in testing the lazy susan effect I have found that even if I get the cameras to come in correctly the final ldiffuse texture has a lot of lighting banding in it. I have yet to find a way to get the lighting to be completely constant and totally even around a rotating model. So far the best and quickest results I have had are from myself rotating around a subject who is standing in the shade.

@Sebcesoir, @Maya4fun: No clue if they are planning to release a standalone or not. The program is going to be free using the cloud system till December then who knows what Autodesk will do then heh.

Another program to try that is standalone is Agisoft Photoscan (http://www.agisoft.ru/products/photoscan/) The program works pretty well and gives a better quality mesh but the diffuse texture it spits out is lacking. The Photoscan also takes about 10x as long to process a scene then the Autodesk one does (The dedicated servers on the autodesk cloud are extremely fast, however after the whole dropbox security fiasco no telling how safe your images/scans really are when dealing with a cloud server in a production pipeline.)

@Sparky3d: If you can upload your images somewhere i'd be happy to take a look at them and see if I can find a solution. In testing I have only came across flat geometry once and it was a result of over lighting the subject and high specular reflections across the nose (might be something else to look for in your images?)

[mix_mash]

@mix_mash: The process can be done with any normal cameras. I didn't use any special settings I just took a camera and started walking in a circle taking pictures. As long as the camera produces EXIF data it will work.

A minimum of 2 pictures are required to generate the 3d effect (same process that was used in the old viewmasters back in the day, and the current 3D craze with TV's.) You just need to take enough pictures for the program to stitch together the subject you want to capture (if you have a camera that does panoramic mode that would help in making sure you don't miss any areas.) Sorry, I hope you didn't think I was taking a pot shot or anything. It's just that when you make a tutorial it's important that you cover all the angles otherwise the viewer can either feel lost or that they are missing information.

But, to reiterate, they are great tutorials.

Cheers,
Paul
(mix_mash)

[GX@W]

Just a random idea, would this works with macro shots of insects?
Superb tutorial btw.

[cw06]

Great discovery.thank you. :D

[sculptor.zb]

Wow!! great. Please make a full tutorial.. Thanks!! ;) :tu:

[Nancyan]

Sorry if this is slightly off-topic here. (here@mix_mash)

@mix_mash. I believe its I who owe YOU an apology. I read your original post in haste and in the wrong frame of mind. After the fact I have notice you also have provided some very useful and free tutorials.

:o <- me

I was defending the OP when no defense was warrented.

Thank you, also for your contributions that have helped us novices so much.

[mix_mash]

Sorry if this is slightly off-topic here.

@mix_mash. I believe its I who owe YOU an apology. I read your original post in haste and in the wrong frame of mind. After the fact I have notice you also have provided some very useful and free tutorials.
Don't worry about it. I can sometimes come off as brusque or arrogant with my comments.

I also misread other peoples' stuff sometimes, too, so don't worry yourself. No offense taken.

Cheers,
Paul
(mix_mash)

[canndis]

Nice results,
I was trying out photofly last week and was really impressed with the results.

Images here

http://www.facebook.com/media/set/?set=a.10150287792532049.379485.537727048&l=0b1b24d56f

[sebastien839]

Thanks for the great tuts Joseph, and for taking the time to explain.
What I'm really curious about is how you take the head scan and clean up the mesh so well. I have no idea how you might do that in Zbrush! Any pointers would be most welcome, if you can ever find the time..
Thanks again, Sebastien

[LeoAMD]

Hey man this is great. Thanks so much for sharing this info with us. I've been mucking around with this all day. I just wanted to ask how you are generating the bump map on your meshes?
Also I think Ive found a way around the issue with the turning chair/lazy suzan approach. I used a swivle chair against an all white bacground and then used the magic wand tool in photoshop to quickly remove the bacground leaving it black. this seems to work quite well

[romm]

This is great !

Here is another option that's been around for a while

http://www.strata.com/products/strata_3d_cx_suite/strata_foto_3d_cx/


It's got a free 30 day trial so some comparisons would be interesting.
It works on [mac] and pc

I'm trail downloading now

[The Namek]

we are considering to use it as a method if displaying objects before and after restoration. We've done a test with a polychrome wooden sculpture of about 28 cm's. 78 photo's



the underlying model is not good enough to be acceptable to use the model as an example for virtual restoration to show possibilities to clients. The photo textures cover up a lot and fake a lot of detail ofcourse. But interms of visualization, there's definitely possibilities.



Example 360 turntable in Zbrush of the photofly2 model (the white on the hand is tape, the hand is badly damaged):

http://www.youtube.com/watch?v=vMrKNuaLvzY***65279;




We took photo's all around it at a normal position and once more all around it from a higher point and at a downwards angle to get the top of the head. every photo consisted out of about 50-60% the object. We didn't take any closeup images and maybe we should have, i'm not sure how the software handles a few closeup images among the 360° ones. I feel like the quality of the texture/model could've been better maybe if we did that.

The photo's were taken in a professional photo studio and with a 30.000$ camera of which I don't know the type.





All in all , it wasn't bad for a first test though.

[BUSNISMANPATRIOT]

Great pipe line ..
How can i download Project Photofly 02 ?

From there site i arrived at :
http://labs.autodesk.com/utilities/photo_scene_editor/eula_text/


and it's no download button here..or I agree button..or anything..

[Sparky3d]

Great pipe line ..
How can i download Project Photofly 02 ?

From there site i arrived at :
http://labs.autodesk.com/utilities/photo_scene_editor/eula_text/


and it's no download button here..or I agree button..or anything..


its here

http://labs.autodesk.com/utilities/photo_scene_editor/

[TheNeverman]

You can't download Photofly - just a client that edits/uploads your photos...
All the work is done in the cloud.

[BUSNISMANPATRIOT]

*edit* I got it Thank you :)

[Sparky3d]

You can't download Photofly - just a client that edits/uploads your photos...
All the work is done in the cloud.

You can download it I did it 3 weeks ago and installed on pc fine. I just tried again because of your post and again it downloaded fine

follow the link on my previos post

[TheNeverman]

Again, you only download a client.
The actual 3d processing resides on Autodesk's end...

Unplug your internet connection next time you get ready to run the software and see how far you get.
;)




You can download it I did it 3 weeks ago and installed on pc fine. I just tried again because of your post and again it downloaded fine

follow the link on my previos post

[nurpur2]

Hi Piggyson. Thank you for the YouTube tutorials. Nice to learn something new about ZBrush and your methods. Look forward to your face tutorials.
TheNeverman. I think it does say on page 1 that the download is an interface for uploading and downloding from the Cloud.

[TheNeverman]

... you mean Sparky....?


Hi Piggyson. Thank you for the YouTube tutorials. Nice to learn something new about ZBrush and your methods. Look forward to your face tutorials.
TheNeverman. I think it does say on page 1 that the download is an interface for uploading and downloding from the Cloud.

[Frenchy Pilou]

...videos! :cool:

[Nancyan]

Ok, here's an idea :)

I just learned about a new free sculpting app for the Ipad from here:

http://www.zbrushcentral.com/showthread.php?p=874519&highlight=Free#post874519

The heads up came from Moonchild (thanks!). I've been playing around with it and its lots of fun, but as yet you can't export a model so the app is more like a demo.

You can however rotate the mesh you sculpt around and take screenshots that get saved as snapshots. I'm wondering how easily those would get turned into mesh by this photo to 3d process. I haven't tried it yet myself, but it might be an excellent marriage of the two products. Hopefully though, this app will get upgraded with a useful export format.

Has anyone tried this?

[StephenCook]

I too have seen the Photofly from Autodesk.

The images you are shooting have background elements that cause artifacts.
Have you tried to use green/blue screen. A solid color would make cleanup easier.
Could try it ans see. I do not have the nice camera for testing it out.

Nice tutorial btw.
Thanks for sharing.

[bp72]

Wanted to share a quick pipeline process I found recently using Zbrush and Autodesk's Photofly. Using a digital camera, Zbrush and Photofly you can create quick highly detailed game geometry extreamly fast. The first images show some of the test examples I have created. The youtube links are to the HD versions of the tutorial Videos.

stump1.jpg (http://javascript<strong></strong>:zb_insimg('256116','stump1.jpg',1,0))
stump2.jpg (http://javascript<strong></strong>:zb_insimg('256117','stump2.jpg',1,0))
wall1.jpg (http://javascript<strong></strong>:zb_insimg('256118','wall1.jpg',1,0))
headexample.jpg (http://javascript<strong></strong>:zb_insimg('256119','headexample.jpg',1,0))

HD Part1: http://www.youtube.com/watch?v=QyL8g1tlMMY
HD Part2: http://www.youtube.com/watch?v=vm9MJKWiyuU
HD Part3: http://www.youtube.com/watch?v=NPuuEIkGKUw
HD Part4: http://www.youtube.com/watch?v=cJdwefzdobU

Happy Zbrushing!!

-Joseph

So you did the headshot the same way? How many pictures for the headshot?

[bp72]

@mix_mash: The process can be done with any normal cameras. I didn't use any special settings I just took a camera and started walking in a circle taking pictures. As long as the camera produces EXIF data it will work.

A minimum of 2 pictures are required to generate the 3d effect (same process that was used in the old viewmasters back in the day, and the current 3D craze with TV's.) You just need to take enough pictures for the program to stitch together the subject you want to capture (if you have a camera that does panoramic mode that would help in making sure you don't miss any areas.)

Another quick example of a head 12 pictures with a Canon 7D were used. (just the raw data from photofly)
headexample2.jpg (http://javascript<strong></strong>:zb_insimg('256298','headexample2.jpg',1,0))

-Joseph


EXIF data is the key. Thank you so much for posting this.

[durda]

Joseph - thanks so much for sharing your insights and experience with the Autodesk Photofly (now 123D Catch) app. This thread and your tutorial videos have been extremely helpful. What a cool app!

I'm having just a bit of a problem, though, in getting a texture to apply in ZBrush and I'm wondering if someone here might be willing to take a look at my files. I'm a bit new to ZBrush and not near experienced enough to recognize what might be a quick recognition of a standard and easily fixable issue.

So, I created a couple of object files from images of some tree trunks. I know that the object files and the associated UVs and texture files are good because the files import into Vue with no problem whatsoever and they render out a fine images.

In ZBrush, however, neither of the two files seem to have their texture images apply properly. In both cases I would expect from the info in the tutorial videos that because there are two texture files there should be two poly groups for each mesh. But when I go to ungroup them ZBrush tells me that this is just a single mesh. And of course I can only apply one texture at a time and neither one goes on properly (ther are sections that are in the right spot, but the rest is scattered randomly). If this is a common behavior with incoming 123D Catch files coming into ZBrush I'll need to figure out a fix to the workflow in order for this to be useful for me.

The files are online here:

http://www.boulder.swri.edu/~durda/arthurs/stump/ (http://www.boulder.swri.edu/%7Edurda/arthurs/stump/)

Any help at all would be greatly appreciated!

Cheers!

Dan



ETA:

I think the solution is here:

http://http://www.youtube.com/watch?v=zqKFWzXGipE

Same behavior I am seeing. Haven't tried this yet, but I'll bet it'll work. This IS a pain, though - I hope Autodesk gets this fixed up in a hurry...

[pedrinalex]

thank you so much

[Jaxx]

Thanks Joseph! I looked at the PhotoFly website a while ago but kinda forgot about it - this gives me new incentive to go on with it :)