Zbrush to Maya Displacement Workflow Tutorial

Hi there.

After years of pulling out my hair in order to get displacement maps to work, I have finally figured out (or have been told) how to get displacement maps to work in Maya and render fairly quickly. Before I begin, I must stress that good uvs are critical to getting a very clean displacement map. It is possible to get very good results using some kind of automatic UV mapping but it isnt guaranteed and is more of a gamble (this is ofcourse, just my opinion). Also, you will need two utilities for this to work: The first is the multi displacement 2 plugin for Zbrush, which can be found here: [http://www.zbrushcentral.com/zbc/showthread.php?t=029789](http://www.zbrushcentral.com/zbc/showthread.php?t=029789) You will also need a nifty little bat file called convertfile.bat, which automatically converts tif/targa/image files to .map files for mental ray, which can be found here: [http://www.datafilehost.com/download.php?file=3ec7a97e](http://www.datafilehost.com/download.php?file=3ec7a97e) So here are the steps: ZBRUSH: 1. Download and load multi displacement 2 into Zbrush 2. Load finished Z-tool, and at the lowest subdivision level, open the z-plugins > multi displacement 2 tab. 3. Use these options: UDIM: leave at default. Initial File Index: leave at default. Max Map Size: I find that 2048 works fine, but you can go up to 8K Map Size Adjust: change this to 0. DpSubPix: leave at 0, will use adaptive instead. Border: can leave at 8, as this will actually overpaint UV seams. In my tests, it actually does an ok job of overpainting if your UV's arent that great. 4. Click the export options button in the multi-displacement 2 menu and make sure that D32 is off (as well as every other setting) and turn on R32. 5. Paste this quickcode into the quickcode box for R32: DE-LBEK-EAEAEA-R32 6. Under your tool menu, go to the displacement sub-menu and check the Adaptive box. 7. Back to the Multi-Displacement 2 tab, click "create all" and choose a filename, all while at the lowest subdivision level of your ztool. 8. Wait for the map to finish and export your level 1 ztool with the "group" and "merge" options unchecked in the tool > export tab. Maya/Desktop: 1. Download the convertfile.bat file and put it on your desktop. 2. Find your 32-bit displacement map that Zbrush generated and drag it into the convertfile.bat icon. This will automatically convert the file to a .map file and place it in the same directory as the original displacement map (you can do this with any targa/tif file...other formats may work but I havent tested them). 3. Open Maya, and import the level 1 Zbrush mesh with groups "off" in the import options box. 4. Select your model, and in the attribute editor, turn off the "feature displacement" setting. 5. Open the hypershade, and assign a new material to the mesh. In the shading group of the material under the "shading group attributes" section, click the checkered box next to "displacement mat.", and select your .map displacment file. Make sure it is selected as a "file" and not as a projection. 6. In the file node under color balance, change the alpha gain to 2.2 and the alpha offset to -1.1 7. Open the Approximation Editor under Window > Rendering Editors > Mental Ray > Approximation Editor, select your model and click "create" for a subdivision approximation. I use the spatial setting and leave the value at 2 (but dont go past 3 or 4). If the attribute editor for the approximation doesnt show up, select your model and click the "edit" button next to "subdivision approximation". Thats it! Now hit the render button and it should finish fairly quickly.

thanks a lot for all this, it’s a very good resource

no prob

After more testing, I have discovered that actually keeping the “feature displacement” tab on produces better results but takes longer to render.

just wondering why you don’t use mental rays displacement approx node?

The “feature displacement” attribute works fairly well, but I suppose if you turned that off, you could use the displacement approximation instead.

if you’re using zonealarm or another firewall when it prompts for imf_disp and imf_copy be sure to click allow and remember the setting, otherwise it will error out

I had problems when just clicking “allow” once

dyaus ,

Thanks a lot ! It worked like a charm . Gonna save this text :cool:

Best Wishes ,

If it weren’t for all the helpful people here, my models would still be confined to ZBrush, so I want to thank you for the workflow tips.

I’ve got a thorough understanding of the entire process all the way up to the .map file conversion step. I am not yet familiar with mental ray, aside from the very basics and setting up approximations. I use a mac, and haven’t been able to understand the process of converting my TIFs to the .map format, so I’ve just skipped this step, making sure my TIFs are simply uncompressed. Aside from this exception, I’ve obeyed all the rules of this process to the letter.

But I think that may be what causes mental ray to finish rendering prematurely on my machine. It thinks for awhile, then renders about 5% of the image, then it finishes the frame with the render time and info readout at the bottom of the screen as if it has done its job correctly. Am I maxing out mental ray’s memory or something? Does anyone know of a good troubleshooting resource or tutorial for mental ray newbs like me? Any help would be so appreciated. I know I can get okay results from the Maya software renderer, but I’ve been told the mental ray can be faster with better results when using displacements.

Thank everyone!

what errors do you get when mental ray stops? out of memory?
hit the render diagnostics button and review that.
also in the mental ray tab of render globals go under translation and switch warning messages to progress messages
then when you render open up console.
on a mac this is not the console inside maya but the system console inside the applications, utilities folder.
as it renders, with this open you will see a stream of information that can help diagnose your render problem.

as fas conversion to .map on macosx, this is done through command line. i’ll post more about this later when i’m in front of my machine. but a .tif works just fine for me.
there is a size limit for your image, i wouldn’t render any textures in you scene larger than 4096x4096. mr chokes on them.

i would be very interested to a response to this too, as i have similar problems when rendering with mental ray, and never understood why. I am on a pc, sometimes when I render with mental ray its fine, but then if i update my pic, shadeers etc, and then do another render it simply shows me the render pic of the previous one, without any update? I can then nor render the new angle/scene at all? Whats going on?

Thanks bobobobson, much appreciated!

I’ll be taking some time tonight to set up a description of my process, with one or two images to help illustrate certain results, etc. One thing I suspect is that I had multiple displacement maps in the scene concurrently, all sizing in at 4096 each. I maybe I am just overloading? I’ll follow the troubleshooting steps you’ve outlined and post the results in a few hours.

Thanks again for the tips, and I look forward to learning how to get these .map thingies that mental ray seems to crave so much.

More to come very soon…

I initially started out wanting to make a character for experimentation, since this is my first venture into displacement mapping, as well as mental ray rendering. But I can’t help myself when I sit in front of ZBrush, and before I knew it, I had a model I really cared about, but no good plan of attack in place to get her back into Maya for animation. So I’ve been following some amazingly thorough tutorials here in ZBrush Central, and I’ve studied all the relevant resources I could find very closely. I understand the process – all the way until the end, that is. Let me describe in detail what is happening:

I’m on a mac, a Dual 2.5 PowerPC G5, OSX 10.4.9
Maya Unlimited 7.0

This may not be the most ideal way of processing such a model, but I tried to be as logical as I could with what I knew when I began.

First, I modeled a character in Maya, and set up all the UVs correctly. I then split the model up into parts before exporting them as OBJs to sculpt in ZBrush.

So back in Maya, I assign shaders to their proper parts, then I turn my model into one single mesh, and convert it into a subdivision surface, making sure that “Feature Displacement” is unchecked. I open up the approximation editor with my mesh selected, and create a Subdivision Approximation with the following settings:
Min = 3
Max = 5
Lenght = 0.01

But one thing I noticed is that my image files weren’t giving me a preview of these displacements in the Hypershade window. I tried to reload one, and the Script Editor showed this:

“AEassignTextureCB file13.fileTextureName “/Users/starcircle/Desktop/New_TIF_Disp_Files/Acc-R32.TIF” “Best Guess”;
// Error: Image conversion (to IFF) failed //
// Result: 1 //
// Error: Image conversion (to IFF) failed //
// Warning: Failed to open texture file /Users/starcircle/Desktop/New_TIF_Disp_Files/Acc-R32.TIF //”
(These last 2 lines repeat several times.)

And here’s what the Render Diagnostics had to say:


  • Subdivision surfaces will only be rendered if the base mesh contains only quads.
    // Warning: 1 warnings, see script editor for details. //"

My original poly model was all quads, so I don’t think my Subdivision surface should be anything different, right? So, is this just a blanket warning, or is this actually a problem?

My Console program (found under my Utilities folder) is constantly running some sort of repeating string about “Looking for devices matching vendor ID=1193 and product ID=8719” which is opening up a whole other world of troubleshooting for me. I flat-out don’t know how to use this, and right now I need to focus on getting mental ray straightened out, so I closed this window for the time being.

When I try to do a render, the render stats show up after only 10 seconds, with nothing visibly rendered in the Render View. Now, the Script Editor has this to say:

“// Info: (mental ray) : number of render thread(s): 2
// Info: (Mayatomr.Scene) : shadows obey light linking requires full shadow tracing, optimize raytrace shadows ignored
// Info: (Mayatomr.Nodes) : derive mental ray global approximation settings from Maya
// Info: (Mayatomr.Nodes) : derive mental ray global displacement settings from Maya
// Info: (Mayatomr.Scene) : update lights and related shaders
// Info: (Mayatomr.Scene) : update cameras and related shaders
// Info: (Mayatomr.Scene) : saving image to file /Users/starcircle/Documents/MAYA/Projects/Reggie_Project/images/tmp/Reggie_001.tga
// Info: (Mayatomr.Scene) : saving image to file /Users/starcircle/Documents/MAYA/Projects/Reggie_Project/images/tmp/Reggie_001.tga
// Info: (Mayatomr.Scene) : update geometry and materials
// Info: (Mayatomr.Nodes) : place2dTexture18.vertexUvOne: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture18.vertexUvTwo: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture18.vertexUvThree: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture19.vertexUvOne: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture19.vertexUvTwo: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture19.vertexUvThree: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture20.vertexUvOne: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture20.vertexUvTwo: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture20.vertexUvThree: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture22.vertexUvOne: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture22.vertexUvTwo: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture22.vertexUvThree: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture23.vertexUvOne: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture23.vertexUvTwo: unsupported type of connection, current plug value used instead
// Info: (Mayatomr.Nodes) : place2dTexture23.vertexUvThree: unsupported type of connection, current plug value used instead
// Prog: (Mayatomr.Scene) : DAG node: Reggie
// Info: (Mayatomr.Scene) : : max displace (6.273986), calculated from displacement bounding box scale
// Warning: (Mayatomr.Geometry) : ReggieShape, subdivision surface contains creases on boundary edges which are unsupported and have been ignored //
// Info: (Mayatomr.Scene) : render camera: persp”

I am a bit out of my realm of knowledge in the world of TIF files and what requirements mental ray has for them. I’ve tried opening them in Photoshop, making sure “Color Management” is disabled, and I saving them back out as “Uncompressed” TIFs, in the “IBM PC” format, but that doesn’t seem to help. I’m trying everything I can think of. It’s been difficult for me to find resources on mental ray, at least ones that I can comprehend. I have never been a quick study when it comes to memorizing things that sound like programming language, etc. I am used to being self taught, and I am not afraid of a good challenge, but mental ray has proven itself to be my biggest challenge since figuring out how to unwrap a model’s UVs. (That process took me awhile to get.)

This problem is keeping me awake at night! I’m just shooting questions out in the dark, like does mental ray require that the TIF files be RGB? (I ask this in case I sometimes don’t use the Multi Displacement plugin.) Does it work more smoothly if they are converted to .map? This may be a big newby thing to ask, but when people talk about using the command line to run imf_copy, are they talking about the command line for entering MEL commands at the bottom of the Maya interface?

I know this is a long post to read through, and maybe not all of it is relevant, but this Displacement process seems to be at the heart of my ambitions for my 3D creations (until animation packages become as agile as ZBrush, anyway) so I really need to understand where I’m going wrong. I thank everybody here for this wealth of info and insight, and if anyone can spot an issue in my workflow, please drop a line or two. I’ll buy you a drink, or send you a movie ticket or something!

Thanks to all the knowledge sharers, and good luck to all my fellow knowledge seekers out there!




"This problem is keeping me awake at night! I’m just shooting questions out in the dark, like does mental ray require that the TIF files be RGB? "

I’m fairly new at the process, but I have had some decent results, and I too use Maya 7.
There’s no way for me to know exactly what’s wrong, but I can point out the things you are doing that are different. Maybe those are the sources of woe.
The first is the whole combining several meshes (and UV sets) while back in Maya. You mentioned the need for detail, but I really think
a. anything over a million polys, using Projection Master, and a Bump Map with Bump Viewer mat, you’ll NEVER lack for detail.
In other words, try a test with something simple like a single cube.
The other thing is I always convert my disp maps (TIFFs) into RGB mode in Photoshop. Apparently Maya needs this to be so. I have yet to ever convert anything to.MAP ( I still haven’t figured it out)
The other thing that really stood out is the whole converting to a SUBDIV SURFACE thing. Just try it as regular poly Quads.
I feel your pain, just get something simple to displace, oh yeah, did you use BOTH a SUBDIV APPROX and A DISP APPROX?
One other thing, I have yet to use the the multi-disp plug-in, so maybe that’s a big difference. Well, good luck…

Just get the process down in a SIMPLE way before you start attempting several , complex, endeavors.

Have you followed the guide by Scott Spencer?

Nice looking work, and you seem to have done things generally the way many do. MR is finicky though, so check out Scott’s guide.

If you did and problems are still occuring, please search the forum as this topic has come more times than it should. :wink:

Not to sound blasphemous against actual religions, but that guide from Scott Spencer has more or less been my Bible for this subject. I do realize that there are many posts about Displacements, and I don’t mean to add any clutter. But following all the rules I’ve found in these threads, as meticulously as possible, is still leaving me “renderless” in mental ray. This is sort of my last attempt at finding the answers before I just go back to using the Maya Software renderer, but I have searched for days now, staying up all night long following clues here in this forum (and elsewhere,) as to why these settings work for so many, yet not here on my mac.

Believe me, I’m not the kind to “post first and try it out later” so to speak. I’ve been quietly stuck on this for a long time, trying, searching, trying, searching…

But your point is very well taken, and I don’t want to make these threads any more jumbled or repetitive on the subject. I just couldn’t go down without at least once asking for help. Thanks again all,

Ok, but help us help you and post that kind of info.

Have you downloaded the sample that Scott used and does it work?

In other words, before attempting something did you test the process to see if it was either you, your apps, or system that was the issue?

If yes, post the version sof each so people can help you a little more effectively.

I don’t use MR for anything, but it worked for me right off when I tested it, following Scott’s tut. On both an old G4 Cube I keep for sentimentality and on the MacBookPro 17" Core Duo maxed out.

If you have all this and someone else has the same setup and you get different results at a stage in the process based on the baseline (Scott’s), then we can narrow down the problem. :wink:

I love displacement mapping and it has always turned out good. But only after your workflow is laid down in stone will you see the results you are looking for. With disp maps you should always convert them to MAP format. Mental ray in maya hates tifs. Another thing that maya hates is rendering 4096 32 bit maps. When dealing with disp maps always make sure you stay around 2048X2048.
I have several video lessons on the subject but my site is going through a upgrade right now. Getting ready for z3 tuts;)
http://www.schoolof3d.com/page2.html look for the 32 bit recap But if its down give it a few hours until my domain upgrades.

Thank you all for the insights – and for that link, cannedmushrooms! I did find the new info and the recap useful, especially knowing to avoid 4096 files in mental ray, and that I may not have been helping things by converting my polys to subDs.

I appologize for not clarifying this sooner, but I have had no problem with the smaller, simpler test head and sword projects, etc. My troubles seem to point specifically to not knowing how to imf_copy my 32-bit TIFs from MD2 into .map files on a mac computer. I’ve found limited resources online for this topic, and what I can find eludes to the need for Unix skills, of which I have none.

I won’t keep filling up this threads with this issue, but I’m basically just asking if anyone can point me towards a tutorial, or mention of this procedure on the mac platform. I know I’m not alone in this query, as I’ve come across other mac users asking this question too, and from what I’ve read, they may not be finding clear answers either.

Thanks again for all the help everyone. My friends are astonished at how fast I’ve gone from an oil painter to a digital artist, and it’s this community that is mostly responsible for that! :smiley: :+1:

sorry to be away so long

i don’t think your problems have to do with the image format or size. i use maya 7.0 on a ppc dual 2.0ghz osx 10.4.x and it works fine for me.
i do see some issues that could be arising from your workflow.
first you exoprted seperate .obj from maya to zb. no issue. but on return you combined the meshes. this probably is the start of the problem. you should try to render this without combining the meshes. one shader to each mesh with their corresponding disp map. second, you convert it to a subd surface. this isn’t necessary, that’s what the subd approx node is for. it takes care of smoothing at render time. other than that you don’t see a preview of 32 bit floating .tif files. try not to have to open them (32 bit floaters) in photoshop, it will change your files and mess up your render. flip them in md2 it’s easier and removes another calculation at render time.
btw- you will need seperate subd approx nodes for each mesh.

i’m going to be bold and suggest a workflow. if your character is seperate meshes, the steps above should work fine. but, if your mesh at the beginning is one mesh, don’t split it up. map the uv’s in seperate uv regions 0-1, 1-2, 2-3, etc. when in zbrush use the uv polygroups to do your work. md2 will export a seperate disp map for each uv group. when back in maya go in to the uv editor and select all the uv’s in a single uv region (0-1, 1-2, etc.) then under the uv editor “select” menu choose “convert selection to faces”. now assign a new material to the faces, this material will receive the disp map for this section. do this again for each uv region.
after all this you only need to assign one subd approx node because there is only one mesh. should help it render faster.
the only thing you need to be aware of is that zb will only export one color texture map. so for painting you need those uv polygroups i mentioned earlier. only paint one group at a time, do your work , export your texture, make a new one, start on another uv group.
hope this helps

ya I won’t even claim to be any mac expert lol
But I do hang out in this forum here somtimes


Looks like it has to do with running the imf_copy in terminal mode.

Whatever you mac hippys are calling it these days;)

Hope that helps