Haven’t tried Mental Ray yet, but I suppose I could. Not sure the results would be the same since Vray and MR use two very different methods for displacing meshes, but it’s worth a go I guess.
Here is the Mental Ray render result from using the identical files that I used for doing my Vray testing tonight.
Interestingly enough, turning the blur setting down to 0.01 in the material editor didn’t help at all as it does with Vray. Besides the banding, I’m sure my MR render could be a bit better, but I didn’t want to balloon the render times. Plus I couldn’t figure out how to get both negative and positive displacement on the mesh at the same time with MR. Anyone know?
I think you can adjust the rgb offset in the bitmap output options something like a value of -0.5 (not sure exactly) to get values below the neutral gray in to the negative.
Yeah that’s what I did, and while it’s the right way to make ZB’s middle grey turn black so that it registers in MR as no displacement, you still only end up with positive displacement and not any negative at the same time unfortunately.
Zeddicus, thanx a lot you have found the reason…
But! Try to increase subbdivision and banding will return… i thank banding is a consequence of big diffirence between resolutin of model (after render subdivide it for displace) and map. All the filtering stuff is made to adjust it…
nevertheless, there is a problem: try to test detailed map and you method, 4X4 thousand for example.
Trogwart: My displacement map was 2048x2048 16-bit and my model was a sphere orginally made in Max (16 segments, 100 units in diameter). I will play around some more and try to see what you’ve mentioned.
Edit: Ok, just quickly tried renders (using the above) at 32, 64, 128, and 256 subdivision (3D mapping mode). With the blur set to 0.01 I wasn’t able to make the the banding reappear.
Ok, i’m playing around too, and i found interesting thing just now.
If your model have less resolution (for example 1 level mesh smooth) and the displacemet resolution set high values (512 for example) you have banding independently from filtering (no bluring make banding more sharp only)
If you set higher resolution on model (maybe 3 levels ms) and low displacement res (32, 64 something like that) no banding!..
(it was all about 2d displacement, but i suspect 3d will be the same)
Is there any ideas?
Just tested and I do seem to be getting both positive and negative with a -.5 offset.
Started with a 50% gray image and it barely moves when displaced compared to non-displaced.
Then I added a darker value to that and it shifted inwards… the 50% areas stayed the same.
Then I added a lighter value and it shifts out above the 50%.
I do however get the same banding on displacements 
Trogwart: I have noticed that when setting the resolution high in 2D mapping mode, regardless of the mesh smooth iterations you might see jaggies around displaced areas. This sin’t the same thing as the banding and is independant. Blur has no effect on it either. I think this is something particular to Vray. About the only thing you can do is lower the resolution so that it is mitigated some what, but not so low that you start getting anomolies in your mesh (like black speck or streaks for example).
Moob: Try increading the displacement ammount and doing a render with MR displacement ticked to off in the material editor, then do a second with it on. That will give you a better idea as to what is happening. As far as I can tell when the offset is set to 0.0, 50% grey to MR is 50% positive displacement. Setting -0.5 offset shifts everything in the bitmap down by half and makes the 50% grey look like a 0% grey (black) and this renders the same as when displacement is turned off for me. (Edit: Just made a 50% grey map in Photoshop and did the test I just recommended to you for my own peace of mind. It would seem black is indeed 0% displacement and white is 100% displacement, meaning no positive and negative displacement at the same time.)
Can you post the best settings you adjust for perfect result for high-res maps?
I’ll try it on this
It is a screen from Zbrush a model i would try to reproduce with vray, i will be a battle-field test. There is a test region on the surface, that will show if displecement would be blury or low res and not perfect.
Yeah that was my point 
but values below 50% on the map are still showing up after being shifted -.5 and they become negative displacement.
Moob: Ah, my apologies. I see now, your right. Sneaky…
Trogwart: I really don’t have any “best” settings for Vray displacement. I just play around with the various values until I get something that looks right, or as close to it as possible.
One of the guys here has a maxscript that reads the offsets contained in the displacement map and sets up vray displacement to the correct amounts automatically. I’m trying to get him to post it. It’s very nice tho have any guess work taken out of the + - amounts.
Glenn
And it appears you can use only 3d mapping in vray disp with zbrush (2d dont understand UV mapping from zbrush)…
grassynoel: I’ve noticed the scale of the mesh affects the displacment amounts. A small mesh requires small numbers and a large mesh large numbers. The script should be interesting, though with Vray as fast as it is playing with the initial settings isn’t all that bad. 
Trogwart: Here is a quick test on an old Z model of mine. The UV’s are AUV and the model+map was rendered with Vray (2D displacement mode), so you can see they do work together properly.
Yees!
It works!
No @##$ banding.
Zeddicus, respect…
4000x4000 disp map, all details are visible.
some artefacts but it works!
Configurations:
3d mapping,
amount 40
shift - 20
face subdivs 8
base model has 16000 polys+ 2meshsmooth.
In result we have model that can be rigged (try to apply skin and bones to a million polys model,he he) and all details are present.
Zeddicus,
2d + AUV vorks? hmm, i tried and get bugy displace placement…map was vertical flipped, all done but didnt work… 
i also tried spherical mapping - didnt work too. (strange bug - all image are dark and deformed).And if you apply then spherical maping in max - it works, but wrong position.
2d much faster, i’ll be beter to use it.
His script takes the scale of the mesh into account and accuately sets it up no matter what.
Glenn
please please where is this script??? 
it drives me crazy to find the correct parameters for the displacement strength.