Greetings!
hmmm yep, maybe you need to bake the layers details to your model, make another copy of your Ztool or the Zproject -just to make that test on it & keeping your original work as is- … bake all the details you need, then try again…
In my case, I once before detailed a character & I even deleted all lower subdivisions levels , kept in in single level of 13 million poly… then added another higher subdivision level just to let me extract displacement map -Exporter conditions that I need to bake while I am on another lower level even if they are just 2 levels only in your model-
& the results were very shape & full contrast details for tiny bump/pores & even heavily displaced places -I didn’t need to modify it in photoshop later-