Hello @nickholl ,
This mesh is probably just too complicated for Group by Normals. That feature does a great job with relatively simple forms with planar surfaces and lower resolution meshes, but the effectiveness drops off the more complicated/curved the form and the higher the mesh resolution.
You can try reducing the tolerance value. If you can reduce the generated polygroups to a fairly small number, you can probably manually correct the assignments with a few clicks of the polygrouping functions in ZModeler. However, the feature may simply not be a good fit for that mesh.
For meshes that are beyond the abilities of the other auto-grouping functions I would recommend Polygroupit. This will require some manual work, but it will probably be able to find the correct zones pretty easily with that mesh.
However, depending on how you are creating that mesh you should have options available to you to define the polygrouping as you go. If you are extruding the geometry most processes that do this should group the extruded section into its own polygroup. So if you start with the target surface as one polygroup, and the extruded area becomes another polygroup, you can simply hide both of those polygroups and group whatever is left into another. To do this easily you need to be familiar with with polygroup visibility shortcuts.
If you are merely displacing/inflating that detail from the surface, then the Polygroup > Group Changed Points function may be effective.
Good luck!
