This frustration you have with masking I can emphathise with. I think the masking in ZB needs a lot of work to improve it. I find it particularly challenging with the non-organic forms I have to create on a daily basis!
The tool you are talking about I think is called “Grow” in 3DS Max Edit Poly, and Maya has something similar. This would greatly improve ZBrush by having this.
Sometimes it is very painstaking masking areas in ZB, I usually hide areas of the mesh at low level, then go up to high, fill with mask then unhide again. The problem is the hiding of the faces is tricky as there’s no backface cull when it does it, also often it misses the face you want to hide. It would be nice if there was an option to toggle hide faces by touch as well as hide faces by surround, with your marque.
Also being able to have greater than 1024x1024 mask map. while on that subject, there is bleeding when you load a mask map back into ZB, even with well laid out UVs, rendering this method useless in my experience. I don’t save masks out anymore, it’s a waste of time, as with the cleanup you might as well re-select the mesh…
Lastly, even if you select a constant mask, there is still ghosting and faint masking in areas you didnt select! This requires cleanup, eating into production time!
Lets hope for ZB 3.5…or 4…