they are so hard to use, but why
the clip brush doesn’t adjust the contour of the outer most edges to match the new border.
if like you did, clip 3/4 of a sphere you get that funky edge you will notice the edge matches the outer most part of the pre clipped sphere. if you masked 3/4 of the sphere, inverted the mask, then shrunk the 3/4s so that its edges are inside the perimeter of the masked part, then when you clip you will get a smooth nicely clipped edge.
it just the way the brush works, it doesn’t know edges, it just shoves the geo straight back. as if you squashed rather than clipped.
so how do I get nice edges
try the trim curve brush the same way on the sphere and you should get the results you expected. but trim brushes have their own draw back.
I pretty much always use slice curve instead of trim curve… and just delete the unwanted sliced polygroup(s), and close holes.
Always clean.
Or Mask > then edgeloop mask.