I’m a hard surface modeller, but have never found zbrush practical for complex HS work for the reasons you mention. I always viewed it as something you could use for spot work here and there.
The new update represent a shift in my thinking though, as dynamesh greatly improves the efficiency and practicality of HS modelling. It’s not quite there yet, but for the first time I legitimately see a time where it could be.
My own tests indicate the time differential is not so significant for me anymore. Not like it was. There are some things a traditional polygonal modelling process is always going to do better, there are things a free form approach free from concerns of geometry do much better. Relatively simple shapes are always faster modeled traditionally. Complex shapes with combinations of smooth curves and hard edges, not so much. Also, it is easier to explore different things in zbrush, than it is during a traditional modelling process. Traditional quad flows tend to force us into predictable designs and patterns in geometry. A free form process in Zbrush lets you think about form first and foremost, not geometry.
Modelling a replica of an established form as quickly and efficiently as possible is one situation. Designing something from scratch is entirely a different matter, and Im not sure I would choose to do this via traditional means anymore, unless I had a job that absolutely required it.
Retopo is still the great barrier to free form modelling as you say, and yes Zbrush needs better tools here. But I see that as true for any free form work, not just hard surface. Furthermore, it was to the point that if I wanted to work in Zbrush at all anymore, I still found myself retopoing even my hard surface stuff. Efficient geometry made in a traditional fashion is not good geometry for sculpting. Economy of polygons means some are going to be longer than others especially near the edges, which makes for poor sculpting. So I found myself building geometry in less efficient ways with more loops simply to facilitate detail work in Zbrush. And laying out toplogy is always over an existing form is generally much faster than creating it from scratch in a vacuum.
Creasing is another issue. In general, unless you have a traditional modeler capable of exporting zbrush quality hight resolution models, the only way to get your hard edges or smoothing groups into zbrush without having to slog though Zbrush’s native creasing functions (which is a truly laborious and quirky process for complex models, involving you to isolate polygons in any number of different ways, and sometime created small geometrical quirks where polygroups would not behave as expected), is to put a fine bevel along all your edges to reinforce the geometry, which again, is bad for zbrush sculpting performance, and bad for uses outside of zbrush. So I found myself retopoing regardless if I want to use Zbrush at all. Geometry created natively in Zbrush, depending on your technique can bypass the need for creasing or smoothing groups, and let you generate crisp lines in your displacement and normal maps without the hassle.
In the end, it comes down to how you want to work. I want to do hard surface work in zbrush for same reasons I want to do organic stuff. I could do organic modelling in a traditional modeller, but its not how I want to work. Even if the process was slower, I would still choose digital sculpting. The time I spend in my traditional modeler, is time I spend wishing I was already in my Zbrush stage. I want fewer barriers of technology between me and the form that I am shaping. I want to feel as if I am working with my hands as much as technology will allow. I want creative endeavors to be given to artists, not technicians. For hard surface, I want to feel like I am carving and detailing that form by hand, not having a computer do it for me with automated operations, or by pulling points with a mouse. I want to be able to open zbrush, and sculpt or carve any kind of form, with my “hands”. I believe in the Zbrush philosophy, and truly believe that before the end of the decade, if Zbrush and programs like it continue developing in the necessary areas, we will be digitally sculpting hard surface form as a first resort, not as an exception. Some people will always have requirements for exacting mathematical precision that will require a more traditional tool, but I definitely think that will be true for people that dont absolutely have to.
when it can be done much faster using more traditional methods, plus you have the production ready model done.