From an IM with Eric Hodel:
"I ran it on [insert scary class name here]
it finally ran into a [method in the r's] about 2 hours later
I think it found no errors
so heckle will tell you one of three things about your code:
its well tested,
its poorly tested,
and its so coupled that test failures must happen because its crap." (emphasis mine)
I have to agree. When I started heckle I was hoping for somehing better than rcov that would tell you that your code is poorly tested... But code so poorly engineered that changing anything as bound to make everything fall apart looks like well tested (read: heckle-proof) code in a sense. We both think that running just the unit tests that directly test the MUT (method under test) would help, but really it wouldn't help enough. I tend to think at that point that you don't have technology problems, you've got people problems. Technology can't help that very well.
Suggestions? How would you differentiate between "well tested" and "tightly coupled crap"?
That's a tough problem to crack, I'm not sure if technology could help. I could only think of one thing. If you could somehow know which methods were mocked or not, and intercept all the method calls you could display some sort of warning to the user if they were using a non-mocked object. You'd have to probably exclude method calls on objects instantiated from "core" classes, like String too of course.
That way you'd be able to test just the method stand-alone without any external interference.