> Both are among the highest-quality software I have ever used and hacked on.
> Both have less tests than I expected.
Interesting bit of context: Zed's Nathan Sobo and Max Brunsfeld are both alumni of Pivotal Labs, a firm _notorious_ for its zealous adherence to, among other things, TDD. I don't, therefore, find it surprising that "neither codebase has tests, for example, that take a long-ass time to run," because the more stringently one test-drives, the less patiently one tolerates slow test suites. Besides that, after a serious investment in test-driving, one starts to learn which sorts of tests have paltry or even negative ROI; "tests that click through the UI and screenshot and compare" were right there at the top of the chopping block for most Pivotal devs, and tests that "hit the network" were explicitly taboo!
I think that Kent Beck quote is great, but it's good advice for people who test too much, rather than devs in general or, god forbid, junior developers!
The way I think about TDD, it's just like how after a while, one gets tired of copy+pasting code into the terminal, and reluctantly writes it to a file. One gets tired of emailing files, and checks them into version control. One gets tired of manipulating state--of clicking through the UI, of newing up a bunch of collaborators in the REPL, of smashing tab while blanking on the name of the method one has only just written--and writes a test.
And maybe one who wakes up tired writes the test first. ;)
I dunno, in my career I’ve found that UI e2e tests were the ones that actually found bugs. All the unit tests that I’ve written ware mostly ceremony, to increase code coverage, etc, and were the first to need refactoring after some code change. A lot of refactoring.
The tests that stayed true were the UI tests that almost always pointed to real problems with real code.
It took a while to figure out how to write them though - trying to rely as little as possible on the internal ids / html / css and write them with what the user sees - e.g. instead of clicking on the button with “testid=login” we would “click on the button with the text login in it”. Identifying fields by the labels to them or tables by their column headers. It was inspired by rails’ capibara testing lib.
And making sure the tests were not flaky, fast to execute and isolated took some time.
But it was so worth the investment - it’s surprising how little those tests would change, as it allowed us to fearlessly refactor stuff without changing any tests. They felt a lot more like a friendly QA helping out rather than an annoyance you had to deal with after you’ve finished writing your code.
And writing them was actually fun, since you didn’t have to understand how the app worked, fiddle with brittle css identifiers etc, you just wrote the steps you thought the user should do and saved it into a file.
Being UI tests kinda meant they tested the whole system with the various micro services involved, databases and other infra. And I think this is where most problems in software arise, at the edges of systems when they try to interact with each other.
Static types, immutability automatic API schemas and validators usually make sure the code one writes executes reasonably well, its where the code one writes starts interacting with all the other systems where people usually can’t anticipate things. And thats where the integration / e2e / UI tests help the most I think.
I was a big fan of the unit testing done at JP Morgan. I found 1 exploitable bug when writing Unit Tests, for an untested swath of code that was of minor concern. Ofc this was one day out of a ticket that took a few days. After a year, when I left, it was the only bug that I found with a unit test, there. I wrote unit tests for everything during my time, and I found it to be a good practice. Do I always do it at other workplaces? No. Do I do it for my own projects? Yes. It's not that I don't care as much at work, but the time pressure is constant and I sometimes skip them if I can.
This mirrors my experience - tests working with real software running on real infrastructure uncover real bugs, easier to write and maintain in the long run. You still need unit test to verify edge case such us database going down or throwing a constraint violation.
> Both have less tests than I expected.
Interesting bit of context: Zed's Nathan Sobo and Max Brunsfeld are both alumni of Pivotal Labs, a firm _notorious_ for its zealous adherence to, among other things, TDD. I don't, therefore, find it surprising that "neither codebase has tests, for example, that take a long-ass time to run," because the more stringently one test-drives, the less patiently one tolerates slow test suites. Besides that, after a serious investment in test-driving, one starts to learn which sorts of tests have paltry or even negative ROI; "tests that click through the UI and screenshot and compare" were right there at the top of the chopping block for most Pivotal devs, and tests that "hit the network" were explicitly taboo!
I think that Kent Beck quote is great, but it's good advice for people who test too much, rather than devs in general or, god forbid, junior developers!
The way I think about TDD, it's just like how after a while, one gets tired of copy+pasting code into the terminal, and reluctantly writes it to a file. One gets tired of emailing files, and checks them into version control. One gets tired of manipulating state--of clicking through the UI, of newing up a bunch of collaborators in the REPL, of smashing tab while blanking on the name of the method one has only just written--and writes a test.
And maybe one who wakes up tired writes the test first. ;)