Testing And Scale
I've never really liked testing.
This has not been a psychically easy position to hold. What does the internet have to say about testing?
A billion is just a number---much harder to discount are the myriad reports of smart people that tests are essential, there is no other way to approach things, any piece of software without tests is an unworkable piece of crap.
I respect these people. How to resolve this?
Things made more sense when I realized that most of my work has been greenfield---and not just greenfield, but not part of a system. Part of the work was figuring out what to build in the first place, what the software should even do.
In that environment, tests lose a lot of their value. Automated testing in particular---the key value proposition of automated testing is that every time the tests run, you get that for free. Multiply it over a lot of runs and a lot of time and that adds up.
But with independent, experimental software, your software is very likely to change, rendering all the tests you wrote useless.
This is really bad! Drag doesn't just mean software gets written slower. Often it means it doesn't get written at all. A testing mentality is probably a net negative at first.
What you want at the beginning is manipulability and observability. A REPL is table stakes here, but easy visibility into the environment is not far behind.
At some point that doesn't scale---there are too many things for you to keep in your head at once. But even then it might not be time for testing---in fact, it's probably not. Instead, it's probably time for abstraction.
So when is testing useful? Well, there is an awful lot of software out there that isn't independent, experimental, or greenfield. The vast bulk of it, actually. Software that talks to external systems with defined interfaces, or that already works (and has been working, for years), and is just undergoing some feature additions. And, of course, big software, with a ton of code---too big for someone to hold in their head.
At that point, tests really are essential.
What's interesting is that at the beginning, tests are a drag and slow you down---anything but "agile," in other words. But as time goes on, it's hard to make changes without a test suite. You never know what you might be breaking.
This has not been a psychically easy position to hold. What does the internet have to say about testing?
I respect these people. How to resolve this?
Things made more sense when I realized that most of my work has been greenfield---and not just greenfield, but not part of a system. Part of the work was figuring out what to build in the first place, what the software should even do.
In that environment, tests lose a lot of their value. Automated testing in particular---the key value proposition of automated testing is that every time the tests run, you get that for free. Multiply it over a lot of runs and a lot of time and that adds up.
But with independent, experimental software, your software is very likely to change, rendering all the tests you wrote useless.
This is really bad! Drag doesn't just mean software gets written slower. Often it means it doesn't get written at all. A testing mentality is probably a net negative at first.
What you want at the beginning is manipulability and observability. A REPL is table stakes here, but easy visibility into the environment is not far behind.
At some point that doesn't scale---there are too many things for you to keep in your head at once. But even then it might not be time for testing---in fact, it's probably not. Instead, it's probably time for abstraction.
So when is testing useful? Well, there is an awful lot of software out there that isn't independent, experimental, or greenfield. The vast bulk of it, actually. Software that talks to external systems with defined interfaces, or that already works (and has been working, for years), and is just undergoing some feature additions. And, of course, big software, with a ton of code---too big for someone to hold in their head.
At that point, tests really are essential.
What's interesting is that at the beginning, tests are a drag and slow you down---anything but "agile," in other words. But as time goes on, it's hard to make changes without a test suite. You never know what you might be breaking.
Comments
Post a Comment