You will never have resources to “test absolutely everything”. It is ALWAYS about building out personas and deriving tests from those.
What this tells us is that one of two things happened:
This was not tested at all
The testing harness resets the environment after every check (e.g. “does process close when killed”) rather than involving a manual reset (i.e. “close and re-open task manager”)
The latter is a lot more common than you would think since it makes it much easier to automate these harnesses rather than having a human at a VM. But… this is what happens when you don’t step through the entire workflow.
They used to have very comprehensive automated testing processes to exercise all sorts of things. Unfortunately, like many tech companies these days like Apple, Google, etc., they’re all punting QA as a concept because they just don’t care - what are you going to do, go use another oligopoly platform?
Well, there may have been a period when MS was trying to improve product quality, and in that time, yes maybe they did have very comprehensive automated testing processes. But before that, up to the time of Windows 7 I guess, their quality was dog shit.
In the early days, MS was an undisputed monopoly though, and not only did they not test thoroughly, they hardly even tried to fix bugs - the userbase had to take care of that too. Earlier versions of Windows had all sorts of workarounds and 3rd party tools to try and get things to work properly.
I suspect that once they’d achieved their objective of improving quality, there just weren’t the incentives there any more for middle management to allocate resources to things like comprehensive tests.
If you are a company the size of Microsoft, you have more than enough resources to test absolutely everything.
You will never have resources to “test absolutely everything”. It is ALWAYS about building out personas and deriving tests from those.
What this tells us is that one of two things happened:
The latter is a lot more common than you would think since it makes it much easier to automate these harnesses rather than having a human at a VM. But… this is what happens when you don’t step through the entire workflow.
Didn’t Microsoft fire their dedicated human testing team in about 2014?
Probably?
This is the kind of thing that a LOT of companies outsource. Mostly for ill.
They used to have very comprehensive automated testing processes to exercise all sorts of things. Unfortunately, like many tech companies these days like Apple, Google, etc., they’re all punting QA as a concept because they just don’t care - what are you going to do, go use another oligopoly platform?
Well, there may have been a period when MS was trying to improve product quality, and in that time, yes maybe they did have very comprehensive automated testing processes. But before that, up to the time of Windows 7 I guess, their quality was dog shit.
In the early days, MS was an undisputed monopoly though, and not only did they not test thoroughly, they hardly even tried to fix bugs - the userbase had to take care of that too. Earlier versions of Windows had all sorts of workarounds and 3rd party tools to try and get things to work properly.
I suspect that once they’d achieved their objective of improving quality, there just weren’t the incentives there any more for middle management to allocate resources to things like comprehensive tests.
Latest news is that Xbox Games Division had been required to aim for a 30% profit margin for the last several years.
Thats why everything sucks and blew up.
Thats a fucking insane baseline target.
Line for next quarter profit must go up, therefore, cut costs.
This is obvious self destructive in the long run, but that doesn’t matter, what matters is C Suite’s golden parachutes.