1. In many cases the books were published by Microsoft Press, so the company benefitted financially as well.
2. The engineers may have a defences in not being given time to document better initially due to project deadlines beyond their control, and in the work being signed-off at the time.
3. Some of both the above mixed with other reasons.
4. they probably also didn't have the time for better API design, or the API design was done "by committee", and if you add to that the holy cow of "backwards compatibility", it's easy to see how an API might end up overcomplicated...
5. maybe M$ even encouraged these practices because of the perceived advantage for in-house applications vs. third party ones?
Microsoft (Press) published a fair number of these books—or at least books that were so much better than the reference docs that the latter could barely be said to explain anything, whatever the intent of the authors. This includes the classics: Petzold, Inside OLE, Essential COM, ...
They still had their monopoly, who cares? Besides, forcing people to invest time and to a lesser degree money to learn and get locked into your ecosystem is probably a net gain.
That's a direct action against the interest of their employer, and an attempt to enrich themselves at the benefit of Microsoft. They are purposefully writing bad software which drives ME market share and reputation among developers down only to sell some books.
Except if MS gets a share of the profits… which it did in many cases, being the publisher. They had no competitors, there weren't any concerns about market share or reputation!
There may not have been any concerns at the time, but the damage to the reputation is real, albeit difficult to measure. Nevertheless, you can see that Windows only ever seems to lose market share. Windows desktop is shrinking, Windows mobile is dead, Windows embedded is phasing out, and Windows server is only used for the bare necessities. You can argue that Microsoft successfully managed to pivot to cloud, but that is a rather shallow moat.
Among all other variables affecting the market share of Windows, the effect of the inscrutability of the Win32 API is almost certainly negligible. Especially given that as far as I know, their APIs designed in the past 20 years haven't typically been as horrible.
Microsoft had no real competition back then, and in their core business (desktop and office) is still quite uncontested. Alternatives to parts of the Windows, MS Office, and various corporate systems exist, but nobody offers the complete package. Extensions to their platform were never required to be perfect, just good enough that being ubiquitous (thanks to the platform) is enough to sell it.
Also, is it really against the interest of their employer to create products that lock in customers and make it harder to migrate away? Creating a market for support and customization, with MS sitting at the source of the knowledge and being able to sell crumbs by printing books and other training materials, and offering training and certification courses?
