About 7 years ago I was in a meeting with a former Windows core graphics engineer.
My team was attempting to figure out some extremely obtuse workflows with MediaFountain and DirectX.
We had a meeting with this guy and a couple of the other engineers that wrote the APIs and the implementation underneath.
This particular guy literally chuckled as we aired our frustrations and this was his response:
“Yeah you won’t figure those APIs out from the documentation. It was on purpose. You have to go buy the book.”
Proceeded to explain to me that this was how he, and many other core Windows engineers lined their pockets for years - write complex implementations, do the absolute bare minimum documentation, then take a 6 month sabbatical and publish a reference book that was absolutely required to actually use the API.
Apparently many of these guys made 10-20x their salaries on this grift and it didn’t really stop until the mid 2000’s.
> “Yeah you won’t figure those APIs out from the documentation. It was on purpose. You have to go buy the book.”
I KNEW it!!! In a prior job, I had the unpleasant task of shepherding a SharePoint installation, and was incredibly frustrated at the documentation AND the available literature. I jokingly commented it must be to keep all those SharePoint consultants busy.
Now I think that might actually have been the case. Those poor souls...
Yep. I also suffered with SharePoint in the late aughts, and eventually consultants were brought in to add some stupid custom feature to out Intranet Wiki that required a month of custom coding.
Plenty of other companies doing that as well. Pretty much every ERP company, for example: from big ones like SAP or IBM, to the ones only popular in their home state.
I remember a consultant telling me “you don’t get it, the point of our app is that you can build ANYTHING the customer withs it”. Well, so is the point of, say, Python.
> I remember a consultant telling me “you don’t get it, the point of our app is that you can build ANYTHING the customer withs it”. Well, so is the point of, say, Python.
Yeah. The idea of generalized, reusable platforms are kind of a siren song for business types.
1. Executive notices the company has 5 apps that sit in a similar place in architectural diagrams. In a meeting, she says: "we have 5 different apps that are creating outputs based on various inputs according to business requirements. She declares, we should have ONE platform to replace those 5 apps. Savings!"
2. Platform team gets stood up. Starts to build the that app according to the new executive strategy.
3. Result: you get a shitty programming language (implemented as "configuration files") with a shitty transformation language. Turns out your platform was actually Java and XSLT, and your previous 5 apps could be understood as "configurations" of that platform.
4. New leadership takes over. The new "platform" gets de-platformized, because it sucks. Now the new platform will be built on the cloud!
The point of SharePoint was to allow end-users to cobble things together themselves. One of the original design philosophies was "Excel on the web", hence the reliance on it's Lists as a core data structure (as inappropriate as it would of course become, when users started treating them like giant RDMS stores). My anecdata is from talking to some of the original design team members, from when Microsoft acquired it from VTI.
It was originally not intended for developers, it was for power-users. The fact that it was built on .NET technologies made it extensible though, and boom - quickly enough there was a huge market.
Well, here's the thing... you're not wrong. You're actually 100%... scratch that, 120% correct!
Software shouldn't really be a constant struggle. It shouldn't be about constantly forcing square objects into tiny round roles.
However, when you're an experienced developer that's easy to see. Sometimes even when you're inexperienced it's also easy... But when you're upper management, and a developer says "we shouldn't do that in Sharepoint" while Microsoft is telling them "this is 100% possible in Sharepoint" and the Microsoft approved consultancy says "this is extremely easy in Sharepoint", the developer ends up looking bad! :(
One approach to dealing with that is to then suggest that if things are so easy then surely the consultancy would be willing to do the project on a fixed price basis?
Yeah, that would do the trick. But then again, in companies with thousands of employees this kinda things tends to get lost in translation between the multiple management layers, or when some over eager product manager decides to make their life goal adding some weird customization to SharePoint at any cost possible :/
It does allow people to build useful things, I'll give them that. But our company didn't have any power users, so us admins got handed the task of doing that as well. (manic laughter)
Heh... I had a client, as recently as 2020 who barely had "users" - who didn't know how to perform basic computer operations (Cut/Copy/Paste, navigate menus, etc).
Hell - yesterday, I was working with a DBA over remote screen-sharing who had no idea how to enable the display of line-numbers in his preferred SQL text editor... (Or even that you have to press "OK" to save your changes in a modal dialog box...)
I don't mind if you build a lot of stuff on top of it. But if the vendor more or less withholds useful documentation from their customers on purpose to make them buy their training and certifications or hire their consultants, that's a dark pattern, isn't it?
And that’s the business model of a lot of ERPs, especially mid-sized ones. Some of them even have private configuration software, and I even remember one having a private “special compiler” that was just applying something equivalent to a ROT13 to the code and zipping to discourage customization by customers.
But the crazy part is that you don’t even need to resort to this. Unless you’re SAP or something like that, there’s no chance in hell that a developer will want to hitch their career to your shitty ERP, so consulting it is.
Any "platform" based approach is going to have that weakness - if you are in the domain where an ERP is a good fit then you'd be crazy not to use one, if you're problem isn't actually a good fit then you can end up in a whole world of pain and you'd be better off with a general purpose dev environment.
Unfortunately deciding this in advance can be tricky.
Indeed, during decision-making time, the people signing the checks always think that everything will fit nicely.
With customizable ERPs, what I have witnessed all my career were companies refusing to change their (universally hated, bureaucratic, insecure and inefficient) internal processes and having to pay more for customization.
The problem in the end is that there is rarely any shift, unless you have major cultural shift and oversight, coming from someone with the intention to save money and custom development time.
I feel like this is much of why I hate windows server administration compared to linux.
If you want to know something on linux, either read the man pages or check online. Someone probably wrote a blog post 13 years ago (but carefully updated over the years to keep the information fresh and accurate) that details exactly what you need to do step by step, along with a conversation about the options that you have as you are instantiating the server and asks for nothing in return other than the pleasure of having shared their knowledge.
If you want to know something on Windows, you can either get extremely lucky and figure it out yourself after hours of trial and error or find the one post on the internet that details the specific issue you have.
Learning how to administrate windows servers from official Microsoft sites? That is like reading an encyclopedia to learn how to do surgery. Microsoft's official education webpages on windows administration is the end user equivalent of watching a youtube video on how to rebuild a lawnmower engine when you need to put gas in your weed eater. It's in the ballpark range of the information you need but is so fundamentally inept and terribly wrong for a useful purpose that it boggles the mind how there isn't a better source of information.
And now we learn that there is a better source of information. Printed books. Intentionally sold by the people who wrote the terrible online documentation.
Because this is 1954 and we don't have any other options, right?
Haha yeah that's why I cringe whenever someone reassures me that Open Source software models are profitable, as "you can just make money from selling the consulting services for it!" Okay, but what are you incentivizing there, then?
I worked at Microsoft from 1996 to 2011. I was a SDE and lead on Windows Media and Media Foundation. This is a cute story, but I never heard of anything like this. There is no way my management would have approved any time off to write a book on a Microsoft product. Sabbatical is possible, but everyone I knew that qualified for a sabbatical was already filthy rich from early 90s stock options, and used that time to sail around the world or something like that.
I can't rule out that such a thing did happen at Microsoft in groups that I wasn't part of, but I would be stunned if it was more than a couple of people.
Well, I hope you inquired as to whether he was born that much of an asshole, or if he had to work his whole life to achieve it, and then refused to address him as anything other than "fucking wanker" for the remainder of the meeting. On behalf of the rest of us.
> he, and many other core Windows engineers lined their pockets for years - write complex implementations, do the absolute bare minimum documentation, then take a 6 month sabbatical and publish a reference book that was absolutely required to actually use the API.
If you don't spend enough time on design, it's easy to wind up with an overcomplicated API that lacks documentation.
I wonder if the book was just a "convenient side benefit" of Microsoft's general failure to invest in developer UX. (At least at the time.)
I wouldn't be surprised if the actual progression went like:
Senior engineers end up with a moderately complex API for various bureaucratic reasons and don't have time to document it well -> Some junior engineer trying to be helpful writes up instructions as he figures out how to use it -> Junior engineer, still trying to be helpful, can't find anywhere good to put these instructions he wrote, so he gets a book published of it -> Rakes in extra money from said book -> Senior engineers see this and think, whoa, that's a nice scheme, but why should this newbie get all the $$$, let's do it more on purpose! -> Next thing you know, all the APIs are actually more complex and everyone's got a book
>If you don't spend enough time on design, it's easy to wind up with an overcomplicated API that lacks documentation.
That sounds about right. Creeping featurism gets ahead of the team's ability to document it all before the product has to be delivered. Hence the need to take some time off and write a book about it. It's not necessarily an evil plan but just the way it works sometimes. Writing documentation is hard. Delivering the documentation on time is even harder.
I wonder that about many of the corporate web frameworks out there too. The guides and evangelist say it's so easy but they have the most dog shit docs out there but hey, plenty of training you can pay for to get certified.
It made me think I was an idiot until I talked to a couple other folks (who didn't have a financial reason to say it was easy) and they agreed it was horrible documentation.
One reason I love open source: you won't have folks clamoring to use or improve your system if it's too shitty. To reach great levels of incomprehensibility you really need corporate backing.
I recall, back when Windows 95 was released, a joke going around that went something like this: "Yes, Windows 95 is awesome and popular. So awesome and popular there are ~450 books written about how to use it!"
have a hard time believing that selling (tens of?) thousands of programming reference books brought in 10-20x salaries
or that entire teams would be following a pied piper leading to such a hellaciously complicated design, contrived to induce desperate reference buying??
That's like asking what Microsoft does. SAP has lots of different products, many from acquisitions, but the core product (what people think of when they say "SAP", although it's actually called S/4 HANA) is an ERP system. The Wikipedia article looks like a decent overview from what I can tell: https://en.wikipedia.org/wiki/Enterprise_resource_planning
Disclosure: I work at SAP, but on internal IT and not on any of the customer-visible products. (I don't know much about how S/4 HANA really works, except that I use it to view my payslip, enter my vacation days and such.)
1. In many cases the books were published by Microsoft Press, so the company benefitted financially as well.
2. The engineers may have a defences in not being given time to document better initially due to project deadlines beyond their control, and in the work being signed-off at the time.
3. Some of both the above mixed with other reasons.
4. they probably also didn't have the time for better API design, or the API design was done "by committee", and if you add to that the holy cow of "backwards compatibility", it's easy to see how an API might end up overcomplicated...
5. maybe M$ even encouraged these practices because of the perceived advantage for in-house applications vs. third party ones?
Microsoft (Press) published a fair number of these books—or at least books that were so much better than the reference docs that the latter could barely be said to explain anything, whatever the intent of the authors. This includes the classics: Petzold, Inside OLE, Essential COM, ...
They still had their monopoly, who cares? Besides, forcing people to invest time and to a lesser degree money to learn and get locked into your ecosystem is probably a net gain.
That's a direct action against the interest of their employer, and an attempt to enrich themselves at the benefit of Microsoft. They are purposefully writing bad software which drives ME market share and reputation among developers down only to sell some books.
Except if MS gets a share of the profits… which it did in many cases, being the publisher. They had no competitors, there weren't any concerns about market share or reputation!
There may not have been any concerns at the time, but the damage to the reputation is real, albeit difficult to measure. Nevertheless, you can see that Windows only ever seems to lose market share. Windows desktop is shrinking, Windows mobile is dead, Windows embedded is phasing out, and Windows server is only used for the bare necessities. You can argue that Microsoft successfully managed to pivot to cloud, but that is a rather shallow moat.
Among all other variables affecting the market share of Windows, the effect of the inscrutability of the Win32 API is almost certainly negligible. Especially given that as far as I know, their APIs designed in the past 20 years haven't typically been as horrible.
Microsoft had no real competition back then, and in their core business (desktop and office) is still quite uncontested. Alternatives to parts of the Windows, MS Office, and various corporate systems exist, but nobody offers the complete package. Extensions to their platform were never required to be perfect, just good enough that being ubiquitous (thanks to the platform) is enough to sell it.
Also, is it really against the interest of their employer to create products that lock in customers and make it harder to migrate away? Creating a market for support and customization, with MS sitting at the source of the knowledge and being able to sell crumbs by printing books and other training materials, and offering training and certification courses?
This is not a bad model in my opinion. Actually, it may be even better than the average. At least, you could buy the expertise compared to a total lack of it.
Making an API knowingly and intentionally far more complicated than necessary is not just bad, it is evil. Doing it to enrich yourself personally is akin to embezzlement.
> I don’t think people who make APIs really appreciate how important it is to get them right, and how much unnecessary work their mistakes can cause for hundreds, thousands, sometimes even millions of other programmers.
When I was a kid I tried to build a windows app. Getting a notepad clone even kind of close was a triumph for me, but I decided I wasn’t smart enough to be a programmer, and it set me back about 8 years at least.
I was about 19 and a jr sysadmin changing air conditioning units when I finally sorted out the vast majority of coding is vastly simpler and vastly more fun than win32. Those windows APIs cost me a lot of time!
In the late 90's, my Dad bought Visual Studio 6 to learn C++. I knew my way around Turbo Pascal, i386 assembler and C/DJGPP on DOS, so I figured I could try to code for Windows, "I am Big Boy now".
The Visual Studio 6 box had an introduction book to win32 and MFC ... Nothing made sense to me, a lot of boiler plate code was there without much explanation about why it was there. I just said "nope, life is too short to suffer this", and went back to DOS. Learned Unix system programming at the university, it felt sane and consistent.
This introduction to win32 API was so traumatizing, even now, I would not touch it with a barge pole, POSIX or GTFO.
I tried to switch from Delphi to Visual C++ around the same time, which made me appreciate even more how easy it was to build a GUI app with Delphi as opposed to the way VC++ did it (having a barebones visual designer and other than that forcing you to use all the gnarly APIs directly), how straightforward Object Pascal was compared to C++, and how much faster the compiler was, and decided to just stick with Delphi for the time being. The later .NET IDEs improved the usability quite a bit (thanks Anders Hejlsberg!), but those were for .NET, they didn't generate native code.
They do generate native code at installation time via the NGEN tool, and Managed C++ (later C++/CLI) can created mixed mode Assemblies, where native code is embedded on the file, which is the reason why mixed mode Assemblies can't be considered safe from CLR point of view.
> The Visual Studio 6 box had an introduction book to win32 and MFC ... Nothing made sense to me, a lot of boiler plate code was there without much explanation about why it was there.
Some of this is a symptom of the fact that Microsoft was still fumbling in the darkness regarding its Windows developer experience.
First generation windows programming was a complicated affair involving a number of command line tools, textual languages, and switching back and forth from the GUI to DOS. The first Visual BASIC product fixed all of that, and made it possible to achieve good results with just point and click and (more or less) scripting code.
My read on Visual C++ (predecessor to Visual Studio) was that Microsoft was pushing hard to graft the Visual BASIC style of point and click development onto its old school Windows development tools. The result was an IDE with a form editor that could superficially work a lot like Visual BASIC. You could drag a button, double click it, and be dropped into a blank C++ (rather than BASIC) function to handle the click event at runtime.
They did this with specific features in MFC, but also with a huge amount of default code generation. Double clicking that button in the form designer would update message maps, write a function, and make a number of other changes in the source text of what was already a large body of generated code.
The trouble is that while this worked, it didn't actually teach people what was going on. The development environment wound up being very brittle and hard to understand, particularly if you did anything that confused the IDE integration.
Do you think they took this approach because it wasn’t yet common wisdom in the industry that tons of generated code is a terrible idea, or because they didn’t want to fragment the developer ecosystem by adding a more comprehensive framework above Win32?
They introduced this system with Visual C++ v1, which is also known as Microsoft C v8. (There are nuances, but that captures the gist.)
The preceding version, MSC v7, was the first version of the Microsoft C compiler to support C++. It's also where Microsoft introduced the first version of Microsoft Foundation Classes.
My understanding of the history of MFC is that MFC is Microsoft's second attempt at a C++ class library for C7. Prior to MFC, they had developed a significantly more object oriented framework, but found it in testing to be confusing to developers of the time. (Who had just ascended the Win16 learning curve itself.) Micsoroft retrenched, and the MFC 1.0 they shipped with C7 was a much thinner layer over the Win16 API than what they had initially planed. (The earlier framework was AFX, which is why there are AFX prefixes in the MFC codebase.)
My presumption with respect to Visual C++ 1 is that Microsoft found themselves short on time and facing dual mandates of maintaining MFC 1.0 source compatibility and producing a C++ development experience a little like Visual BASIC. So a bunch of IDE trickery and codegen logic was the logical path forward. (IIRC, the Visual BASIC compatibility mandate extended as far as enabling VB custom controls to work in Visual C++ projects.)
I haven't used it, but Borland Delphi post dates all of this, and gets it better. Borland was able to specifically build a class library and extend the programming language itself in a way that supported visual design tools. My understanding is that this can give some of the Visual BASIC style point-and-click, but also makes it easier to delve into the underlying framework code and make your own components. (Also important to note that Delphi was Borland's second attempt at a Pascal based windows development tool.... the earlier Turbo Pascal for Windows was a lot closer to the Win16/MFC-like experience)
I had a similar experience. I'd been doing C++ for DOS with some x86 assembly for a couple years when I somehow got access to Visual C++ (probably version 5) with MFC. Just getting to Hello World took me a solid week. Eventually I wrapped my head around it, but moved to C#/WinForms as soon as that came out.
Boggled my mind in university that c# was so smooth to get an application running, and the same ideas could not be applied to C++, in basically the same Coding environment. Tried for 4 separate times, always threw the towel.
There was a period of time when Microsoft were very enamoured of the document-view model of desktop applications; you could choose to build MDI (multiple document interface) or SDI (single ...) applications, complete with menu management, so if you want to do a Word clone, knock yourself out. however if you just wanted a simple doohickey with a few buttons and whatnot - like the vast majority of cases - you were out of luck. There was of course a way to do this (and it became easier in later releases of Visual C++), but you had to fight the stupid M/SDI model the whole way.
Then let me tell you that programming X Windows is even worse, from the point of view of someone that knows Windows since Windows 3.0, and learned it via Turbo Pascal for Windows and Turbo C++ manuals.
Its a been a while since I did any X development but as far as I recall as an application (client) developer you generally only needed to know about the client library you were using together with some lower level details from Xlib?
Win32 programming was more akin to the X-windows protocol/API and boy.. let me tell you that Win32 was kids play in comparison to the fragmented mess that is raw X-windows (It's not for nothing that KDE is/was built on QT and Gnome on GTK).
Yes, the auto-generated boilerplates was stupid (and obscured so much) but it was their way of trying to get people ahead in an "easier" way.
When I messed around with X programming I thought the base layer was primitive, but straightforward for the most part. You had to do damn near everything my hand, but it was clean and orthogonal.
Where the nightmare started is the X toolkit. It slathered on three extra layers of complexity before providing any benefit for the programmer and was so damn confusing even when you did get it working. And what did you get out of it? Athena widgets.
It’s funny how different this was ~10 years later. Visual studio was basically best-in-class as an ide (I think) and you could throw together a simple app pretty easily just using the mouse to design the form and writing functions in C# with a bit of google or just looking through the autocomplete list. But maybe that’s a bit rosy and if one wants something more complicated, it all drops back into win32 api stuff.
I have the exact opposite experience --- I've been programming in win32 for about as long as it existed, and did win16 before that. There's no better way to make super-efficient single-EXE applications that can run on every version of Windows since Win95 (and if you try a little harder, you can get to Win3.11 with Win32s.) Everything newer has felt far more bloated and complex.
That said, some parts of Windows API are horrible; everything COM-related comes to mind.
COM is a very good idea, very badly executed, ignoring VB 6, .NET Native and C++/CX attempts at improving it.
Unfortunely WinDev seems blind to how it is perceived from outside Redmond, is keen in sabotaging DevDiv efforts to make it usable, and keeps pushing it everywhere, moreso after Longhorn.
Instead of dabbling with the windows API directly, a better option is to use the .NET WinForms API. You can use any CIL language like C# or VB.NET. Granted that it is a framework or toolkit like many others but one of the most efficient and relatively easiest to use that I've found so far. I've tried hello worlds with many other frameworks like Java Swing/AWT/SWT, Python Tkinter/PySide, Electron.js framework, etc. and none came close to WinForms API, especially when it's powered using the super efficient Visual Studio Community Edition.
Wasnt there an issue with your apps then needing the user to install the “.NET runtime” or something? I remember it being the main reason why people stayed on win32 for so long
The .NET runtime usually comes pre-installed on all machines above Windows 7. It's just that if your app needs some advanced feature, you need to upgrade runtime version in those cases, this is pretty much how other runtime based systems like Java and Python also work.
Most people who stayed with win32 for long were C programmers who preferred the "purist way". But those who wanted developer productivity soon switched to the frameworks of .NET, Java, Python, etc.
=> NET Framework 1.0 is an integral component of Windows XP Media Center Edition and Windows XP Tablet PC Edition. Installation CDs for the Home edition and the Professional edition of Windows XP SP1, SP2 or SP3 come with .NET Framework 1.0 installation packages.
I'm not really trying to prove anything, I jsut remember that at the time, doing something with windows forms was a guarantee of getting people saying "ah it tells me i have to download .NET"
I've always wondered who were the people that were responsible for such terrible Windows APIs. They must have had an axe to grind or followed some ideology.
As a kid, I wanted to make games, and just found the win32 API inscrutable. I too lost a couple years.
The MS of the 90s, being the monopolists that the were, probably wanted Win32 to be as radically and annoyingly different from Unix and MacOS as possible just to make it hard to port software.
I’ve read that MFC was a very close copy of MacApp, so not sure how true that is.
Also of course that’s Microsoft of the late nineties: e.g. COM/OLE2 originally had a Macintosh port and an obscure commercial-Unix one, VC++ 4 shipped with a Macintosh cross environment complete with an MFC port, etc. Don’t know if anybody ever seriously used those except the Mac ports of Microsoft software (Office, IE).
In the 90s, there were other options besides Windows and POSIX.
The Mac OS from the 90's had zero POSIX, and was less capable than Windows, hence Copland, which by the way also had zero compatibility with UNIX and was going to be a microkernel OS with C++ userspace.
Although fun to speculate that it must have been deliberate, it's more that Win32 originates in a time when APIs were uniformly terrible by today's standards. UNIX and classic MacOS have many of the same problems, they're just smaller APIs and people have now forgotten classic MacOS. The industry has got better at API design over time, mostly by abandoning C.
Some of the key problems that makes Win32 unpleasant to use:
1. Preferring to be consistent with past mistakes rather than fix them. Core Win32 isn't namespaced in any way, and is full of generic function names like "StartTrace" that can easily conflict with user code. 30 years later they are still adding new Win32 APIs that also aren't namespaced in any way. They still use the "Ex" convention and so on.
2. A pervasive assumption that the APIs shouldn't have any opinion about memory allocators or heaps. You get this in old UNIX APIs as well. Why does the ETW API mocked in this article require so much pointer arithmetic and memory munging, well, because Microsoft don't want to have an opinion on where memory should be held. They see malloc as a utility convenience, so nothing in the API will malloc memory for you, or if it does, then it's done using its own API specific wrappers around malloc/free. By implication Win32 generally doesn't have functions that construct objects for you, instead it's always up to you to allocate the memory and pass it in. That means the API might be called with allocations of various different sizes depending on when the app was compiled, so, then the caller is typically expected to place the structure size in the structure itself as a crude form of versioning. It's all a lot of tedious boilerplate that's easy to get wrong.
3. An inconsistent and ad-hoc (API specific) approach to versioning and backwards compatibility. There just doesn't seem to have ever been much planning ahead. Win32 is full of places where they left room to extend the API yet never used them, and places where they didn't and then needed to introduce an Ex or 2/3 call. Again COM introduced better ways to manage this problem, but they didn't use that consistently either.
4. Frequent use of random GUIDs to namespace things. The article has examples of this. Nowadays we take for granted internet based namespaces, like how Java/macOS use reverse DNS names to identify code. Win32 originates in a pre-internet era, more or less, so a lot of APIs are built on the assumption that nobody can communicate with anyone else and there are no central naming registries that work. Giant random numbers are a reasonable solution to this, but again, Microsoft stuck with it for way too long after the concept became obsolete.
COM fixes some of these things but is used inconsistently and it was never retrofitted onto the core API properly until WinRT. For example, if you want to work with Direct3D you need to use COM and instantiate some COM objects, but if you want to then connect that Direct3D surface to a window, there's no COM object to represent a window, that's all done with legacy C APIs. Why did they never switch entirely to COM? Probably internal politics - developer experience became seen as some other department's job (VC++), and Bill Gates became obsessed with Big Chunky Features he could easily understand without doing Windows programming as his day job (like WinFS).
Truly terrible APIs never originate from one mind.
Instead they're the result of a whole bureaucracy of people each of whom have to inject their own particular requirements, coding preferences and "what if in the future..." guesses.
Oh, yes, I remember reading that while back. As Joel says, "if you write convoluted, dense academic prose nobody will understand it and your ideas will be misinterpreted", which doesn't let Simonyi off the hook completely.
As for Apps Hungarian, I was just discussing this the other day on a forum devoted to John Ousterhout's A Philosophy of Software Design. In Chapter 14, "Choosing Names" Ousterhout discusses a case where a variable named 'block' was used in a place where there were both physical and logical blocks, both integer types. His suggestion, which isn't wrong, was to use different variable names for different kinds of blocks, making it possible for a programmer to tell if a logical block variable was being used in some physical block handling code. This is similar to Joel's example of using 'rw' and 'col' prefixes for variables that are both integer types.
When I see a convention like this, I ask, "why can't they be different types?". In modern languages you can define new types easily, and define different interfaces or APIs to operate on the types. With 'physicalBlock' and 'logicalBlock' types, or 'row' and 'col' types, programmers can lean on the compiler or interpreter to prevent many errors. Beyond looking wrong to the programmer, the code is wrong, and won't compile or interpret correctly, provided there is a reasonable type system for the language.
The obvious concern is a proliferation of types, and yes that can happen when taken to extremes. One area where Joel's example makes a very strong case for types is 'us' and 's'. Strings are handy, but also the wrong abstraction. A 'SafeString' type would very clearly communicate the abstraction at the proper level: the thing in question is not a string, it's a representation of encoded user input. An attempt to assign a SafeString to a plain old string would be an error. Passing a string to a function that declares a SafeString would be an error.
There's even a name for using ints and strings when different types would be better: Primitive Obsession. A sign of this obsession would be passing around a user ID and password as strings, and a corrective would be a Credential type that encapsulates both ID and password types, and includes the necessary parsing and validation on creation. This eliminates an entire class of errors much more strongly than a convention where every string that contains a password has a 'pw' prefix, and userID strings are prefixed with 'userid' or just 'id'.
Of course, if the language doesn't provide useful typing, then naming conventions are essential. In languages with good type systems, always be asking, "does the language's primitive type really represent the abstraction in use here?"
Maybe ironic but I wouldn’t be where I am without Windows APIs.
I was 19, and I was a member of some forums where folks shared knowledge on how to (ab)use various technologies for profit; many of these technologies were Web-based. I didn’t know any Web programming, but I knew VB.NET from some community college courses (with some fundamentals from high school) and I figured out how to drive this browser and interact with pages.
This allowed me to automate various Web interactions and make a few bucks writing code, which eventually led to my current career as a professional software developer.
I was getting paid (not much lol) to build apps before I knew either of brands like “port” but I hope that, no matter how much money is in the table, y’all feel good about every deal! Respect
Oh God you just stirred a long forgotten memory of my youth. Trying to make custom-shaped windows.
I also felt it was unreasonably hard in MFC/Visual C++ to just have a button in a different colour. The RAD system was so barebones compared to what I was used to previously with Visual Basic, and the people in Delphi could make vibrant, colourful GUIs which were a pain in the butt to do in C++.
I will always remember Delphi as the cool RAD system that could create beautiful GUIs but the language is a bit ass, so I never took the time to learn it. (I have never liked the feel of Pascal for some reason)
I looked at WPF a few users ago and my impression was, I could spend 2 hours doing something I could easily do in the browser in 15 minutes. Maybe it's not that bad once you get the hang of it, but it looked so arduous I immediately decided I didn't have the will to go farther.
To be fair, making an app in 2008 (when WPF released) with JS was a completely mess. WPF was usable and they introduced data binding/MVVM. In 2018 (React/Redux), it's fair that JS seems to be advanced vs WPF. Though its productivity isn't so bad (unless you want to fully customize everything).
Also on "to be fair", JS libs have a nasty habit of either completely redoing the API, or dropping maintenance entirely and encouraging everyone to switch to the new hotness which also has a completely different API, every 6 months or so. It's a lot easier to have a nice API if you get to rewrite it on a whim and tell anyone who doesn't like it to fork off. WPF may be weird and messy, but your app will still work unchanged even now (well I think, haven't used it in a while).
I don't even get this arduousness argument. A barebones WinMain is extremely long-winded (what causes most people to "nope out"), certainly, but once you get passed that it's fairly consistent and the API calls reasonably formed. I don't think it's an example of a perfect API, but the concepts and functionality of it mean once you learn the basics of it (handles, resources, parentage, text encoding, type annotation, etc) they apply to any other widget you'll implement, from legacy ones all the way back in 1993 to modern ones introduced in Windows 7+.
I have the opposite experience: I learned to program with Delphi, and all the React libraries feel like hard, slow ways to make stuff I'd make in five minutes in Delphi.
The simple distinction is this (using abstract examples):
* Declarative: here is a UI that I designed ahead of time. I've set up these controls with these hooks. Your logic then plugs into my UI independent of it's design. (When myBtn is clicked, it fires MyCode::MyBtn() in your code).
* Imperative: here is my main.c. In my main(), I call a function createWindow and then createButton with various parameters (size, text caption, styling, etc) that returns a handle to the widget/window. I give a callback to my widgets. When that button is clicked, the callback is fired. I can call a function called destroyWidget with that handle and erase it.
Or, to simplify solely to definitions: in declarative UI, I create (or declare) the UI ahead of time and hook it into my logic. In imperative UI, the UI is rendered as requested (an imperative). The definitions are directly orthogonal to imperative vs declarative programming, in general.
As to "reactive", this is an independent descriptor that simply means that the UI adjusts to it's environment. It's usually used in the context of Web or mobile devices, since an Android app that appeared the same on Phones and Tablets would offer a subpar experience for users on one or the other.
Given an imperative UI and a small amount of time (depending on the surface area of the API) I can create a declarative interface to it.
I cannot do the reverse given a declarative UI.
My observation of declarative UIs is that, sooner or later, due to necessity, they tend towards the inclusion of imperative elements ("If we are running as Super-User, then $FOO must be a button, else it must be a link to a page requesting Super-User privileges").
At the end the "declarative" UI is declarative for only the most trivial usages. Any non-trivial usage includes hacks or workarounds to implement conditional elements, loops (for collections of elements) and subcomponent specialisation (for element reuse).
I don't think the statement about declarative UIs needing hacks is true. In Flutter or Elm for example, you can write something like:
if condition then
ConditionalWidget1
else
ConditionalWidget2
Where this if-statement is embedded in a declared UI/widget tree. That's hardly a hack in my opinion.
There are similar ways of building a dynamic list of widgets declaratively which don't feel hacky at all. For example, look at ListView.builder[0] in Flutter. You provide an array and a template function of how each element in the array should look and then you have your dynamic list of widgets. (Of course more complicated use cases require more complicated code, but it's still not hacky and fits well with the larger sstem.)
Could simply be that my understanding of declarative I a wrong.
AIUI, declarative is listing what you want, while imperative is listing the steps to get what you want. Naturally, I feel that extending a declaration with conditionals or looping constructs make that not-declarative anymore.
That's because, I feel, a conditional is a sequence of steps, not a declaration.
Hence I regard (perhaps incorrectly) that declarative syntax with support for conditionals is "hacky".
I can understand that point of view. It seems likely that UI-as-code variants (like Flutter and Elm) are more imperative (even if they have a declarative API) than separate markup languages like HTML and XAML since you have the ful power of a programming language to express what you want.
(The XAML way of expressing conditionals and loops does feel hacky to me since you need the code and markup to communicate with each other instead of expressing directly in code.)
For me it was the opposite. Raw Win32 is low level: you manipulate the event loop directly, and you have to deal with the layout yourself, but otherwise rather simple.
MFC had a mix of generated code and handwritten code. Writing MFC code by hand was hell, and using a graphical IDE was nowhere as convenient as Visual Basic or anything by Borland.
When teaching myself as a kid, I quickly realized that win32 was way over my head. However, web stuff seemed easy enough, so I started teaching myself web development instead. This was in the early 2000s, before everything started moving to web applications. It certainly managed to pay off!
This was also me as a kid with the Mac - we got our first home computer in 1993. Pascal? WindowPtr? Compilers? Yikes - I'm only 9 years old! I'd done some QBasic and Logo at school but couldn't wrap my head around the 'abstract' GUI stuff: If I tell the turtle to go Forward, it goes forward...
Then I discovered HTML - much easier to wrap my head around at the time.
I remember trying some Win32 stuff in the early 2000s and being gently ribbed in a chat room for not knowing what a message pump was. At the time I was thinking I was so junior, I didn't even know what something so commonplace was. Now... not so much.
I'm convinced that the only reason I got into programming is because my first exposure to it was via HyperCard, which is still just the most friendly and enticing introduction to programming I've even seen.
> When I was a kid I tried to build a windows app. Getting a notepad clone even kind of close was a triumph for me,
I remember having similar struggles. Windows programming was a huge lift. Not only were there new API's to learn, it was really an entirely different and novel architectural style.
Kerry Nietz writes about the experience of porting FoxPro from DOS to Windows... his description is apt: "Turn the program inside-out". For me, that was hard to do at a time when I was still working through how to make a normal program.
Constantly reminded of this every time I see any Win32 API code. Horrid names kept me away from native windows development as a kid. They didn't make any sense to me, which I found very aggravating. Having 30yrs under my belt has only convinced me more of how horrible that shit is.
ETW is pretty bad but, for Windows APIs, it feels pretty middle-of-the-pack to me.
OpenSSL, on the other hand, is an endless nightmare to work with: clusters of APIs that are behaviorally similar (retrieving the attributes within an X.509 certificate, for example) have wildly different function signatures. Even when function signatures for similar APIs resemble each other, their interior behaviors vary widely (around assuming that lengths include NULs, maintaining or not maintaining interior reference counts, etc.). Seemingly basic functionalities ("turn a buffer of PEM-encoded certificates into a `STACK_OF(X509)`") have no APIs at all; other APIs solve simple tasks through obscenely overpowered (and dangerous) primitives (such as `X509V3_EXT_conf`). And so on.
I'm building an app that uses the Web Crypto API and the annoying thing about cryptography is that in practice you're calling a ton of different algorithms with slightly varied argument, which of course you have to name, and makes code reuse practically impossible.
You can have an `encrypt` function, that takes an RSA key, but wait, if you're encrypting with AES-GCM you also need a salt and an IV, that other times is called a nonce (which is similar to an IV but not exactly). But wait, if you're encrypting a key, you're actually "wrapping" a key, so your API should clarify all these little variations in process, which is a pain. Given that the error conditions are very opaque, to avoid mistakes you need a very clear and verbose API on top of the bare crypto functions to stay sane.
Literally my morning plan for today is refactor the cryptography functions in my project because they're an unwieldy mess.
Would be pretty cool to use multiple dispatch, a la Julia. The encryption function knows which one you're using based on the types you pass as arguments.
I would not find that cool. I think for a typical crypto use case:
(A) you want to use exactly one encryption method (eg no algorithm negotiation)
(B) you want to be very careful about calling the right functions with the right arguments
Relying on compiler magic doesn’t sound good for that to me. Though the grandparent comment does seem to go against (A) so maybe the situation is different.
For many years (since 2014) the API for Metrc (the cannabis reporting engine in Colorado and other states) would NEVER return any useful data at all, simply an empty JSON array. So after each operation, one would have to query the search endpoint for plants or inventory to discover the new items -- and hope you could positively identify which one was the new one you were looking for.
That and not supporting B2B transactions via their API. They would alternately claim it's for security or that the States were not asking for it. States would tell other software vendors that Metrc coludn't do it. Neither party could move the ball.
Only now, in 2023, have they started responding with the ID of the newly created items (why not a fully inflated record of what was just created) and working to support B2B.
These state run APIs, from a for-profit corporation, are not good for the public (they hide, or cannot handle important laboratory testing data) and they are not good for the licensees, especially the small businesses. And there is no accountability -- agency points to vendor & vendor points to agency.
Sadly, getting these State agencies to understand the beauty of distributed/federated systems -- and the difficutly for small businesses to respond to RFPs -- keep us from having nice things.
My favorite Metrc API wart was submitting harvests when their API servers were bogged down. Sometimes, your "create harvest" request would get stuck in a transaction pool that wouldn't timeout for 10 minutes. You'd have to wait the full 10 minutes to see if it succeeded or not because setting a shorter timeout would risk double-submitting the harvest, thereby completely screwing up that cannabis business's compliance data.
Oh, if only it were that simple across the board! One of my favorites was getting any number of errors (timeouts, various internal errors) only to discover later that the reporting did sometimes, eventually, unpredictably succeed despite whatever error had been returned and regardless of how fatal the error message might have seemed to be. And often this occurred during waves of intermittent outages, where determining any given success might take hours. And in any case, correlating success for some resources was partly guesswork, matching data to the timestamps we reported, which of course lost fidelity in their system because… SQL Server stores it that way, I guess.
That reminds me of a certain dataset (public transport timetables) managed for the Czech state by a private company (CHAPS). They have a duty to release the data for free, but instead they only release near-useless cut-down versions. If you want usable data, you have to pay them for the "enhanced" version.
The ministry that oversees this is not interested in changing anything, so despite external efforts, this has been going on for almost a decade.
Yes, I can't think of an API has caused me more grief than METRC's.
Recently, I discovered that they do some strange XSS protection on text fields that sanitize any appearance of the characters "SCRI" appearing in sequence. I was scratching my head wondering why the "described" we were sending through the API turned into "debed" on the record itself.
> Whenever you evaluate an API, or create a new one, you must always, always, ALWAYS start by writing some code as if you were a user trying to do the thing that the API is supposed to do. This is the only way to get a nice, clean perspective on how the API would work if it had no constraints on it whatsoever. If it was “magical”, as it were. And then, once you have that, you can move forward and start thinking about the practical problems, and what the best way is for you to get to something implementable.
Such good advice. It's so obvious but so easy to miss when you're excited to get things done. I think it's like writing a unit test before writing the code.
It changes your POV from an "API creator" to an "API consumer" - for whom this whole thing will be all about eventually.
Agree; it also helps to write documentation supporting the example. On several occasions I ended up reworking APIs I'd been working on after writing documentation for them, as some of the elements were too complex, looked inconsistent, or just didn't feel right.
I find TDD antethetical to this... you really need to actually understand the problem space - that requires large scale experimentation, not the micro-structuring TDD preaches.
> Maybe we should invent a paradigm and write a manifesto about it. We can call it... test-driven development.
Isn't test-driven development the opposite, though?
If the program is layered as layers A (main program), B (business logic), C (data objects) and D (persistent storage), TDD would have you first do the interface (and tests) for D before you do anything for C.
You only realise if D is missing something (or worse, has superfluous features) once you start doing C. Same when you get to B.
When going API first, you go top-down. You first figure out what the interface to the main program (A) should be, then you figure out what the interface to B should look like, etc.
I always have to program from both sides, and hopefully meet in the middle. You need your low level stuff before your high level stuff works. but you don't know what low level things you will need until you do the high level things.
My understanding is that TDD would start with the main program (A), testing functionality that is exposed to the final user. The later layers would only be written if and when they are required for the end-user functionality.
I think that User-Centered Design (UCD) would be a better term here for what this concept is trying to say. In modern design, we tell user stories that help guide development of interfaces, including APIs.
Too many developers put too much emphasis on the "A" (application) and too little on the "P" (programming, which should really be thought of as "programmer").
An API is a user interface in every sense of the term. Programmers, not abstract "applications," are your users, and so the same care taken for graphical UIs should be taken in designing APIs — e.g., account for your user's mental model, consider the outcomes your users want to achieve, consider the outcomes your business wants to achieve
I used to be a Windows programmer and dealt with APIs like this all the time.
Honestly, the Event Tracing API is consistent with all of the Win32 APIs. They're pretty freaking horrible. It's like filling out a bunch of tax documents to do anything.
I think this explains ETW's horribleness pretty well:
> Now, normally, a game developer would have no reason to use the Event Tracing for Windows API directly. You can use tools like PerfMon to view logged information about your game, like how much working set it was using or how much disk I/O it did. But there is one specific thing that directly accessing Event Tracing gives you that you can’t get anywhere else: context switch timing.
It's been awhile since I've run wpa and wpr but I thought you got context switch timing these days with the built in tools such that there's no need to do it yourself. IMO having a good ETW API is probably low ROI.
There's also been xperf and Process Monitor for a long time. It's not clear to me from the article why they needed to use the underlying API instead of the tooling built on top.
> Whenever you evaluate an API, or create a new one, you must always, always, ALWAYS start by writing some code as if you were a user trying to do the thing that the API is supposed to do.
I’ve been telling this to the developers I have worked with for the last 15 years. Similar to Amazon’s “working backwards” concept, the only way you can reliably get a good API is to engineer it from the customer’s point of view. That means write the code that you want your customers to write (or that you would want to write, if you were the customer) and then engineer backwards from that.
One of the biggest problems with API design is that often the developers of the API have a vastly different mental model of the functionality that they’re encapsulating in an API, than their customers. Often, customers can’t even develop that mental model because the encapsulated code is a black box.
Another common API design problem is that you’ll just write a simple CRUD API around your object model. This often causes customers to have a bad experience if they use objects in conjunction with each other, and have to do a lot of housekeeping to associate objects with each other and keep track of multiple objects to perform simple tasks. My example here is AWS Route 53; the API is a real pain to use if you just want to do something simple.
I agree with you, but I will say, worse than no abstraction is a poor one. Give me CRUD any day over something that doesn't expose things I need or makes me jump through too many hoops to get something simple done. At least then I can build what I need.
It's really on-brand for Microsoft Windows APIs. They're all about making you fill out a giant tax form of a struct and passing the OS a callback. It's kind of a house style that Microsoft adopted back in the Windows 3.x or before days and it stuck.
My favorite is the Dir() function in Visual Basic which is used for directory walking. Only it's non-reentrant, so you can't walk subdirectories with it! Near as I can tell it's traceable back to DIR$ in GW-BASIC, which in turn was an exposure of DOS's broken, non-reentrant GetDirectoryEntry and NextDirectoryEntry system calls! DOS will just never die!
Which is why they can keep the OS ABI compatible, while at the same time evolving that struct.
By having a size field, the OS can know for which version of the struct the application was compiled for, and what fields it expects to have in that struct.
I don’t think that’s a common complaint in itself. Sure, new people wonder why that sizeof line is needed, but once you learn about ABI compatibility that makes perfect sense.
People here complaining about Microsoft. I fully expected SAP. I think every API is reflecting the organization it was developed in. Everything is based on top of something older, bureaucratically refined.
Win32 definitely has some stinkers. But, video encoding APIs definitely take the cake for me. I've only tried a couple of FFmpeg's APIs and AV1's API, but my God, these are the worst API's I've ever had to deal with.
Just as an example, all I wanted was an API like this[0] for FFmpeg. In order to implement that API (which in my opinion is reasonable), I had to write this monstrosity[1]. It took me a solid week to find an example of how to do this, then another few days of fiddling until I finally just barely got something working. Then I threw in the towel even though the performance was horrible. I tried again a year later and spent another month wrestling with AV1 :/
The amount of leakage going on in these APIs is absolutely insane. I shouldn't have to know the intimate details of how video encoding works to use your library. If I do, then I may as well write my own encoder at that point.
Haha, yeah. In ffmpeg's sort-of defense, the libav* APIs are supposed to be semi-private, which is how they try to get away with them being terrible & unstable. But in practice they're public, and everyone uses them, because the only alternative is gstreamer and oh boy do you ever not want to dive into that mess.
FWIW if you just want to en/decode AV1 and don't care about anything else like container formats, you can just use the codec directly. Dav1d is the current gold standard, I believe. Even some container APIs aren't too hard to deal with, if you do need that.
> If I do, then I may as well write my own encoder at that point.
I've actually done this with jpeg & mpeg1. I spent maybe a hundred hours trying to get existing libraries to play by my rules. When you completely own the implementation everything gets out of the way.
I don't like that I had to do this, but it was the only way to wrap my head around the problem and bend it to my will.
I worked at Microsoft early-career and one of the projects I worked on was to integrate with ETW from a Go program to plumb events from Go code into ETW. I would also agree that this is one of the APIs that take a really long time to reason about it and some parts just feel like this API should not've been exposed to anyone who's not on the Windows team at all.
Although it's a single API, the application event tracing doesn't seem to have much in common with the kernel event tracing. I got the impression that they were separate projects, someone made the decision to jam them together, and it's just a mess. But I have no inside knowledge.
Thank you for always bringing previous discussions up. It's nice to get a broader view of what people think, especially when there's little activity in the current thread (which isn't the case here but sometimes is).
I think most of it is made up but I’ve also seen a database schema and usage pattern (in an enterprise / MSSQL application) that’s far worse than anything I’ve read on that site so maybe it isn’t. But a lot of the stories have this ring of made-upness to it.
Why would someone put an example of how to delete the database in the comments and it happens to exactly line up with whatever comment based pragma system "Tom" developed?
Yeah, gloves are off for something so performance sensitive, and that also has to be backwards compatible with eons of older Windows versions. Who cares if it's not more intuitive.
That said, certainly there should be some decent documentation with working examples. I assume there are not hence the author's arduous trip down shonky lane.
I had to build and edit spreadsheets with the OpenXML SDK.
I have absolutely zero clue what the hell microsoft was thinking with this. Most of the use cases don’t give two rats shits about the different “parts”, but the api forces you to manually create them anyway. Why? Presumably so you know the parts exist?
Never mind that some, but not all (of course) formatting parts of the api require undocumented magic values in magic slots.
I don’t know if it’s as bad as this event tracing, but I do know that OpenXML is by far the worst api I’ve ever personally been abused by.
OpenXML in MS products was a "tickbox feature" to fight the spread of OpenOffice formats. They didn't expect people to use it for real; in fact they'd probably prefer that you didn't.
> If you simply want to receive a list of the context switches, even if only for your process, it must be running with full administrator privileges
That's actually a reasonable security requirement, greatly narrowing attack surface. Note that ETW API was designed before CPU timing side channels like Spectre were a thing.
Dealing with the EventLog API was when I realized how awesome Golang is.
As a new golang developer it took minutes to hack the Elastic Beats code to develop a PoC. That's when it ocurred to me that if it were written in C I'd still be fighting with header files for days...
The worst API I ever interacted with was the RingCentral Fax API circa mid 2000s
It was just a URL, you could POST a document at it and it would respond with 1 or 0
There was no debug information, no status indicators. Just 1 or 0
In retrospect maybe it wasn't the worst, at the time I was used to status messages and more rich APIs. It actually did function, and it did what it was supposed to so maybe I should take back my comment.
Worst status messages maybe? Things like this matter and can't just be an afterthought.
Ya gotta create an environment with mdb_env_create. So okay, that is ready to use, right? Constructor has been called, we have an object, right? Nope! You have to open it now, with mdb_env_open.
But what the above page doesn't tell you is that mdb_env_create has flags. Oooh yeah. MDB_FIXEDMAP, MDB_NOSUBDIR,
MDB_RDONLY, MDB_WRITEMAP, MDB_NOMETASYNC, MDB_NOSYNC, MDB_NOTLS, MDB_NOLOCK, MDB_NORDAHEAD, MDB_NOMEMINIT, ...
What are all these for? Should you be using any of them? Are the defaults good?
So, once we have an env with the right flags, we can mdb_txn_begin. This takes flags again. Luckily, only MDB_RDONLY applies. Probably ....
So we are ready to transact now, right? Nope. "Once a transaction has been created, a database can be opened within it using mdb_dbi_open()." [Which has seven flags!]
But wait, the documentation hastens to add: "[a]lso, mdb_env_set_maxdbs() must be called after mdb_env_create() and before mdb_env_open() to set the maximum number of named databases you want to support."
Finally, "[w]ithin a transaction, mdb_get() and mdb_put() can store single key/value pairs if that is all you need to do (but see Cursors below if you want to do more)."
Ah cursors. Here, mdb_cursor_create doesn't have flags, phew! But there is an integer op code. mdb_cursor_get is basically the ioctl of database access.
MDB_FIRST,
MDB_FIRST_DUP,
MDB_GET_BOTH,
MDB_GET_BOTH_RANGE,
MDB_GET_CURRENT,
MDB_GET_MULTIPLE,
MDB_LAST,
MDB_LAST_DUP,
MDB_NEXT,
MDB_NEXT_DUP,
MDB_NEXT_MULTIPLE,
MDB_NEXT_NODUP,
MDB_PREV,
MDB_PREV_DUP,
MDB_PREV_NODUP,
MDB_SET,
MDB_SET_KEY,
MDB_SET_RANGE.
BerkeleyDB is similarly awful to use, with a billion little settings to configure before you can do anything useful, and endless fun ways to corrupt your database. It must be a thing with key-value store developers.
The Perl core module for it isn't bad, it's one line to tie a hash to a DB file and then you can use it as normal. The main flags are to determine if it's RO/RW and whether to create the file if it doesn't exist.
Every flag you don't use is more than what you need or want and could be a compile-time configuration option to remove the associated code which implements the flag.
Functions represented as integer codes that go through a common dispatch function similar to ioctl lead to compiled code from which the linker cannot remove unused functions.
If instead of a MDB_GET_BOTH_RANGE opcode to mdb_cursor_get there was a mdb_cursor_get_both_range function, then if you don't use that function, it could be removed out of the executable. (If it's put into its own object file, and the database library is statically linked.)
Oh, please let's not this go into another "compile time" flag, because I've seen enough where this goes. Unless you are doing strictly embedded, or whatever clearance software, you don't need this "compile time" thing.
(I'm speaking from experience, having and still having to deal with tons of #ifdef code - gamedev, great for shipping to a console, awful for the rest of the code - game tools).
If we specify FOO_BUILD_TIME_FLAGS symbol which expands to a bitwise OR or various flags (or maybe just zero!) then the flag checks in the code all become constant expressions, subject to dead code elimination.
If we don't specify that, then there is a flags field that is checked at run-time.
I'd rather have most of the flags go away. Pick a behavior and hard-coded. Those who want it some other way can UTSL: use the source, Luke.
If you have 8 flags, that's 256 combinations to test for interactions.
Thanks for the chuckle. Last week I tried using the Win32 APIs to make a little ssh/sftp file browser, gave up, and started rewriting all my work in Rust. A half-baked fledgling Rust GUI library with a short gitbook's worth of documentation is more usable than the API Microsoft has been shipping for... decades?
> is more usable than the API Microsoft has been shipping for... decades?
Those things are directly related.
The Win32 API is the result of decades of accumulated cruft. Mistakes made 20 years ago must be supported forever. Rust APIs are the result of learning from those decades of experience and doing to better.
It's amazing what can be done when there's basically no technical debt to grapple with (the Rust GUI library I'm using is called dioxus, by the way, and it's brilliant). At the same time, I'm often perplexed by the nonsense that gets pushed as "modern software design."
And I tried to use the ETW API in the past and had a similar kind of wtf, I am not touching this feeling and decided to just not profile what I was profiling.
At that time no MSDN documentation made sense and there weren't any blog posts explaining the proper usage.
I suspect this post has just become the definitive documentation for ETW.
Good guess but it wasn't varargs, those were default value overrides IIRC. Thing that was objectionable it was that each an every argument was Object type.
That package dates from 2016, and this article was written in 2014. In those ancient days, you had no choice except to use the ETW api directly.
Worse, the existence of a nice wrapper that smooths out the API doesn’t actually make the API itself any better. The existence of the wrapper is itself rather damning.
As Raymond Chen would say, this API was designed with "kernel-colored glasses".
The StartTrace() is designed to notify the kernel that it should hook different locations to begin recording events. Since all kernel objects have to be installed in the namespace (just like creating a file in Linux for your terminal or in plan9 for network sockets) you must specify a name; tracing cannot be anonymous.
As with processes having the CreateProcess() and OpenProcessHandle() function pair, there is -- consistently -- a StartTrace() and OpenTrace() function pair. The OpenTrace() takes a name so it can look up the trace in the namespace.
StartTrace() could -- and should have the ability to -- fail. Kernel tracing is a privileged function, which is why it requires Admin privileges to start. By having OpenTrace() separate from StartTrace() you can have the StartTrace() function called from a Windows Service running with elevated privileges on behalf of a user-mode process, and then the user-mode process can call OpenTrace() and still be able to read the events. In fact, if the user-mode process knows that a trace is already running it doesn't even need to call StartTrace(). This applies to non-kernel traces as well as kernel traces (remember -- the name of the trace can be looked up in the namespace).
Requiring a callback -- from a kernel perspective -- also makes sense. Imagine what would happen if the user-mode process was responsible for reading events. The kernel is busy pumping events out into the buffer and, just then, the user-mode program gets hung waiting for network I/O. What happens? If the buffer never drains, maybe the kernel gets hung waiting for space (system deadlock). Or maybe the processes "misses" events, and the developer thinks the API is broken. Or better yet, what happens when multiple processes A and B are reading from the buffer and one of them (process A) pauses for a GC? Does this mean that process B gets blocked waiting for A to read events? Or does the process B "lose" events because A doesn't consume them?
The intent for having a callback on a dedicated read-thread is supposed to be like the OS handling a processor interrupt: do the minimum amount of work to record the interrupt details and then return to the kernel. In this case, the callback should copy the data into a process-specific buffer and notify a different thread that data is reading (classic producer-consumer / publish-subscribe behavior). Anything more than that and you threaten the stability of the system.
The overall architecture is both a workable design and consistent with all other Windows APIs. The problem is that the API is not user-friendly (the packed structure is required so that all required data can be passed to the kernel, and the kernel does not need to reference user-mode memory -- but still, packed structures are no fun!) and there are too many fields in the ETW structures such that users will be easily confused. Fortunately, however, most of the fields are zero.
i think the credential provider and LSA interfaces of windows internals (3+ string types why not) are a lot more gnarly than etw because of several opaque datatypes and generic handlers etc
My team was attempting to figure out some extremely obtuse workflows with MediaFountain and DirectX.
We had a meeting with this guy and a couple of the other engineers that wrote the APIs and the implementation underneath.
This particular guy literally chuckled as we aired our frustrations and this was his response:
“Yeah you won’t figure those APIs out from the documentation. It was on purpose. You have to go buy the book.”
Proceeded to explain to me that this was how he, and many other core Windows engineers lined their pockets for years - write complex implementations, do the absolute bare minimum documentation, then take a 6 month sabbatical and publish a reference book that was absolutely required to actually use the API.
Apparently many of these guys made 10-20x their salaries on this grift and it didn’t really stop until the mid 2000’s.