Did Microsoft really reserve secret APIs in Windows?

Question

In the nineties when the antitrust case was building against Microsoft, various accusations of abuse of monopoly power were leveled against the company, some of which were proven true.

One of the more persistent accusations was that Windows contained some useful APIs that Microsoft kept secret, for use only by its own applications, to give them an unfair advantage against rival applications. So this presumably implies that e.g. Microsoft Word was better than WordPerfect because Word was calling some undocumented Windows API functions that the WordPerfect developers didn't know about.

This always struck me as rather odd. What, exactly, would those secret APIs do, that you couldn't do with the documented APIs? And the Windows division was trying hard to make their operating system attractive to third-party developers. (This was not just pro forma. They were really trying, with much success. "Developers, developers, developers!") Would they really be willing to degrade the quality of their operating system, for the sake of their buddies over in the applications division? That would require a level of interdepartmental cooperation that would be at least mildly unusual in a company the size of Microsoft.

I don't remember the claim ever being definitely resolved either way, just fading away as the industry moved on to focus on other things. But I might have missed something.

Did Microsoft really reserve any such secret APIs for its own applications? Or – it's hard to prove a negative across the board, but were any instances of the claim ever resolved in the negative down to a level of detail like 'oh yes, Word did poke such and such implementation detail of Windows, but this was invalid, and the Windows team was just as miffed when they found out about it as they would've been about a third-party developer doing it, here's an internal email transcript to prove it'?

Answer 1

The 1992 book Undocumented Windows: A Programmer’s Guide to Reserved Windows API Functions by Andrew Schulman, David Maxey, and Matt Pietrek, lists them. Its section on “Microsoft’s Use of Undocumented Windows” begins on page 28, and it also has a discussion of the then-ongoing FTC case against Microsoft.

These were mainly used in programs such as anti-virus software, however. Microsoft Word did not beat WordPerfect because of any secret API.

There’s one other case, then recent, the book discusses that ambiguously falls under this: the “Geary incident.” Microsoft did not provide a documented means to hook into its font rasterizer, so Adobe Type Manager scanned the system DLL and hot-patched it. After an Adobe programmer, Michael Geary, posted about this trick on a Compuserve forum, Microsoft both accused him of revealing trade secrets and changed Windows so that the Adobe product would patch a block of decoy code instead. They might have wanted to ensure that ATM did not accidentally modify the wrong block of code in some later version of Windows. While this did not involve creating any secret API, and it’s understandable that Microsoft did not want to be locked into supporting hot-patching of the kernel binary by a third party, it did have the effect of preventing a competitor from selling its own improved font renderer for Windows.

Answer 2

It is absolutely normal for large software products to not expose quite a number of functions (note I'm deliberately not calling these API functions) to customers. Any software has library functions used internally only that you don't see fit to be exposed to end users and only to your internal developers for various reasons (the most blunt but still legit one being "you simply don't feel like it").

Exposing generally means a lock-in (you'll never be able to change anything from that point on because 3PP apps depend on it) and need for documentation suitable for external customers, so development teams are generally very hesitant to officially expose an API beyond what marketing absolutely asks for for external usage.

So, all in all this is perfectly legit and nothing special. The story becomes a bit more hairy when your company produces both the operating system and applications for that OS competing with external developer's products. If you allow your internal application team to use internal OS functions (i.e. not cleanly fence off your application developers from your OS developers), your app development team starts to gain an illegal advantage over competition.

Quite a bit of Windows functionality comes in DLLs, and, because DLLs can easily be analysed for exposed functions, it's easy to see the difference in sets of exported and exposed (documented) DLL calls. If I recall right, there never was any proof that for example, Word for Windows used any non-documented API calls. There are, however, numerous proofs where external applications used non-documented calls into Windows to achieve special stuff you couldn't do with the official APIs, the "Geary incident" and Adobe being one of the more famous ones.

I'd say "Hacking" into an OS was much more common and also much more commonly accepted 20 years ago than it would be today where it's really considered more than a bad practice and would certainly not be accepted in a commercial product - The anti-trust cases against MS were, while generally somewhat of a blessing for the industry as a whole at that time, in retrospect lack quite a bit of proof, and would today maybe end up quite differently.

So, to answer the question from your heading: Yes, definitely. To answer the (similar) question from your last paragraph: Probably not.

Answer 3

I suspect it was less about "secret" APIs, and about the fact that Microsoft controlled the entire end-to-end system, so they could compel one division (e.g. Windows OS) to accommodate the needs of another (e.g. Office suite), in a way that external companies could not compel that kind of cooperation or accomodation of the architectures of their own application suites.

It also probably wasn't that the API accomodations made were kept particularly secret after release, but that Microsoft knew internally that they existed (and the reasons for creating them) long before the results were released to the public, so that Microsoft were always ahead of their competition.

And the accommodations that did exist, accomodated only applications with the same architecture as Microsoft's application architecture. So if a competitor did things differently, the accomodations that actually existed might not be suitable for their application.

It might be rose-tinted glasses now, but it seemed one good thing about Microsoft's monopolisation and subtle crushing of competition, was the degree of consistency, integration, and standardisation that arose through the 90s.

Answer 4

While many points can be made about Microsoft's anti-competitive behaviour, I do not see much evidence in the Windows API. It should always be considered that there are

• official (aka documented) API calls which are guaranteed to be supported - and have a high chance to receive continued support in future versions and
• internal ones, usually not publicly documented, which may or may not be present in any future version and may or may not keep their structure.

A good example are functions in the Win32(s) system. Many functions provided by KERNEL32.DLL are based on the very same function provided by NTDLL.DLL. They're prefixed by 'Nt'. Both can be called from user level and will work fine, except only the KERNEL32 version is documented for general use. In addition there is often another version prefixed 'Zw' which is again the same, but intended to be called when in kernel mode.

The functions provided by NTDLL.DLL are often called the 'Native API', as functions in other DDLs are often just wrappers and redirectors to them. In fact, for many of the 'undocumented' functions of NTDLL.DLL documentation is freely available as part of the Platform DDK, intended for device driver programming (*1). It's just not the stuff for average hello world programs.

Every OS has lots of internal functions which are not intended to be called directly from user code. This includes of course Windows as well. Usually they are less visible, but the nature of Windows being build as a set of dynamic libraries means that anything between these libraries (and in many cases within) needs to be exported as a symbol. Thus it's extremely easy to look into a DLL and see many names not to be found in programming manuals outside of MS. Must be all secret and made to hinder competition, right?

Not really. But a series of articles and books did go along with that notion ... after all, everything sells better if called a secret. Those did a good job in supporting a vague notion of evil scheming.

Now, as with every good conspiracy story it gets better as more details can be made to stick. For example with certain hooks provided for extensions which were used by MS and third party software but not freely documented. One example are file system hooks for virus detection. Besides not being exactly end user material, keeping them low profile does make sense.

As Davislor already mentioned there were cases where MS changed code on purpose to screw third party applications (*2). But there are as well cases known like that of Borland's JBuilder which hacked its way around a dialogue function, leading to MS changing later versions to comfort this. So this might have worked both ways.

---

*1 - Same was already true before Win32(s). The DDK exists since Win 3.0 and documents some 'internal' functions since then.

*2 - Although I'd rather call that self defence, as changing Windows code isn't exactly a stable interface.

Answer 5

In all large software projects, many functions - most functions - are not exposed publicly. Isolation of complexity, as well as isolation of instability, are critical in even medium-sized software projects.

The publicly exposed functions provide the API that the company guarantees will continue to work in the future, while the hidden functions are an implementation detail that is very likely to change, and relying on it may break.

Microsoft has a history of working absurdly hard to prevent 3rd party software from breaking even when they use the public API wrongly, have bugs in their program, or exploit non-public functions.

This always struck me as rather odd. What, exactly, would those secret APIs do, that you couldn't do with the documented APIs?

It's not so much that you can do different things, it's that your code can run faster.

Because of security sandboxing between applications - mostly to prevent Windows from crashing when a program crashes, but also to protect from viruses - any time a program crosses the boundary between Application Code, System Code, or Kernel Code, there is a teensy bit of a slowdown, that adds up if you're making a lot of system API function calls.

If there is undocumented functions that allow you to do something in bulk, or to do in one function call what it might normally take three calls to do, it can have substantial performance increases over the course of hundreds or thousands of function calls.

Public success of software can definitely be swayed if they think your software runs smoothly, and your competitor's interface is "laggy". Half the popularity of Apple is in making their interfaces appear smooth and responsive even on slow hardware - they really prioritize the "feel" of using the software, and are known for making it not feel laggy.

That being said, I am aware of one little-known story of a secret Microsoft API being used for competitive advantage that I read about. But it wasn't used by Microsoft!

In the non-fiction book "Renegades of the Empire: How Three Software Warriors Started a Revolution Behind the Walls of Fortress Microsoft", it discusses various colorful characters who worked at Microsoft, in various departments, especially with regard to gaming technology on Windows (DirectX and so on).

At one point, Microsoft was going to add DirectX 3D rendering capabilities to Internet Explorer, so websites could run 3D hardware-accelerated code inside the browser.

Alex St John, who worked on the (at the time) small DirectX team, had assisted in implementing prototype code for allowing websites to access DirectX 3D functions from within Internet Explorer.

Microsoft demo'd that prototype publicly, but eventually dropped the idea.

Alex St John later left the company for unrelated reasons.

After he left the company, Alex St John realized that the prototype Internet Explorer DirectX code was forgotten about and accidentally shipped in later public releases of Internet Explorer, undocumented.

So Alex St John used that secret undocumented code to gain a competitive edge and create one of the first online travel mapping websites (competing with MapQuest, AltaVista, and Yahoo! Maps) using the secret API to be able to stream and render in realtime the stitched-together map tiles. He sold it (and some related patents) to Google for $50,000, and Google turned it into the first versions of Google Maps.

Answer 6

At least in my experience, most such claims were basically nonsense.

When you ran code from most of the people who complained the most (including both Wordperfect and Corel, for two I remember well) under the Windows debug kernel, you found out the truth: the code simply sucked.

In case you've never used or heard of it, the debug kernel had extra checks to detect and report things like using GDI handles and memory handles after they'd been freed, using a handle to the wrong type of GDI object when you called an API function, etc.

One program generated so many debug messages that it took something like 20 minutes or so to start up. Another one I don't really know exactly how long it took to start--I never had the patience to watch, but I did start it once before going to bed, and by the time I woke up, it was running.

They took so long to start because they were generating literally hundreds of thousands of debug messages. Their code ran like crap because it was written like crap (or more accurately, it simply was crap).

Now, it is true that Windows had internal "stuff" what wasn't documented. It's also true that some of the APIs they supplied were awkward to use (at best). Some of them could have been a whole lot cleaner and simpler to use.

But (despite doing a lot of looking at them) I never saw any of Microsoft's programs bypass the clumsy APIs and use some secret function that made their life a lot easier instead.