12

A static library, also known as an archive, is one intended to be statically linked. Originally, only static libraries existed. Static linking must be performed when any modules are recompiled. (From the Wikipedia article on libraries (computing).)

When did operating systems transition from static libraries to dynamic libraries? Particularly interested in Unix and MS-DOS / Windows.

When using a GCC front-end with the -lfoo switch, libfoo.so or libfoo.a are searched for. If only one of these are in the search path it will select that, but if it finds both in the same search directory, it will prefer libfoo.so. Was this always the case, or was there a time when libfoo.a would have been the default if both were present?

Most modern operating systems can have shared library files of the same format as the executable files. This offers two main advantages: first, it requires making only one loader for both of them, rather than two (having the single loader is considered well worth its added complexity). Secondly, it allows the executables also to be used as shared libraries, if they have a symbol table. Typical combined executable and shared library formats are ELF and Mach-O (both in Unix) and PE (Windows). (From the Wikipedia article on libraries (computing).)

Was the change from static libraries to combined executable and shared libraries for Unix and Windows always like this, or was there a time when shared libraries were distinct from executables?

Stephen Kitt
  • 121,835
  • 17
  • 505
  • 462
Single Malt
  • 1,839
  • 1
  • 7
  • 28
  • 6
    I don’t think MS-DOS directly supports shared libraries. Win 3.1 has DLLs, though, and probably 3.0, too. – RonJohn Aug 02 '21 at 14:15
  • @RonJohn yes there is presumably less need in non-multitasking operating systems. There was MS-DOS 4.0 (multitasking), but DLLs probably only appeared in Windows, and from you comment at least Win 3.1. Am unsure if DLLs were the first or only shared library mechanism by Microsoft. – Single Malt Aug 02 '21 at 14:26
  • I know that overlays existed in DOS, and used them regularly to break the 640K limit, but don't know if they could be hacked to be used as shared libraries. – RonJohn Aug 02 '21 at 14:45
  • 3
    OS/360 predates MSDOS and Unix, and its job control language had dynamic linking commands. I don't know the exact time line however. – alephzero Aug 02 '21 at 15:18
  • 4
    ... A quick internet search found the the concepts of dynamic linking etc in an OS/360 "Principles of Operation" manual copyright 1965 so that clearly predates MSDOS and Unix. – alephzero Aug 02 '21 at 15:33
  • 4
    One could say that MS-DOS implements shared libraries in the form of TSRs with a well-defined API. I think of programs like the mouse driver, that provides services (like drawing a mouse cursor) to a lot of different applications, which is more than just a "pure hardware driver" that only receives mouse data and presents it in an protocol-agnostic way. – Michael Karcher Aug 02 '21 at 16:31
  • 3
    1965 is the earliest so far then. The range of features included with OS/360 must have been staggering back then, although with their resources it possibly shows kudos to the project managers of bringing it all together. – Single Malt Aug 02 '21 at 16:31
  • 2
    @SingleMalt The /360 was precious - one system to replace them all :)) No wonder it had next to everything that was state of the art at the time. also, the 60s brought a huge explosion of computing usage and with it methods and knowledge. – Raffzahn Aug 02 '21 at 18:40
  • 2
    @RonJohn, the first version of dynamic libraries in Linux were loaded at a fixed virtual address for all users. A veritable nighmare to make them all fit together with no clashes, but the idea is similar to a TSR for DOS. – vonbrand Aug 02 '21 at 21:03
  • 1
    @vonbrand it's exactly like getting a multitude of drivers and TSRs all working together. Definitely the Bad Old Days~~ – RonJohn Aug 02 '21 at 21:06
  • 1
    A shared library definitely isn't "the same format as an executable". What is the same is the general format of statically and dynamically linked executables. – vonbrand Aug 02 '21 at 21:08
  • 3
    "Dynamically linked" has different meanings in different systems, FWIW. There's the case where the library is loaded into your address space at process startup, doing any fixups your ISA and file format require. Then there's the case where it's not loaded until the application code intentionally says "load this module". I prefer the keep the word "dynamic" for the latter case; the former is just shared libraries. – dave Aug 02 '21 at 21:47
  • 1
    @SingleMalt "The range of features included with OS/360 must have been staggering back then" Quite so. At the end of the 1960s I was working on an OS/360 system entirely written in Fortran, except for a couple of 10-line Assembler routines that were equivalent to Unix "fork" and "exec" system calls. Execution was was driven by a "shell" interpreting a (Turing complete!) custom scripting language and running processes loaded dynamically from a library with hundreds of modules. But it never occurred to us to call it "Unix" and get our names in the history books :) – alephzero Aug 03 '21 at 16:59
  • 1
    re: "The range of features included with OS/360 must have been staggering back then" -- and boy did they stagger. See anything by Fred Brooks on OS/360 development. – dave Aug 03 '21 at 22:17

2 Answers2

17

In the Unix world, dynamic libraries first appeared in SunOS, on top of its virtual memory infrastructure (which allows libraries to be shared in memory, and demand-paged). See Shared Libraries in SunOS for details. This was generalised in System V Release 4.

Windows has had dynamic linking ever since version 1.0, even though early versions didn’t ship any obvious DLLs (those appeared in 3.0) — the core operating system was effectively dynamic libraries disguised as executables, and device drivers (with a .DRV extension) were DLLs. Early versions of Windows avoided any paging requirement by loading everything in a shared memory space, with no protection. OS/2 also supported DLLs from the beginning.

MS-DOS has supported overlays since version 2.0, through interrupt 0x21 service 0x4B03, but that doesn’t implement dynamic linking — the overlays were relocated, but there was no provision for symbol resolution between the loaded program and its overlay. Some compilers or runtime environments provided their own support for dynamic linking. DOS/4G is one example, see How 3Dfx Voodoo 1 Emulator works for previous discussion; HX is another (with support for Win32 PE DLLs).

All the above used the same file format for dynamic libraries and executables, from the beginning.

Earlier operating systems supporting dynamic linking include OS/360 and Multics.

(I don’t know about your GCC question, I’ll update this later.)

Stephen Kitt
  • 121,835
  • 17
  • 505
  • 462
8

Historically, the linking and sharedness of libraries are orthogonal. Shared libraries reduce the needed memory size. Dynamic linking makes it possible to update and relocate the library code without recompiling the application.

Modern systems only have two cases:

  • Static non-shared libraries: When you link with a static library, the entire code is included in the application executable and is not shared with other applications. Though on many systems, the code will be shared between multiple processes of the same application.
  • Dynamic shared libraries: Dynamic libraries share the storage on disc and also the memory at runtime between several processes. Dynamic linking fixes the call addresses to point to the actual location of library in the address space.

Historically there has been all combinations:

  • Most early reusable libraries were distributed as source code. Compiled directly to the executable, this was equivalent to a static, non-shared library.
  • Common routines included in ROM on early home computers were a form of statically linked shared libraries. Only one process was running at a time, but sharing the common routines reduced required storage space. If the ROM was updated, it was done carefully so that important link addresses remained unchanged.
  • First Machintosh System software used a common dynamic loader for both code and graphical resources. It could swap out code segments that were not in use. Because there was no virtual memory addressing on the machine, address lookup for jumps between code segments was performed at runtime. There was only a single application running at a time, so this was dynamically linked non-shared library, though in some cases storage could be shared if multiple applications were on the same floppy.
  • Dynamic shared libraries seem to have first appeared in Multics in late 1960s. It would seem logical that UNIX would have inherited this capability, but I haven't been able to verify whether there were any early UNIX versions without shared libraries.

With regards to Windows, it has always included the support for dynamic shared libraries, already since Windows 1.0. Early Windows programs usually only made use of the system-provided shared libraries and statically linked all other code.

You asked also about the difference between binary formats for executables and libraries. As far as I can tell, all systems that have supported shared libraries have supported a closely related format for executables. Many systems also support legacy executable formats, like .COM on Windows that dates back to MS-DOS.

jpa
  • 1,696
  • 7
  • 17
  • Why wouldn't plug-ins be regarded as dynamic non-shared libraries? – supercat Aug 03 '21 at 15:04
  • @supercat Hmm, I guess they could be - the same way as any DLL that is only used by a single application. But in their structure there is nothing that prevents them being shared. – jpa Aug 03 '21 at 15:08
  • This is informative and provides context. Likewise did not find mention of the rationale for changes from Multics to Unix specifically about this, possibly was a low priority to keep with shared libraries, or static better suited them. The file system and shell seem to have been two of the main features borrowed, and looking only at the contents of Brian Kernighan’s book UNIX: A History and a Memoir get their own sub-sections, although this is referring to the 1975 Unix version. – Single Malt Aug 06 '21 at 18:17