Working with DLLs on Windows, Mac, and Linux

So.

Let’s talk about dynamic libraries.

How to write them, how to build them, how to connect them to your programs.

How to get a result that actually keeps working once you send your program off to someone else.

Exactly how this stuff works, and what you need to do, varies depending on your operating system and build tools. There are four major cases I plan to cover:

  • Building on Linux with gcc or clang
  • Building on Mac with clang
  • Building on Windows under MSYS2 with gcc or clang
  • Building on Windows with Visual Studio

Irrespective of your operating system, gcc and clang often end up behaving very similarly, and clang also has a mode called clang-cl that apparently mimicks the command-line interface of the Visual Studio tools.

There are two semi-major cases that I’m not going to cover: directly using the Visual Studio command-line tools, and the BSD-like systems, which have a different binary format from the three OSes I am covering.

I’m skipping the Visual Studio command line tools because they are verbose, arcane, and exceptionally easy to find documentation for, assuming you can make it happen in Visual Studio itself.. If it is very important to track down the relevant flags for some build, Visual Studio’s Preferences dialog boxes are extremely transparent about what each entry in each dialog box controls, listing the relevant command-line options either in the description of the setting or alongside each possible option. I’ll only bring it into play if there’s something that’s so much easier to do with the CLI that you’d normally break out of Visual Studio to do it.

BSD, on the other hand, I’m skipping because I know basically nothing about it other than that sometimes it is surprisingly and distressingly different. My advice for gcc/clang on Linux or Mac may perhaps be applicable.

Some Words of Encouragement Before The Abyss

Getting a program to load and run while depending on shared libraries is not generally considered a difficult problem. Indeed, it has been a solved problem for decades. As such, the purpose of this article is twofold.

  1. To be an example of what exactly it means for something to be “a solved problem for decades.” One gets the impression that the details of this stuff are not widely known not because they’re too simple to bother explaining, but because people have been aggressively forgetting it as hard as they can once it works well enough that they can walk away.
  2. To be a usable tutorial for people who have ended up on the rusted pointy end of building or distributing software that has custom shared libraries in it. They may be suffering impostor syndrome, wondering how they could be screwing up such an obviously basic thing. Let my barely-contained ranting reassure you, and with luck the actual instructions in here may get you where you need to be.

Making and Using Shared Objects on Linux

Linux calls its dynamically-linked libraries “shared objects” and gives them the suffix .so. Uniquely amongst the three systems we’re discussing here, Linux actually requires files to be compiled specially if they are to be included in a shared object. Windows relies on load-time relocation techniques (like we discussed back when I worked my way through the history of loading programs into home computers) but Mac and Linux do not do this for their shared libraries. They demand that the code all be position-independent, and I believe the idea behind this is on Linux that this way it can map the same chunk of physical memory into multiple processes loading the same library and not pay the extra RAM cost. (The macOS documentation describes this as needed for address-space layout randomization, which it makes default starting in 10.7.)

If you’re working with a compiled language, all you have to do is pass -fPIC as an option to gcc or clang when compiling your source files. If you’re writing assembly language code, you need to do this work by hand. Unfortunately, the Intel instruction sets are really quite bad at position-independent code, at least compared to systems like ARM. The NASM documentation discusses what you need to do to get this to work on Linux (or on BSD, which has a different mechanism entirely and which I’m not discussing in this article).

To create the final .so file, make sure you’re naming the file properly, and also give gcc the argument -shared. Library files need to start with the three letters lib.

As usual, you can also merge the compilation and linking steps into one, so I could build a library directly from some source files with a command like this:

gcc -shared -o libbumbershoot.so -fPIC bumbershoot.c utils.c

This will result in a file named libbumbershoot.so in my current directory, ready for linking against other programs.

Finding Libraries at Build Time

Linux has two kinds of library files. There are the shared objects (or dynamic libraries), and also static libraries. Static libraries have the .a extension, are produced by the ar utility, and the compiler treats them in basically all ways as a zip file full of .o files that it can pull stuff out of as needed.

When you are building a program and want to use functions from libraries, the syntax is identical for both static and dynamic libraries:

  • The -L option works like the -I option, but for naming directories with libraries in them instead of header files: -Llib will search the lib directory, and -L. will search the current directory.
  • The -l option, on the other hand, names a library to link in. The argument -labc will search the system library directories, and any directories you named with -L, for files named libabc.so or libabc.a. If it finds the latter, it will pull any .o files out of it that implement functions that are called but not defined in the main program and stick them into the program itself, just as if you’d written them yourself. If it finds the former, it makes a note in the executable that it will need to load that library at run time just before the program starts.

And if all we had were static libraries, or if all our dynamic libraries live in system folders like /usr/lib, that’s the end of the story. If we don’t, say, because we’re linking against a shared object that’s in our current directory because we just compiled it, we’ve still got problems:

[mcmartin@bumbershoot dlls]$ gcc -shared -fPIC -o libbumbershoot.so bumbershoot.c utils.c
[mcmartin@bumbershoot dlls]$ gcc main.c -L. -lbumbershoot
[mcmartin@bumbershoot dlls]$ ./a.out
./a.out: error while loading shared libraries: libbumbershoot.so: cannot open shared object file: No such file or directory
[mcmartin@bumbershoot dlls]$

We have several options for dealing with this, depending on what, exactly, our plan is.

Finding Libraries at Run Time

The most direct way to tell a program where its libraries are in your system is to tell it directly as you are starting the program. I mentioned in my previous article that the output of ldd depended on a variety of things, including the current environment variables. That’s because you can use environment variables to tell a program where its libraries are: if, in our previous example, we instead ran the program with LD_LIBRARY_PATH=. ./a.out, the libbumbershoot.so file would be found without complaint.

We could make this a general solution. We could place our binary and libraries where we wished, and then have the “real” program that the user actually runs be a shell script that sets the library path appropriately and then invokes the executable. This works, but it is more than a bit inelegant.

The preferred solution for this is to encode the directories to search into the executable or library itself. The set of directories a binary will check on its own is called its rpath, and while gcc and clang don’t let us set this directly, the underlying linker will. We can reach that through gcc with some extremely ugly syntax. The command gcc main.c -L. -Wl,-rpath,. -lbumbershoot will tell gcc to tell the linker to encode the current directory into the final binary’s rpath.

This is almost but not quite what we want. What putting the current directory into the rpath does is check the current directory of the user, when they run the program. If they are in some other directory when they invoke the binary, it will search that directory for the libraries. What we need is a way to specify paths relative to the executable itself.

We can do that by using the magic value $ORIGIN in the rpath. We’ll need to surround it all in single quotes so that our command shell doesn’t interpret it as an environment variable in its own right, but this does precisely what we want. For a case where the shared objects are in the same directory as the executable:

gcc main.c -L. -Wl,-rpath,'$ORIGIN' -lbumbershoot

And for a case where the executable will be installed in a bin/ subdirectory while the libraries will be installed in a lib/ subdirectory:

gcc main.c -L. -Wl,-rpath,'$ORIGIN/../lib'

Note that this refers to the location of the file being built, not necessarily the location of the main program. In our example above, if the libraries in lib/ linked against each other, they would want to define their rpath as '$ORIGIN' alone.

The full set of options available here are outlined in the man page for ld.so(8). The options I’ve given here, though, should in the end be all you need.

Controlling Symbol Visibility

Linux will, by default, have shared objects export every single one of their symbols. This does make linking against them later much easier, but if your have a large library with a restricted interface you may prefer to hide the existence of your other functions away from the rest of the world. After all, if you export internal helper functions, some maniac is going to come along and actually use them in their own program, and then get mad at you when you change them incompatibly later. The maniacs are easy to blame, but if you also like to name your functions things like get_next_item you also don’t want your library to start mysteriously failing just because someone linking against your app used the same function names. Avoiding that kind of thing is why I generally recommend hiding your functions by default.

Controlling what symbols do and do not appear as available from a shared object is intrinsically a system-dependent operation—C has no opinions about the existence of shared objects at all, after all. What gcc (and clang, when acting like gcc, which is most of the time) do is let you annotate functions, classes, and data items to alter their visibility, and to let you alter file-wide defaults. To default to hiding all functions and data, compile and link with the option -fvisibility=hidden.

To hide just one function even with otherwise default export-everything settings, mark it with __attribute__ ((visibility("hidden"))) in the same place you would mark it static. Similarly, if you have set the default visibility to hidden, you will need to mark your exported functions __attribute__ ((visibility("default"))) in the same way.

This can get tedious very quickly, so it’s best to do this with preprocessor macros. Fortunately, there is standard boilerplate for this; Microsoft Visual Studio has a nastier set of requirements for DLL export annotations, and the boilerplate for that meshes very neatly with gcc’s and clang’s requirements. I’ll cover that all at once once we get to Visual Studio.

Summary

  • Compile your files with the -fPIC option.
  • If you want to link a library with the option -lwhatever, give it the name libwhatever.so and compile with the -shared option.
  • Use -Ldir to identify where your libraries live at build time.
  • Use -Wl,-rpath,'dir' to identify where your libraries will live at run time.
  • Use $ORIGIN in your rpath definition to refer to the place the user has happened to install the file you are building.
  • If you want to go out of your way to only export the functions that are part of your library’s public API, compile and link with the -fvisibility=hidden option and annotate each exported function with __attribute__ ((visibility("default"))). Use the EXPORTS macro tree (at the end of this article) to set up your header files to do this automatically.

Making and Using Dynamic Libraries on macOS

Since OS X 10.0, macOS has referred to its dynamically-linked libraries by that name, and it usually gives them the suffix .dylib.

The good news is that the rules for creating and linking against dylibs is almost precisely the same as the rules on Linux; in fact, they are easier, because you do not need to demand a switch for position-independent code—it just happens. Clang respects all the same command-line switches and attribute markers as gcc, so visibility and library selection all works exactly the same way.

That’s the end of the good news, though. The rules for rpaths and more generally for wrangling dylibs at runtime are by a very wide margin the most inconvenient on Macs.

When Dylibs May Be Loaded

When a dylib is created, it is assigned a special name that informs it of its full install path. When a dylib is linked against, the executable being created records that ultimate install path, as recorded by the dylib, into itself. At runtime, that path is the only place it checks, and if the dylib isn’t there, it will refuse to run. In fact, if the dylib is there, but its header thinks it “ought” to be in a different location, it will also refuse to load and run.

This design is great for Unix-like filesystem hierarchies where you generally put all your libraries into fixed directories like /usr/lib and also want to make sure that nobody gets in your way to rewrite functions you depend on, but it is utter madness for something that is supposed to be providing the Mac user experience. After all, the primary feature of Macintosh applications is—and has been since 1984—that you install them by copying one item into wherever it is you want it, and you uninstall them by deleting that item. Starting with OS X, these items changed from single files into directories, but the general principle still held. So now we face the problem of how to name the dylibs that are in our application directory in a way that we can still load them wherever they happen to actually be. The $ORIGIN trick we saw on Linux doesn’t work here, either, because that was a quirk of ld-linux.so.

It’s similar, though. It’s just that we have three variables to choose from instead of just the one.

Special Dylib Directories

There are three special values you can start a library’s internal install path with that have special relative meanings:

  • @executable_path refers to the directory in which the main program lives. Most app bundles link their bundled frameworks with something like @executable_path/../Frameworks/Bumbershoot.framework/Bumbershoot/Versions/A/Bumbershoot. Frameworks that are installed within an app bundle side by side can refer to each other this way, too, since it’s the same executable linking them.
  • @loader_path is basically the same thing as $ORIGIN, in that it is the path that the file itself—library or main program—lives. It’s mostly only useful for cases where a single framework wants to link against separate dylibs that it itself includes. That’s rarer than you’d think. I don’t think I’ve ever seen this option in the wild.
  • @rpath is a stand-in for any directory that the loading binary has listed as part of its rpath. This is handy if you are distributing a framework that other people are expecteed to incorporate in their own code, because the same path can be used irrespective of where they want to actually package the binary in their own code; they just need to set their own rpath appropriately.

(For further reading, Greg Hurrell has a very good overview of these three flags and how to use them, including some worked examples.)

Setting Install Names and Rpaths at Build Time

Like Linux, we can provide values for these constants at build time by passing custom options to the linker. Also like Linux, we can use the -Wl, option to pass comma-separated arguments to the linker, and -rpath is a working option. Macs add the install_name option to set the library’s own idea of where it belongs.

As an example, suppose that Main.app intends to have its main program (Main.app/Contents/MacOS/Main) link against a library it’s keeping with its resources in Main.app/Contents/Resources/libexample.dylib. When building the files, we would use commands roughly like these:

$ clang -shared -o libexample.dylib example.c -Wl,install_name,@executable_path/../Resources/libexample.dylib
$ clang -o Main main.c -L. -lexample

These files won’t find each other if you tried to run them out of your build directory, but once they were put in place properly within an application bundle the system would start up just fine.

Setting Install Names and Rpaths at Packaging Time

If you’re using somebody else’s frameworks, they may not have actually set their install names the way you want, and this may also result in your built programs looking for libraries in the wrong places. If you’re trying to use the same library in different directory structures, this is almost guaranteed unless it’s self-identifying with @rpath.

Fortunately, the Xcode command line tools have an additional tool to edit these linkage tables after the fact. This tool is called install_name_tool (link is to its man page). This has modes for altering both linked and assumed install names, and also permits you to add, remove, or edit rpath entries after the fact. Of course, if those libraries themselves link against further libraries, you may need to do some kind of recursive traversal to get everything you need properly into place. I’ve had to write special-purpose bundling scripts for many of my projects to actually properly copy everything into place and then fix up all the linkages. (One related but non-dylib-related caveat there: frameworks on macOS usually have internal symbolic links as part of their structure, so it’s best to use the ditto tool instead of cp to copy them into place.)

Summary

  • If you want to link a library with the option -lwhatever, give it the name libwhatever.dylib and also use the -shared option with clang.
  • Use -Ldir to identify where your libraries live at build time.
  • Use -Wl,-install_name,@rpath/libwhatever.dylib to identify the installed location of your library relative to the executable’s rpath.
  • Use -Wl,-rpath,'dir' when building your main executable to identify what @rpath should mean.
  • Use @loader_path in your install name to refer to the place the user has happened to install the file you are building.
  • Use @executable_path in your install name to refer to the place the user has happened to install the main executable for your application bundle.
  • If you need to alter the previous values after the fact, use the install_name_tool utility.
  • If you want to go out of your way to only export the functions that are part of your library’s public API, compile and link with the -fvisibility=hidden option and annotate each exported function with __attribute__ ((visibility("default"))). Use the EXPORTS macro tree (below) to set up your header files to do this automatically.

Making and Using Dynamically Linked Libraries on Windows

Windows is the least flexible about DLLs of our three OSes here. Fortunately, though, it also has the most immediately useful defaults. The search order for DLLs is basically fixed, with a few variations based on registry keys that in practice should never matter. The rule for ordinary software development is simple: put your DLLs in the same directory as your EXE, and everything will just work. The complications on Windows are twofold: you might depend on system libraries that don’t exist on your client system, and you may need to deal with “import libraries”.

Both of these also turn out to only be issues if you are using Visual Studio. If you are using MinGW on MSYS2, all of this is automated and you get to use a stripped-down version of the Linux workflow: there is no rpath and DLLs load from the executable’s path by default. MinGW wraps the MSVC6 runtime, which is present on all modern versions of Windows without any additional dependencies required.

Managing Missing System Libraries

This isn’t really a DLL problem per se, but if you’re writing desktop software on Windows, you are probably using either C++ or C#. Visual Studio comes with custom runtimes that need to be installed to the system as a whole before programs built with that version of Visual Studio can run. These days most versions of Windows pre-install compatible versions of the earlier devtools, but you can’t rely on this. Your copy of Visual Studio should ship with redistributable installers that will install the relevant version of the language runtime libraries if they are not already present. Your task there is simply to make sure that your installer actually includes and runs them.

Managing Import Libraries

I get the impression that in the early days of the Windows platform, there was no real mechanism for automatically linking and loading symbols out of DLLs, and that this needed to be done by hand with equivalents of LoadLibrary and GetProcAddress. To make DLL work even remotely feasible, this got somewhat automated and any time you built a DLL you would also, as a side effect, build an import library to go with it. These are static libraries whose purpose is simply to provide entry points that load and interact with the underlying DLLs as needed.

For the most part, Visual Studio handles this for you and you don’t have to bother with it; if you plan on linking bumbershoot.dll, you make sure bumbershoot.lib is in your “additional dependencies” list i the Project Properties page. The hard part is specifying the library and main programs’ interactions with the import library.

Unlike Linux and macOS, symbols defined in a Windows DLL are not exported by default. Instead, they should be marked for export with the compiler-specific attribute __declspec(dllexport). This is much like the attribute-setting code we saw on the other two platforms, so that is not so bad. What is bad, however, is that the code that uses the import library needs to declare the same functions with __declspec(dllimport). This lets the compiler take advantage of some quirks of the PE32 executable format to turn these DLL calls into indirect jumps through the DLL’s own import address table, but it also means that the header as used in the library and the header as used in the main program need to be identical in all ways but one set of attributions.

There’s a standard way to do this, and it is generic enough that it covers the Linux and Mac cases as well if you want to set their default symbol visibility to “hidden”. We’ll cover that in the section below, since it’s ultimately common to all platforms. It’s relevant to Visual Studio, though, because in Visual Studio it is mandatory.

Summary for MinGW on MSYS2

  • If you want to link a library with the option -lwhatever, give it either the name libwhatever.dll or whatever.dll and also use the -shared option with gcc.
  • Use -Ldir to identify where your libraries live at build time.
  • Put your DLL and EXE files in the same directory when the time comes to run the program.
  • If MinGW ends up linking in additional dependencies, copy those out of the MSYS filesystem into your install directory too.

Summary for Visual Studio

  • Use the EXPORTS macro tree (below) to organize your file headers.
  • If your DLLs and main program are part of the same project, declare dependencies appropriately within Visual Studio.
  • If the DLLs are externally provided and are not part of your main project, adjust the “additional library paths” and “additional dependencies” sections under “Linker” in your main program’s project properties.
  • Install any necessary language runtimes from the redistributable packages that came with your copy of Visual Studio.
  • Put your DLL and EXE files in the same directory when the time comes to run the program.

Controlling Visibility With a Single Set Of Macros

The Visual Studio declspec regime is rather annoying, because it means that we need different header text for the library and the client code for that library. But this is a superset of a similar issue in our Linux and Mac code; if we are setting the visibility of a symbol within a library, we do not want to have that show up when we are compiling the main program. It probably won’t hurt, but it doesn’t really help.

A protocol has evolved to take advantage of the C preprocessor in these cases. It’s mostly only used on Windows, where it is mandatory, but it’s useful enough on the other platforms that it shows up there at times too.

Here’s how it works (assuming a library named bumbershoot.dll):

  • Every function or data item that is to be exported as part of bumbershoot.dll gets annotated with the symbol BUMBERSHOOT_EXPORT.
  • The value of BUMBERSHOOT_EXPORT specifies either the “dll export” attribute (or the “default visibility” tag on Linux) or the “dll import” attribute (which is usually just the empty string on Linux) depending on whether or not BUMBERSHOOT_EXPORTS is defined.
  • The build scripts or configuration files for bumbershoot.dll define BUMBERSHOOT_EXPORTS. For GCC and Clang this is managed with the command line option -DBUMBERSHOOT_EXPORTS, added to the C flags inside the Makefile; on Visual Studio this is set under the “C/C++ → Preprocessor → Preprocessor definitions” text field in your project properties.
  • Client code does not define it.

The header file for the library then opens with some boilerplate to set everything up properly:

#ifndef BUMBERSHOOT_EXPORT
#ifdef BUMBERSHOOT_EXPORTS
#ifdef _WIN32
#define BUMBERSHOOT_EXPORT __declspec(dllexport)
#else
#define BUMBERSHOOT_EXPORT __attribute__ ((visibility("default")))
#endif
#else
#ifdef _WIN32
#define BUMBERSHOOT_EXPORT __declspec(dllimport)
#else
#define BUMBERSHOOT_EXPORT
#endif
#endif
#endif

Conclusion

This tutorial doesn’t cover everything that you can do, nor indeed everything that can theoretically go wrong with a deployment. But with luck, this has enough in it to let you actually put together a build that successfully makes use of binaries from multiple sources.

Good luck out there.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s