HOWTO Specify the Location of Libraries for use with MinGW

FIXME: Page under construction; some value here now, but not yet complete.

I will add to this as time permits; please bear with me. My intent is to offer advice, complementing the information relating to include file search paths, on circumventing the limitations of MinGW's GCC when installing, and subsequently linking with, user added function libraries.


Introduction

This Mini-HOWTO further develops the theme introduced by the IncludePathHOWTO, progressing to a complementary consideration of where the MinGW implementation of GCC, (or more accurately, of the linker, ld, which is actually a component of the GNU binutils package), searches for object code libraries. In particular, it address the question: "How do I ensure that the MinGW linker will find my object libraries?"

In the case of header files, we observed that the GCC Manual makes a definitive statement, regarding the search path for header files, yet we also noted that this statement is not applicable to the MinGW implementation of GCC. In the case of libraries, the GCC Manual makes a much less definitive statement, which is paradoxically more applicable to the MinGW implementation, regarding the default search path:

-llibrary
-l library
    Search the library named library when linking.  (The second alternative
    with the library as a separate argument is only for POSIX compliance and
    is not recommended.)

    It makes a difference where in the command you write this option; the
    linker searches and processes libraries and object files in the order they
    are specified.  Thus, `foo.o -lz bar.o' searches library `z' after file foo.o
    but before bar.o.  If bar.o refers to functions in `z', those functions may
    not be loaded.

    The linker searches a standard list of directories for the library, which is
    actually a file named liblibrary.a.  The linker then uses this file as if it
    had been specified precisely by name.

    The directories searched include several standard system directories plus
    any that you specify with -L.

    :

Note: the second paragraph of the above extract is not strictly relevant to the subject of this Mini-HOWTO; I've chosen not to elide it, because it answers another FAQ: "Why do I get undefined reference errors, when the symbols in question are definitely present in the libraries I've specified with -l options?" -- the usual answer is that the -l options have been placed too early in the command line argument sequence. The remainder, and in particular the final paragraph, are pertinent, vaguely indicating that libraries will be searched for in several standard system directories; these will be identified more explicitly in this Mini-HOWTO.

Consulting an Alternative Reference Source
The GNU Binary File Utilities Manual's Perspective

We have observed that the GCC Manual makes no definitive statement, regarding the default directories to be searched for libraries. Since the actual library search is performed by the linker, ld, which is a component of the GNU Binary File Utilities, (more commonly known as binutils), we might hope to find a more definitive statement in the applicable section of the Binutils Manual. Unfortunately, while obviously a useful reference for the GNU Binary File Utilities in general, this manual appears to be equally noncommittal; the only references to be found, to library search paths, are in the Command Line Options section:

 :

-Lsearchdir
--library-path=searchdir
    Add path searchdir to the list of paths that ld will search for archive
    libraries and ld control scripts.  You may use this option any number of
    times.  The directories are searched in the order in which they are specified
    on the command line.  Directories specified on the command line are searched
    before the default directories.  All -L options apply to all -l options,
    regardless of the order in which the options appear.

    If searchdir begins with =, then the = will be replaced by the sysroot prefix,
    a path specified when the linker is configured.

    The default set of paths searched (without being specified with `-L') depends
    on which emulation mode ld is using, and in some cases also on how it was
    configured.  See Environment.

    The paths can also be specified in a link script with the SEARCH_DIR command.
    Directories specified this way are searched at the point in which the linker
    script appears in the command line.

 :

and in the Linker Scripts / File Commands section:

 :

SEARCH_DIR(path)
    The SEARCH_DIR command adds path to the list of paths where ld looks for
    archive libraries.  Using SEARCH_DIR(path) is exactly like using `-L path'
    on the command line (see Command Line Options).  If both are used, then the
    linker will search both paths.  Paths specified using the command line option
    are searched first.

 :

while the Environment reference, appearing within the Command Line Options reference, apparently has nothing to say, which is related to this subject.

Determining MinGW's Default Library Search Path

So, if the applicable manuals don't give us any definitive indication of which directories will be searched for libraries, how can we identify where the MinGW tools will search?

The ld manual tells us that the directories searched will be those specified on the command line, using -L options, followed by those specified in the effective linker script, using the SEARCH_DIR script command. We may identify the search directories specified within the default linker script, by running the command:

ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

which, with a standard MinGW installation, might produce output similar to:

$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
SEARCH_DIR("/mingw/mingw32/lib")
SEARCH_DIR("/mingw/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib")

The first thing we might notice about these, is that they are all specified in the style of POSIX paths; however, since the MinGW tools are all native MS-Windows applications, the linker will be unable to resolve these paths in any manner other than as absolute with respect to the current drive, at the time when the linker is invoked. Thus, even if these directories do exist, (which is unlikely in a standard MS-Windows installation), they may represent different locations, depending on whichever drive the user has made current, at the time when the linker is invoked. (For users of MSYS, they will definitely not be resolved relative to the root of the MSYS virtual file system, even if that is what the user might like).

Now, we have seen that none of its pre-configured search paths seem to be particularly useful as default search paths, for use with the MinGW linker. That may be considered to be a configuration error, (which a future binutils package release may address); nevertheless, it does not appear to present a significant obstacle to operation in practice, since suitable search paths may always be specified using -L options. Of course, any such options which we are required to specify on the command line, can hardly be described as default search paths; however, if we adopt the recommended practice of always invoking ld via an appropriate GCC front-end, then the GCC driver has an opportunity to supply such options, in a manner which may be so described, and in fact, this is precisely what happens.

Thus, we have established that the linker's default search paths have little value, in a standard MinGW installation, but that GCC itself furnishes the effective defaults, by supplying appropriate -L options. To discover what these default -L options are, we might use the command:

gcc -print-search-dirs

If we run this, we will see a sprawling mess of nigh incomprehensible output; some judicious filtering, to extract only the library search paths, normalise them to canonical forms, and present them one per line, can reduce it to a form which is more readily interpreted:

$ gcc -print-search-dirs | \ 
> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \ 
> tr \; \\012
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  /mingw/lib/gcc/mingw32/3.4.5/
  /usr/lib/gcc/mingw32/3.4.5/
  c:/mingw/mingw32/lib/mingw32/3.4.5/
  c:/mingw/mingw32/lib/
  /mingw/mingw32/lib/mingw32/3.4.5/
  /mingw/mingw32/lib/
  /mingw/lib/mingw32/3.4.5/
  /mingw/lib/
  c:/mingw/lib/mingw32/3.4.5/
  c:/mingw/lib/
  /mingw/lib/mingw32/3.4.5/
  /mingw/lib/
  /lib/mingw32/3.4.5/
  /lib/
  /usr/lib/mingw32/3.4.5/
  /usr/lib/

Once again, we may observe that this list includes a number of POSIX style paths, which most likely are not relevant for a MinGW installation. If we further filter, to remove the POSIX paths, and keep only the MS-Windows style paths, this list reduces to:

$ gcc -print-search-dirs | \ 
> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \ 
> tr \; \\012 | \ 
> grep -v '^ */'
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/mingw32/3.4.5/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/mingw32/3.4.5/
  c:/mingw/lib/

From this, we might deduce that GCC will pass each of these paths to the linker, using an appropriate -L option; to verify that deduction, we might use GCC's "-###" option, in a command such as:

gcc -### -o foo foo.c

to reveal exactly what command GCC will pass to the linker; (again, the output is not formatted for easy reading, but some judicious filtering will extract just the -L options, and present them in a form which can be more easily read by humans):

$ touch foo.c
$ gcc -### -o foo foo.c 2>&1 | \ 
> tr -s \\040 \\012 | \ 
> sed '/^"-L/!d;s,,,;s,"$,/,;:1;s,/[^/.][^/]*/\.\./,/,;t 1' | \ 
> sed 's,^,  ,;1h;1s,.*,libraries:,;1G'
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/

Oops! What happened there? In these examples, GCC's -print-search-paths option tells us that there are six MS-Windows paths it will search, but only four of those are passed on to the linker, as -L options. Why are all six not passed on?

Well, recall that we arbitrarily chose to ignore the POSIX style paths in GCC's built-in list, when we formulated the list of MS-Windows paths we expected to be searched, but why would GCC itself make such an arbitrary choice? In reality, it doesn't. Let's re-examine that full list of built-in paths, but this time we will filter it on an analytical basis, selecting only those paths which represent actual physical directories on our host machine:

$ ( echo libraries:
>   drive=`pwd -W | sed 's,:.*,,'`
>   for dir in `gcc -print-search-dirs \ 
>     | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,' \ 
>     | tr \; '\012' \ 
>     | sed "s,^/,$drive:/,"`
>     do test -d $dir && echo "  $dir"
>     done
> )
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/

which, we observe, now exactly matches the list of -L options we see passed to the linker, when we invoke GCC with the "-###" option. Thus, we may refine our previous deduction, to conclude that GCC will examine it's built-in list of pre-configured library search directories, passing on to the linker, in the form of -L options, those which represent actual physical directories on the host machine; therefore, this becomes the list of default library search directories for MinGW.

Installation and Use of Supplementary Libraries with MinGW

When a project uses only the standard core libraries, either as provided by the MS-Windows operating system itself, or those provided as integral components of the MinGW distribution, the user need take no action to ensure that those libraries will be searched at link time; the compiler driver will take care of it automatically. Even in cases where the project relies on optional libraries supplied as standard operating system components, and for which MinGW provides import libraries, the user need do no more than specify an appropriate -lNAME option to the compiler driver, and the library will be linked, as required. In either of these cases, the necessary libraries are located within the default search path, and no particular action is required of the user, to specify where that might be.

Conversely, when a project supplements the standard libraries, with one or more which it creates exclusively for its own use, then those libraries will not be found in the default search path, so their location must be explicitly identified to the compiler driver, to ensure that they can be found at link time. In such cases, in which the libraries are not installed for use by other projects, the most common, and indeed the expected practice, is for the project maintainer to specify the library locations explicitly, by addition of appropriate -L options to the linking commands within the project's build system infrastructure. When this is done correctly, there is again no action required of the user, to specify the location of such supplementary libraries.

The situation becomes much more interesting, when a project provides one or more supplementary libraries for use by other projects. In such cases, there is an expectation that the libraries will be installed in some central location, whence they will become readily accessible when client projects are linked, without a need for any particular intervention by the user building the client project, to specify that location.

On typical POSIX systems, and in particular on those conforming to the GNU model, it is common for the standard system libraries, including those provided as standard components of the compiler suite, to be installed in /lib or /usr/lib, while supplementary libraries may be installed in /usr/local/lib. In fact, as we have already observed, these three locations are already included, by means of SEARCH_DIR("...") commands in the default linker scripts, as defaults to be searched by the MinGW linker; unfortunately, they are specified exactly as such, in the original POSIX format, which cannot be readily interpreted in the file system context in which the MinGW linker must operate! This is a limitation of the MinGW linker, which the user installing supplementary libraries must work around, possibly by adopting one of the following strategies:--

  1. Install in a non-default location, and require every user of the installed libraries to explicitly add the requisite -L options to the linking commands, for every subsequent project which requires any supplementary library. This is inconvenient for subsequent use, and is generally not recommended.

  2. Install in a location which is already considered to be a default by the compiler driver. This often recommended option is the simplest possible, for both the user installing the libraries, and for those using them. If using MSYS as the command line environment for building and installing the libraries, and the libraries themselves are delivered as GNU standard source packages, such installation may be readily accomplished by the command sequence:
    $ ../path/to/configure --prefix=`cd /mingw ; pwd -W` ... && make && make install

    (Note: here we explicitly force the use of MS-Windows native path format for the prefix; this is normally necessary, if the libraries being built hard code the prefix within object files).

    It may be noted that, in this example, we appear to have arbitrarily chosen the top level lib sub-directory, within MinGW's own installation directory, as the preferred location for installation of supplementary libraries; in reality, of the pre-existing default search directories this is the only sane choice, since the others are much more specifically associated with the component libraries provided by the compiler suite itself, and it is undesirable to pollute them with "foreign" libraries. Indeed, many users also consider it undesirable to pollute the top level MinGW library directory with such "foreign" libraries; such users may prefer the following alternative installation strategy.

  3. Install in a non-default location, just as in the first option, but also customise the linker scripts, or the compiler driver specs, to implicitly add the requisite -L option, such that this location becomes an additional default search path. Available techniques for achieving such customisation will be discussed, in the following section.

Customising MinGW's Default Library Search Path

So, you've decided you would prefer to segregate your add-on libraries from the core MinGW libraries? The first issue to address, is where you would like to install them.

Here, you have two basic options:

  1. Establish a set up, analogous to the common /usr/local hierarchy of typical Unix systems, with all additional libraries, and their associated header files, collected into this one location.

  2. Provide a separate directory for each individual library package, and customise the GCC search paths to consider all of them, as appropriate, when looking for libraries and header files.

It should be fairly obvious that the first of these will require less GCC customisation than the second. In fact, the principles are the same in both cases; however, whereas the first requires the addition of only one include path, (-I option), and one library path, (-L option), the second will require one of each per library package installed, which may be detrimental to GCC performance. Thus, we will describe only the first option in detail; users preferring the second may extend the principles described, as necessary, to accommodate their preference.

Having chosen to install all add-on libraries in a common location, the choice of that location is fairly arbitrary. MSYS users may choose to simply use the conceptual /usr/local of their existing MSYS installation, in which case, they must ensure that that directory physically exists, (it may not in a bare, standard MSYS installation), and they must identify its true native Windows path, for use in the GCC customisation, as shown in the following example, for a standard MSYS installation in its default installation directory:

$ mkdir -p /usr/local
$ ( cd /usr/local && pwd -W )
c:/msys/1.0/local

Caution: Although this choice of directory may seem attractive, particularly because it matches the default prefix assumed by the majority of GNU packages, it may not represent the best choice for all MSYS users. This is particularly true for those users who may wish to develop MSYS applications. The special MSYS build of GCC used to compile such applications reserves /usr/local for its own purposes; in this case, it must not be polluted by headers and libraries intended for use with the MinGW build of GCC.

Alternatively, masochists who prefer to persist with cmd.exe, or MSYS users who prefer to keep MinGW related add-ons more closely associated with their MinGW installations, might choose to create an installation directory locally within their MinGW tree:

C:\> mkdir c:\mingw\local

or adjacent to it:

C:\> mkdir c:\mingw-local

Caution: Whichever of these options you choose, please do not choose to create any directory with white space appearing anywhere in its absolute path name. If you adopt this asinine practice, do not be surprised if it turns around and bites. When it does, please do not come here to complain; you have been warned!

Once the tree into which supplementary libraries, and their headers, are to be installed has been selected, and created, the next step is to install the requisite library packages into this tree. In the case of standard GNU library packages, and assuming you have chosen C:\MinGW\local as your supplementary installation path, (MSYS users may refer to it simply as /mingw/local), this may be accomplished from the MSYS shell, by running:

$ ../path/to/configure --prefix=/mingw/local ... && make && make install

(or, if you suspect that the library may hard code its installation path within itself):

$ ../path/to/configure --prefix=`cd /mingw/local ; pwd -W` ... && make && make install

Note: If the library package is not a standard GNU package, you will need to adapt its installation procedure to achieve the effect of the above. Details of how this may be achieved are package specific, and are beyond the scope of this HOWTO.

Finally, having installed supplementary libraries and their headers into the supplementary directory tree, they must be made available to the GCC compilers. Of course, this may always be achieved by adding appropriate -I ... and -L ... specifications to the commands, when the compilers are invoked, but that hardly seems consistent with the level of convenience we are striving to achieve! Obviously, the desired objective can best be met if it can be arranged for the compilers to search in the supplementary tree, by default; this may be achieved by adding the path name for the supplementary include file tree to the compilers' default search paths, as described in the include file search paths HOWTO, accompanied by either of:

  1. Adding the path name for the supplementary library directory to the semicolon separated list of library search directories specified by the LIBRARY_PATH environment variable, (which must be exported, if it is assigned from within the MSYS shell):

    $ export LIBRARY_PATH
    $ LIBRARY_PATH="C:/mingw/local/lib;$LIBRARY_PATH"
    

    Note: The GCC manual states that this technique may be used only when GCC is configured as a native compiler; thus, this method of specifying the supplementary library path may not be used when your MinGW GCC is configured as a cross-compiler. Indeed, it would be illogical to use this technique with a cross-compiler, as it would lead to confusion between those libraries which are specific to the native GCC compiler on the build platform, and those which are specific to the cross-compiler.

  2. Follow the method, as described in the specs file customisation HOWTO, to add a supplementary path to the default library search path.

    Note: This method is applicable equally to MinGW GCC configured as a native compiler or configured as a cross-compiler; the additional search path applies only to the specific instance of the compiler with which the modified specs file is associated.

Re: HOWTO Specify the Location of Libraries for use with MinGW

The LIBRARY_PATH environment variable is used by gcc. This is what the man page says about it:

The value of LIBRARY_PATH is a colon-separated list of directories, much like PATH. When configured as a native compiler, GCC tries the directories thus specified when searching for special linker files, if it can’t find them using GCC_EXEC_PREFIX. Linking using GCC also uses these directories when searching for ordinary libraries for the -l option (but directories specified with -L come first).

Re: HOWTO Specify the Location of Libraries for use with MinGW

keith's picture

Yes indeed. Notice that this is still a work in progress, and LIBRARY_PATH is one of the customisation options, which needs to be mentioned in the text yet to be written. Do note, however, that on MS-Windows it is not a colon-separated list; it needs to be semicolon-separated, with all of its members specified in native MS-Windows format; (however, MSYS users may specify it as a colon-separated list, if all of its members are specified in MSYS' emulated POSIX style, and it will be converted appropriately, but mixed POSIX and MS-Windows styles are unsupported).

A further point to note: the manpage snippet cited is somewhat ambiguous, but it suggests that LIBRARY_PATH is not used if mingw32-gcc is built as a cross-compiler. This feature may be of use only to those running MinGW as a native compiler on MS-Windows; it may have no value, for those of us who run it cross hosted.

Thanks for this comment. The focus of the article is more on setting the system up to avoid dependency on a proliferation of environment variables; however, the comment is pertinent, and a valuable source of additional information, until such time as I cover this aspect within the body text. When I get to that point, I will discard this attached comment thread.

Re: HOWTO Specify the Location of Libraries for use with MinGW

Thank you for explaining in more details that man page paragraph. When reading it, It's indeed easy to miss the fact it's not usable with a cross-compiler!
I was also wondering what the man page refers to with "searching for special linker files". Are these the specs files described in the SpecsFileHOWTO page?

Re: HOWTO Specify the Location of Libraries for use with MinGW

keith's picture

No, the specs files are specific to configuration of the GCC front-end drivers. As I understand it, special linker files would be the linker scripts, referred to in the ld section of the binutils manual.

Site Status

Site maintenance completed May 25th, 2012 at 12:38 UTC