Revision of HOWTO Specify the Location of External Libraries for use with MinGW from 2009, April 1 - 13:24

Printer-friendly version

FIXME: Page under construction; some value here now, but not yet complete.

I will add to this as time permits; please bear with me. My intent is to offer advice, complementing the information relating to include file search paths, on circumventing the limitations of MinGW's GCC when installing, and subsequently linking with, user added function libraries.


Introduction

This Mini-HOWTO further develops the theme introduced by the IncludePathHOWTO, progressing to a complementary consideration of where the MinGW implementation of GCC, (or more accurately, of the linker, ld, which is actually a component of the GNU binutils package), searches for object code libraries. In particular, it address the question: "How do I ensure that the MinGW linker will find my object libraries?"

In the case of header files, we observed that the GCC Manual makes a definitive statement, regarding the search path for header files, yet we also noted that this statement is not applicable to the MinGW implementation of GCC. In the case of libraries, the GCC Manual makes a much less definitive statement, which is paradoxically more applicable to the MinGW implementation, regarding the default search path:

-llibrary
-l library
    Search the library named library when linking.  (The second alternative
    with the library as a separate argument is only for POSIX compliance and
    is not recommended.)

    It makes a difference where in the command you write this option; the
    linker searches and processes libraries and object files in the order they
    are specified.  Thus, `foo.o -lz bar.o' searches library `z' after file foo.o
    but before bar.o.  If bar.o refers to functions in `z', those functions may
    not be loaded.

    The linker searches a standard list of directories for the library, which is
    actually a file named liblibrary.a.  The linker then uses this file as if it
    had been specified precisely by name.

    The directories searched include several standard system directories plus
    any that you specify with -L.

    :

Note: the second paragraph of the above extract is not strictly relevant to the subject of this Mini-HOWTO; I've chosen not to elide it, because it answers another FAQ: "Why do I get undefined reference errors, when the symbols in question are definitely present in the libraries I've specified with -l options?" -- the usual answer is that the -l options have been placed too early in the command line argument sequence. The remainder, and in particular the final paragraph, are pertinent, vaguely indicating that libraries will be searched for in several standard system directories; these will be identified more explicitly in this Mini-HOWTO.

Consulting an Alternative Reference Source

The GNU Binary File Utilities Manual's Perspective

We have observed that the GCC Manual makes no definitive statement, regarding the default directories to be searched for libraries. Since the actual library search is performed by the linker, ld, which is a component of the GNU Binary File Utilities, (more commonly known as binutils), we might hope to find a more definitive statement in the applicable section of the Binutils Manual. Unfortunately, while obviously a useful reference for the GNU Binary File Utilities in general, this manual appears to be equally noncommittal; the only references to be found, to library search paths, are in the
Command Line Options section:

 :

-Lsearchdir
--library-path=searchdir
    Add path searchdir to the list of paths that ld will search for archive
    libraries and ld control scripts.  You may use this option any number of
    times.  The directories are searched in the order in which they are specified
    on the command line.  Directories specified on the command line are searched
    before the default directories.  All -L options apply to all -l options,
    regardless of the order in which the options appear.

    If searchdir begins with =, then the = will be replaced by the sysroot prefix,
    a path specified when the linker is configured.

    The default set of paths searched (without being specified with `-L') depends
    on which emulation mode ld is using, and in some cases also on how it was
    configured.  See Environment.

    The paths can also be specified in a link script with the SEARCH_DIR command.
    Directories specified this way are searched at the point in which the linker
    script appears in the command line.

 :

and in the Linker Scripts / File Commands section:

 :

SEARCH_DIR(path)
    The SEARCH_DIR command adds path to the list of paths where ld looks for
    archive libraries.  Using SEARCH_DIR(path) is exactly like using `-L path'
    on the command line (see Command Line Options).  If both are used, then the
    linker will search both paths.  Paths specified using the command line option
    are searched first.

 :

while the Environment reference, appearing within the Command Line Options reference, apparently has nothing to say, which is related to this subject.

Determining MinGW's Default Library Search Path

So, if the applicable manuals don't give us any definitive indication of which directories will be searched for libraries, how can we identify where the MinGW tools will search?

The ld manual tells us that the directories searched will be those specified on the command line, using -L options, followed by those specified in the effective linker script, using the SEARCH_DIR script command. We may identify the search directories specified within the default linker script, by running the command:

ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

which, with a standard MinGW installation, might produce output similar to:

$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
SEARCH_DIR("/mingw/mingw32/lib")
SEARCH_DIR("/mingw/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib")

The first thing we might notice about these, is that they are all specified in the style of POSIX paths; however, since the MinGW tools are all native MS-Windows applications, the linker will be unable to resolve these paths in any manner other than as absolute with respect to the current drive, at the time when the linker is invoked. Thus, even if these directories do exist, (which is unlikely in a standard MS-Windows installation), they may represent different locations, depending on whichever drive the user has made current, at the time when the linker is invoked. (For users of MSYS, they will definitely not be resolved relative to the root of the MSYS virtual file system, even if that is what the user might like).

Now, we have seen that none of its pre-configured search paths seem to be particularly useful as default search paths, for use with the MinGW linker. That may be considered to be a configuration error, (which a future binutils package release may address); nevertheless, it does not appear to present a significant obstacle to operation in practice, since suitable search paths may always be specified using -L options. Of course, any such options which we are required to specify on the command line, can hardly be described as default search paths; however, if we adopt the recommended practice of always invoking ld via an appropriate GCC front-end, then the GCC driver has an opportunity to supply such options, in a manner which may be so described, and in fact, this is precisely what happens.

Thus, we have established that the linker's default search paths have little value, in a standard MinGW installation, but that GCC itself furnishes the effective defaults, by supplying appropriate -L options. To discover what these default -L options are, we might use the command:

gcc -print-search-dirs

If we run this, we will see a sprawling mess of nigh incomprehensible output; some judicious filtering, to extract only the library search paths, normalise them to canonical forms, and present them one per line, can reduce it to a form which is more readily interpreted:

$ gcc -print-search-dirs | \ 
> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \ 
> tr \; \\012
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  /mingw/lib/gcc/mingw32/3.4.5/
  /usr/lib/gcc/mingw32/3.4.5/
  c:/mingw/mingw32/lib/mingw32/3.4.5/
  c:/mingw/mingw32/lib/
  /mingw/mingw32/lib/mingw32/3.4.5/
  /mingw/mingw32/lib/
  /mingw/lib/mingw32/3.4.5/
  /mingw/lib/
  c:/mingw/lib/mingw32/3.4.5/
  c:/mingw/lib/
  /mingw/lib/mingw32/3.4.5/
  /mingw/lib/
  /lib/mingw32/3.4.5/
  /lib/
  /usr/lib/mingw32/3.4.5/
  /usr/lib/

Once again, we may observe that this list includes a number of POSIX style paths, which most likely are not relevant for a MinGW installation. If we further filter, to remove the POSIX paths, and keep only the MS-Windows style paths, this list reduces to:

$ gcc -print-search-dirs | \ 
> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \ 
> tr \; \\012 | \ 
> grep -v '^ */'
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/mingw32/3.4.5/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/mingw32/3.4.5/
  c:/mingw/lib/

From this, we might deduce that GCC will pass each of these paths to the linker, using an appropriate -L option; to verify that deduction, we might use GCC's "-###" option, in a command such as:

gcc -### -o foo foo.c

to reveal exactly what command GCC will pass to the linker; (again, the output is not formatted for easy reading, but some judicious filtering will extract just the -L options, and present them in a form which can be more easily read by humans):

$ touch foo.c
$ gcc -### -o foo foo.c 2>&1 | \ 
> tr -s \\040 \\012 | \ 
> sed '/^"-L/!d;s,,,;s,"$,/,;:1;s,/[^/.][^/]*/\.\./,/,;t 1' | \ 
> sed 's,^,  ,;1h;1s,.*,libraries:,;1G'
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/

Oops! What happened there? In these examples, GCC's -print-search-paths option tells us that there are six MS-Windows paths it will search, but only four of those are passed on to the linker, as -L options. Why are all six not passed on?

Well, recall that we arbitrarily chose to ignore the POSIX style paths in GCC's built-in list, when we formulated the list of MS-Windows paths we expected to be searched, but why would GCC itself make such an arbitrary choice? In reality, it doesn't. Let's re-examine that full list of built-in paths, but this time we will filter it on an analytical basis, selecting only those paths which represent actual physical directories on our host machine:

$ ( echo libraries:
>   drive=`pwd -W | sed 's,:.*,,'`
>   for dir in `gcc -print-search-dirs \ 
>     | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,' \ 
>     | tr \; '\012' \ 
>     | sed "s,^/,$drive:/,"`
>     do test -d $dir && echo "  $dir"
>     done
> )
libraries:
  c:/mingw/lib/gcc/mingw32/3.4.5/
  c:/mingw/lib/gcc/
  c:/mingw/mingw32/lib/
  c:/mingw/lib/

which, we observe, now exactly matches the list of -L options we see passed to the linker, when we invoke GCC with the "-###" option. Thus, we may refine our previous deduction, to conclude that GCC will examine it's built-in list of pre-configured library search directories, passing on to the linker, in the form of -L options, those which represent actual physical directories on the host machine; therefore, this becomes the list of default library search directories for MinGW.

Re: HOWTO Specify the Location of Libraries for use with MinGW

The LIBRARY_PATH environment variable is used by gcc. This is what the man page says about it:

The value of LIBRARY_PATH is a colon-separated list of directories, much like PATH. When configured as a native compiler, GCC tries the directories thus specified when searching for special linker files, if it can’t find them using GCC_EXEC_PREFIX. Linking using GCC also uses these directories when searching for ordinary libraries for the -l option (but directories specified with -L come first).

Re: HOWTO Specify the Location of Libraries for use with MinGW

keith's picture

Yes indeed. Notice that this is still a work in progress, and LIBRARY_PATH is one of the customisation options, which needs to be mentioned in the text yet to be written. Do note, however, that on MS-Windows it is not a colon-separated list; it needs to be semicolon-separated, with all of its members specified in native MS-Windows format; (however, MSYS users may specify it as a colon-separated list, if all of its members are specified in MSYS' emulated POSIX style, and it will be converted appropriately, but mixed POSIX and MS-Windows styles are unsupported).

A further point to note: the manpage snippet cited is somewhat ambiguous, but it suggests that LIBRARY_PATH is not used if mingw32-gcc is built as a cross-compiler. This feature may be of use only to those running MinGW as a native compiler on MS-Windows; it may have no value, for those of us who run it cross hosted.

Thanks for this comment. The focus of the article is more on setting the system up to avoid dependency on a proliferation of environment variables; however, the comment is pertinent, and a valuable source of additional information, until such time as I cover this aspect within the body text. When I get to that point, I will discard this attached comment thread.

Re: HOWTO Specify the Location of Libraries for use with MinGW

Thank you for explaining in more details that man page paragraph. When reading it, It's indeed easy to miss the fact it's not usable with a cross-compiler!
I was also wondering what the man page refers to with "searching for special linker files". Are these the specs files described in the SpecsFileHOWTO page?

Re: HOWTO Specify the Location of Libraries for use with MinGW

keith's picture

No, the specs files are specific to configuration of the GCC front-end drivers. As I understand it, special linker files would be the linker scripts, referred to in the ld section of the binutils manual.

Site Status

Site maintenance completed May 25th, 2012 at 12:38 UTC