ld: Using -rpath,$ORIGIN inside a shared library (recursive)

Question

I just made a basic example of using ld's -rpath option with $ORIGIN here (see 2nd response for a working version). I'm trying to create an example where main.run links to foo.so, which in turn links to bar.so, all using rpath and $ORIGIN.

The run-time file-structure is:

• project/
  • lib/
    • dir/
      • sub/
        • bar.so
      • foo.so
  • run/
    • main.run (failing to build)

I'm building foo.so using:

g++ -c -o obj/foo.o src/foo.cpp -fPIC
g++ -shared -o lib/dir/foo.so obj/foo.o -Wl,-soname,foo.so -Wl,-rpath,'$ORIGIN/sub' -Llib/dir/sub -l:bar.so

Which builds fine. ldd lib/dir/foo.so can even find bar.so.

However, when I try to link main.run to foo.so, foo.so can't find bar.so.

I'm building main.so using:

g++ -c -o obj/main.o src/main.cpp
g++ -o run/main.run obj/main.o -Wl,-rpath,'$ORIGIN/../lib/dir' -Llib/dir -l:foo.so

This works fine if another version of foo.so is used that doesn't recursively link. (Uncomment lines in make.sh, in project below, to test).

However, using the normal foo.so I'm getting this error when building main.run:

/usr/bin/ld: warning: bar.so, needed by lib/dir/foo.so, not found (try using -rpath or -rpath-link)

So my questions are:

1. Does $ORIGIN within foo.so resolve to project/lib/dir (where foo.so is) or project/run (where main.run (the executable linking it) is)?
   ldd would seem to indicate that it's project/lib/dir, which would seem to be the best way (although I tried assuming both).
2. How do I get these to link (while preserving relocatability) - preferably without using -rpath-link.

You can download the project here. It's as simple as I can make it. 4 short sources and a script.
After extracting, just run ./make.sh from within project/.

Note: I'm using -l:. This shouldn't change anything except that the libraries are named like foo.so instead of libfoo.so, and lunk with -l:foo.so instead of -lfoo.

Answer 1

Well, I have something working. But I do not really understand why it works. This feels like a bug in ld to me.

I ran strace -f -o /var/tmp/strace.out -- g++ ... for the main.run compilation. The static linker is actually trying to open files whose literal name looks like "$ORIGIN/lib/dir/sub/bar.so", among 20-30 other things. (In other words, it is looking for an actual directory named $ORIGIN. Seriously.)

It also appears to be searching the -rpath-link path for the name "lib/dir/sub/bar.so", not just "bar.so". I have no clue why.

Anyway, this is the link for main.run that is working for me:

g++ -o run/main.run obj/main.o -Wl,-rpath,'$ORIGIN/../lib/dir' -Wl,-rpath-link,. -Llib/dir -l:foo.so

It is identical to yours but with -Wl,-rpath-link,. inserted.

[addendum]

OK I think I see what is going on. First, the static linker (GNU ld) simply does not honor $ORIGIN in the libraries it links against.

Second, the behavior when you use -lbar versus -l:bar.so is very different.

Run readelf -a on foo.so. In your build, it shows a dependency on "lib/dir/sub/bar.so". This is why setting the rpath-link to "." fixes the build of main.run; it causes the static linker to search "." for "lib/dir/sub/bar.so", which it finds.

If you rename bar.so to libbar.so, and link foo.so to use -lbar instead of -l:bar.so, the same readelf shows that foo.so now depends on "libbar.so" (with no path component). With that foo.so, you can get the main.run link to work using -Wl,-rpath-link,lib/dir/sub, as you would expect if you knew that the static linker simply does not honor $ORIGIN.

By the way, I do not see the -l:bar.so syntax documented anywhere in the GNU ld manual. Out of curiosity, how did you come up with it?

Assuming it is a supported feature, this looks a bit like a bug (-l:bar.so creating a dependency on lib/dir/sub/bar.so instead of just bar.so). You can either deal with this bug by setting rpath-link to '.' for main.run, or you can rename stuff in the usual way (libxxx.so).