dlclose doesn't really unload shared object, no matter how many times it is called

Question

My program uses dlopen to load a shared object and later dlclose to unload it. Sometimes this shared object is loaded once again. I noticed static variables are not re-initialized (something which is crucial to my program) so I added a test (dlopen with RTLD_NOLOAD) after dlclose to see if the library is really unloaded. Sure enough, it was still in memory.

I then tried calling dlclose repeatedly until the library is really unloaded, but what I got was an infinite loop. This is the code I'm using to check if the library was unloaded:

dlclose(handles[name]);

do {
  void *handle = dlopen(filenames[name], RTLD_NOW | RTLD_NOLOAD);
  if (!handle)
    break;

  dlclose(handle);
} while (true);

My question is, what are the possible reasons for my shared object not being unloaded after dlclose, given that my dlopen calls are the only places where it is loaded. Can you suggest a course of action to track down the source of the problem? Also, why are repeated calls to dlclose have no effect, they are each decrementing the reference count, aren't they?

EDIT: Just found out that this happens only when I compile with gcc. With clang, everything is just fine.

Answer 1

The POSIX standard actually does not require dlclose to ever unload a library from address space:

Although a dlclose() operation is not required to remove structures from an address space, neither is an implementation prohibited from doing so.

Source: The Open Group Base Specifications Issue 6

That means other than invalidating the handle, dlclose is not required to do anything at all.

Sometimes unloading is also delayed by the system, it just marks the library as "to be removed" and will actually perform that operation at some later time (for efficiency or because it would simply not be possible to perform that operation right now). However, if you call dlopen again before it ever was performed, the flag is cleared and the still loaded library is reused.

In some cases the system knows for sure that some symbols of the library are still in use, in that case it will not unload it from address space to avoid dangling pointers. In some cases the system doesn't know for sure that they are in use, but it also can impossibly tell for sure that they are not, better being safe than sorry, it will just never really remove that library from memory in such a case.

There are other more obscure cases depending on operation system kind and often also on version. E.g. a common Linux issue is if you created a library that uses STB_GNU_UNIQUE symbols, that library is marked as "not unloadable" and thus will simply never be unloaded. See here, here (DF_1_NODELETE means not unloadable) and here. So it can also depend on what symbols or kind of symbol a compiler generates. Try running readelf -Ws on your library and look for objects tagged as UNIQUE.

In general, you cannot really rely on dlclose to work as you might expect. In practice I saw it "fail" more often than "succeed" in the last ten years (well, it never really failed, it just often did not unload the library from memory; yet it worked as required by the standards).