Can a dynamically linked library override a static one?

Question

nokogori gem comes with its own version of libxml2. Moreover it warns about libxml2.so of a different version being loaded before it was required:

      if compiled_parser_version != loaded_parser_version
        ["Nokogiri was built against LibXML version #{compiled_parser_version}, but has dynamically loaded #{loaded_parser_version}"]

It basically compares LIBXML_DOTTED_VERSION macro and xmlParserVersion global variable:

  rb_const_set( mNokogiri,
                rb_intern("LIBXML_VERSION"),
                NOKOGIRI_STR_NEW2(LIBXML_DOTTED_VERSION)
              );
  rb_const_set( mNokogiri,
                rb_intern("LIBXML_PARSER_VERSION"),
                NOKOGIRI_STR_NEW2(xmlParserVersion)
              );

And I'm experiencing it firsthand. When rmagick (which dynamically links to libxml2.so, ldd confirms that) is required before nokogiri, the latter complains.

From what I can see nokogiri is linked to libxml2 statically. First that is the default (supposedly). Then when rmagick is not required I can't see libxml2.so in /proc/PID/maps. I neither can see another version of libxml2.so. ldd doesn't list libxml2.so as a nokogiri.so's dependency. objdump lists xmlReadMemory (and friends) as a nokogori.so's symbol (probably a sign that it was linked statically).

So how come can nokogiri access libxml2.so's variables? Does that mean that loading libxml2.so overrides any statically linked versions? Can that happen in the middle of code execution?

I think perhaps you are confusing static linking. Any mention of .so means you are looking at a dynamically linked library, not statically linked. — Gem Taylor, Dec 18 '19 at 15:54
@GemTaylor I'm indeed not sure what are sure signs that it's linked statically. But everything suggests that it is. That is, that `nokogiri.so` is linked statically to `libxml2`. And I'm sure that `rmagick.so` is linked dynamically to `libxml2`. Ruby loads `nokogiri.so` and `rmagick.so` when they are `require`d. — x-yuri, Dec 18 '19 at 16:08

score 2 · Accepted Answer · answered Dec 20 '19 at 04:27

So how come can nokogiri access libxml2.so's variables?

This is by design (and due to the fact that nokogiri was built incorrectly).

UNIX shared libraries are designed to emulate archive libraries. This means that the first ELF image to export a given symbol wins (there are some complications for libraries linked with -Bstatic flag, but we'll ignore them for now).

Does that mean that loading libxml2.so overrides any statically linked versions?

Yes, if statically linked version is also exported and calls to it go though PLT.

Can that happen in the middle of code execution?

With lazy symbol resolution (which is default, except when LD_BIND_NOW or -z now are in effect) it will always happen in the middle of code execution.

Now, the problem is that if nokogiri links in a static copy of libxml.a, it should hide that fact by localizing that copy within itself and not exporting any of its symbols. That would prevent end users from having to deal with symbol conflicts.

Your best bet is to either build your own nokogiri compiling and linking it against the same version of libxml, or to contact nokogiri maintainers and ask them to fix their builds.

Awesome answer. But can you elaborate please? What are archive libraries? To build correctly one has to add [`-fvisibility=hidden`](https://stackoverflow.com/a/43626454/52499)? — x-yuri, Dec 20 '19 at 08:39
And by any chance do you know a reason why they decided to emulate archive libraries? Also, I've filed a [bug report](https://github.com/sparklemotion/nokogiri/issues/1959). In case you want to correct my phrasing or elaborate on the solution. — x-yuri, Dec 20 '19 at 10:52

Can a dynamically linked library override a static one?

1 Answers1

Linked