9

I'm trying to cross-compile a rust project for arm-linux-musleabihf and am hitting a linker error when using musl-cross-make. The rust project has a dependency on libgit2 and this is the dependency that seems to be causing the problem.

Using:

  • the latest rust (1.43.1 via rustup)
  • the arm-unknown-linux-musleabihf target
  • the latest musl-cross-make with TARGET=arm-linux-musleabihf
  • pointing TARGET_CC_linux_arm-unknown-linux-musleabihf and CARGO_TARGET_ARM_UNKNOWN_LINUX_MUSLEABIHF_LINKER at /opt/musl-cross-make/output/bin/arm-linux-musleabihf-gcc

I get an error when building:

error: linking with `/opt/musl-cross-make/output/bin/arm-linux-musleabihf-gcc` failed: exit code: 1
...
  = note: /opt/musl-cross-make/output/bin/../lib/gcc/arm-linux-musleabihf/9.2.0/../../../../arm-linux-musleabihf/bin/ld: /tmp/rustcvSvGAJ/liblibgit2_sys-e56c2f9bd024a0a9.rlib(odb.o): in function `git_odb__add_default_backends':
          odb.c:(.text.git_odb__add_default_backends+0x24): undefined reference to `__stat_time64'
          /opt/musl-cross-make/output/bin/../lib/gcc/arm-linux-musleabihf/9.2.0/../../../../arm-linux-musleabihf/bin/ld: /tmp/rustcvSvGAJ/liblibgit2_sys-e56c2f9bd024a0a9.rlib(config.o): in function `git_config_add_file_ondisk':
          config.c:(.text.git_config_add_file_ondisk+0x34): undefined reference to `__stat_time64'
          /opt/musl-cross-make/output/bin/../lib/gcc/arm-linux-musleabihf/9.2.0/../../../../arm-linux-musleabihf/bin/ld: /tmp/rustcvSvGAJ/liblibgit2_sys-e56c2f9bd024a0a9.rlib(config_file.o): in function `config_file_read':
          config_file.c:(.text.config_file_read+0x48): undefined reference to `__stat_time64'
...etc...

It looks like the linker is having difficulty resolving the musl-specific time64 symbols, and it's not clear why.

This works fine if:

  • I use the x86_64-linux-musl target on both rust and musl-cross-make
  • I build musl-cross-make with MUSL_VER=1.1.24

I also wrote a little C program that uses both time and stat, and this builds on musl 1.2.0 on the cross compiler without any issue.

What's going on here? What's special about libgit2 which means that it can't find the right __time64 symbols?

growse
  • 3,554
  • 9
  • 43
  • 66

3 Answers3

7

The problem is that the MUSL libc that ships with Rust versions up to and including 1.44 is MUSL libc 1.1.X based, not 1.2.0.

When Rust builds libgit2 is also needs to build libgit2-sys, which is C code. When building libgit2-sys it uses the MUSL libc version that you've built and installed yourself with musl-cross-make. This is 1.2.0 by default, which has reworked time_t 32/64 bit support, having a dependency on __stat_time64 and various other similarly named methods (such as __time64).

However, when Rust is linking your final application it is using the version of MUSL libc shipped with Rust for the x86_64-linux-musl target, which doesn't include that new time_t support, and doesn't have __stat_time64, __time64, etc. So the link fails.

Right now you have 2 options:

  • Use musl-cross-make to build 1.1.24 of MUSL lib. I'm not 100% what version is included with Rust 1.44. It may be 1.1.22, although from my own testing I found 1.1.24 works just fine (as have you).
  • Build your own version of Rust and/or the target liblibc.rlib with your own version of MUSL libc. I've never managed to successfully achieve this myself, although this might be a good place to start if you want to try this.

The general issue of using libc shipped with Rust, instead of a locally provided version, is being worked on, e.g. here.

Piers
  • 186
  • 1
  • Thanks for this - it makes sense. I might see if I can build my own rust with musl 1.2.0, and also follow along the linked issue. – growse Jun 23 '20 at 19:58
  • I'm not entirely sure how to go with option 1: - install musl-cross-make - download the source for MUSL lib of the correct version - compile it But then how do I get Rust (or cargo?) to use that version? – V Tolmer Nov 19 '21 at 14:27
3

You could try the instructions listed in https://github.com/richfelker/libcompat_time32:

Add -Wl,--whole-archive -lcompat_time32 -Wl,--no-whole-archive to the link command line. Special hacks (like prepending -Wl, to -lcompat_time32) may be needed if libtool is intercepting and mangling the command line.

Solomon Ucko
  • 5,724
  • 3
  • 24
  • 45
  • 1
    Doesn't seem to make much of a difference. Building `libcompat_time32.a` and trying to add it in with `-Clink-arg=-L/path/to/libcompat_time -Clink-arg=-lcompat_time32` still gives the same issue. – growse Jun 03 '20 at 16:04
0

I would recommend trying out https://github.com/rust-embedded/cross : it sets up the environment for cross-compilation in docker and seamlessly works, even for the case highlighted here.

AFAICT, they recompile musl with the correct version.

V Tolmer
  • 160
  • 9