arm-unknown-linux-gnueabihf build failed

[zonyitoo]

Problem:

  Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/linux-arm/crypto/ to /target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out/build/aws-lc/crypto
  In file included from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/bcm.c:82:
  /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/cpucap/cpu_arm_linux.c: In function 'aws_lc_0_17_0_OPENSSL_cpuid_setup':
  /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/cpucap/cpu_arm_linux.c:121:38: error: 'AT_HWCAP2' undeclared (first use in this function); did you mean 'AT_HWCAP'?
       unsigned long hwcap2 = getauxval(AT_HWCAP2);
                                        ^~~~~~~~~
                                        AT_HWCAP

It looks like that AT_HWCAP2 was missing.

It could be reproduced in this project: https://github.com/shadowsocks/shadowsocks-rust by running:

cross build --release --target arm-unknown-linux-gnueabihf --features "full"

Relevant details

AWS-LC for Rust versions or commit: (crates.io aws-lc-sys v0.17.0)

System information: for linux, below info can be collected by running uname -srvmp

• CPU architecture: ARM
• CPU name: unknown
• OS: Ubuntu 20

Build log:

   Compiling libc v0.2.155
   Compiling proc-macro2 v1.0.83
   Compiling unicode-ident v1.0.12
   Compiling cfg-if v1.0.0
   Compiling once_cell v1.19.0
   Compiling autocfg v1.3.0
   Compiling serde v1.0.202
   Compiling quote v1.0.36
   Compiling syn v2.0.66
   Compiling jobserver v0.1.31
   Compiling lock_api v0.4.12
   Compiling cc v1.0.98
   Compiling scopeguard v1.2.0
   Compiling getrandom v0.2.15
   Compiling rand_core v0.6.4
   Compiling pin-project-lite v0.2.14
   Compiling version_check v0.9.4
   Compiling smallvec v1.13.2
   Compiling bytes v1.6.0
   Compiling typenum v1.17.0
   Compiling parking_lot_core v0.9.10
   Compiling generic-array v0.14.7
   Compiling socket2 v0.5.7
   Compiling parking_lot v0.12.3
   Compiling itoa v1.0.11
   Compiling signal-hook-registry v1.4.2
   Compiling num_cpus v1.16.0
   Compiling memchr v2.7.2
   Compiling slab v0.4.9
   Compiling futures-core v0.3.30
   Compiling crypto-common v0.1.6
   Compiling futures-sink v0.3.30
   Compiling thiserror v1.0.61
   Compiling tracing-core v0.1.32
   Compiling fnv v1.0.7
   Compiling futures-channel v0.3.30
   Compiling futures-io v0.3.30
   Compiling subtle v2.5.0
   Compiling futures-task v0.3.30
   Compiling pin-utils v0.1.0
   Compiling equivalent v1.0.1
   Compiling untrusted v0.9.0
   Compiling hashbrown v0.14.5
   Compiling ring v0.17.8
   Compiling tinyvec_macros v0.1.1
   Compiling tinyvec v1.6.0
   Compiling indexmap v2.2.6
   Compiling spin v0.9.8
   Compiling ppv-lite86 v0.2.17
   Compiling inout v0.1.3
   Compiling serde_derive v1.0.202
   Compiling tokio-macros v2.2.0
   Compiling tracing-attributes v0.1.27
   Compiling thiserror-impl v1.0.61
   Compiling futures-macro v0.3.30
   Compiling zeroize_derive v1.4.2
   Compiling futures-util v0.3.30
   Compiling zeroize v1.7.0
   Compiling rand_chacha v0.3.1
   Compiling rand v0.8.5
   Compiling openssl-src v300.3.0+3.3.0
   Compiling cmake v0.1.50
   Compiling byteorder v1.5.0
   Compiling fs_extra v1.3.0
   Compiling openssl-probe v0.1.5
   Compiling pkg-config v0.3.30
   Compiling dunce v1.0.4
   Compiling vcpkg v0.2.15
   Compiling rustls v0.21.12
   Compiling aws-lc-sys v0.17.0
   Compiling openssl-sys v0.9.102
   Compiling log v0.4.21
   Compiling mio v0.8.11
   Compiling tracing v0.1.40
   Compiling cipher v0.4.4
   Compiling tokio v1.37.0
   Compiling ring v0.16.20
   Compiling bitflags v2.5.0
   Compiling tokio-util v0.7.11
   Compiling pin-project-internal v1.1.5
   Compiling unicode-normalization v0.1.23
   Compiling universal-hash v0.5.1
   Compiling http v1.1.0
The following warnings were emitted during compilation:

warning: aws-lc-sys@0.17.0: Generating bindings - external bindgen. Platform: arm-unknown-linux-gnueabihf

error: failed to run custom build command for `aws-lc-sys v0.17.0`

Caused by:
  process didn't exit successfully: `/target/release/build/aws-lc-sys-dae18cecdecec9a6/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  cargo:warning=Generating bindings - external bindgen. Platform: arm-unknown-linux-gnueabihf
  cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
  cargo:rustc-cfg=use_bindgen_generated
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  CMAKE_TOOLCHAIN_FILE_arm-unknown-linux-gnueabihf = None
  CMAKE_TOOLCHAIN_FILE_arm_unknown_linux_gnueabihf = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_arm-unknown-linux-gnueabihf = None
  CMAKE_GENERATOR_arm_unknown_linux_gnueabihf = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_arm-unknown-linux-gnueabihf = None
  CMAKE_PREFIX_PATH_arm_unknown_linux_gnueabihf = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_arm-unknown-linux-gnueabihf = None
  CMAKE_arm_unknown_linux_gnueabihf = None
  TARGET_CMAKE = None
  CMAKE = Some("cmake")
  running: cd "/target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/cargo/registry/src/index.crates.io-6f17d22bba1[500](https://github.com/shadowsocks/shadowsocks-rust/actions/runs/9237206906/job/25414562078#step:5:501)1f/aws-lc-sys-0.17.0" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_BUILD_TYPE=release" "-DBORINGSSL_PREFIX=aws_lc_0_17_0_" "-DBORINGSSL_PREFIX_HEADERS=/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/generated-include" "-DBUILD_TESTING=OFF" "-DBUILD_LIBSSL=OFF" "-DDISABLE_PERL=ON" "-DDISABLE_GO=ON" "-DCMAKE_SYSTEM_NAME=Linux" "-DCMAKE_SYSTEM_PROCESSOR=arm" "-DCMAKE_INSTALL_PREFIX=/target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv6 -marm -mfpu=vfp" "-DCMAKE_C_COMPILER=/x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-gcc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv6 -marm -mfpu=vfp" "-DCMAKE_CXX_COMPILER=/x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-g++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv6 -marm -mfpu=vfp" "-DCMAKE_ASM_COMPILER=/x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-gcc" "--no-warn-unused-cli"
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 8.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-gcc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Go not found. Disabling some code generation and using pre-generated code in generated-src/
  -- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
  -- Run check_run file_to_test 'memcmp_invalid_stripped_check.c', flag_to_set 'MEMCMP_INVALID_STRIPPED', and compile_flags '-O3 -DNDEBUG'.
  -- stdalign_check.c probe is positive, enabling AWS_LC_STDALIGN_AVAILABLE
  -- builtin_swap_check.c probe is positive, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
  -- The ASM compiler identification is GNU
  -- Found assembler: /x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-gcc
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE  
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out/build
  running: cd "/target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out/build" && MAKEFLAGS="-j --jobserver-fds=8,9 --jobserver-auth=8,9" "cmake" "--build" "." "--target" "install" "--config" "Release"
  [  0%] Built target boringssl_prefix_symbols
  Scanning dependencies of target fipsmodule
  [  1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o
  aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/build.make:75: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o' failed
  CMakeFiles/Makefile2:257: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all' failed
  Makefile:135: recipe for target 'all' failed

  --- stderr
  Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/linux-arm/crypto/ to /target/arm-unknown-linux-gnueabihf/release/build/aws-lc-sys-f9c203f2eb6d1229/out/build/aws-lc/crypto
  In file included from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/bcm.c:82:
  /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/cpucap/cpu_arm_linux.c: In function 'aws_lc_0_17_0_OPENSSL_cpuid_setup':
  /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/cpucap/cpu_arm_linux.c:121:38: error: 'AT_HWCAP2' undeclared (first use in this function); did you mean 'AT_HWCAP'?
       unsigned long hwcap2 = getauxval(AT_HWCAP2);
                                        ^~~~~~~~~
                                        AT_HWCAP
  /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/crypto/fipsmodule/cpucap/cpu_arm_linux.c:121:38: note: each undeclared identifier is reported only once for each function it appears in
  make[2]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o] Error 1
  make[1]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all] Error 2
  make: *** [all] Error 2
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 2

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
Error: Process completed with exit code 1.

[justsmth]

I believe the problem related to the Cross.toml configuration that you have here. See my comment on your other issue. Since I believe these have the same root cause, I will close this issue.

[rami3l]

@justsmth Wait, actually Rustup is now on the same boat (again with rust-lang/rustup#3898) 😅 For cross compilation we have historically used the Docker images provided by rust-lang/rust, and maybe that error was caused by a very outdated version of GCC (we use 8.5.0, and the log above uses 8.3.0)?

The log here is more or less the same thing: https://github.com/rust-lang/rustup/actions/runs/9720503984/job/26831972336?pr=3898

I mean I could try to use cross on our side instead but that migration might not be easy :]

[rami3l]

The real problem is, since I have no intention of using fipsmodule (it's not supported for the target in question according to https://aws.github.io/aws-lc-rs/requirements/linux.html), why I'm getting an error on that part of the codebase?

[justsmth]

..., why I'm getting an error on that part of the codebase?

It's just due to how the code is structured. Code that is related to the certification has been placed into a "fipsmodule" directory, but the additional "delocation" and runtime checks required for certification are only activated for a "FIPS" build. We have some detail about how a "FIPS" build differs from our default build here: https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/FIPS.md

[justsmth]

This comment might be relevant to this build failure: https://github.com/aws/aws-lc/blob/5a4a7d17d589af6d7110eded4155ebaaf5026029/crypto/fipsmodule/cpucap/cpu_arm_linux.c#L116-L121

I have a fairly simple patch for it, but I need to test it:

---
 crypto/fipsmodule/cpucap/cpu_arm_linux.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/crypto/fipsmodule/cpucap/cpu_arm_linux.c b/crypto/fipsmodule/cpucap/cpu_arm_linux.c
index e4ab5ab49..a2c5a1809 100644
--- a/crypto/fipsmodule/cpucap/cpu_arm_linux.c
+++ b/crypto/fipsmodule/cpucap/cpu_arm_linux.c
@@ -118,7 +118,11 @@ void OPENSSL_cpuid_setup(void) {
     // this is now rare (see Chrome's Net.NeedsHWCAP2Workaround metric), but AES
     // and PMULL extensions are very useful, so we still carry the workaround
     // for now.
+#if defined(AT_HWCAP2)
     unsigned long hwcap2 = getauxval(AT_HWCAP2);
+#else
+    unsigned long hwcap2 = 0;
+#endif
     if (hwcap2 == 0) {
       hwcap2 = crypto_get_arm_hwcap2_from_cpuinfo(&cpuinfo);
       g_needs_hwcap2_workaround = hwcap2 != 0;
--

[rami3l]

This comment might be relevant to this build failure: https://github.com/aws/aws-lc/blob/5a4a7d17d589af6d7110eded4155ebaaf5026029/crypto/fipsmodule/cpucap/cpu_arm_linux.c#L116-L121

I have a fairly simple patch for it, but I need to test it:

---
 crypto/fipsmodule/cpucap/cpu_arm_linux.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/crypto/fipsmodule/cpucap/cpu_arm_linux.c b/crypto/fipsmodule/cpucap/cpu_arm_linux.c
index e4ab5ab49..a2c5a1809 100644
--- a/crypto/fipsmodule/cpucap/cpu_arm_linux.c
+++ b/crypto/fipsmodule/cpucap/cpu_arm_linux.c
@@ -118,7 +118,11 @@ void OPENSSL_cpuid_setup(void) {
     // this is now rare (see Chrome's Net.NeedsHWCAP2Workaround metric), but AES
     // and PMULL extensions are very useful, so we still carry the workaround
     // for now.
+#if defined(AT_HWCAP2)
     unsigned long hwcap2 = getauxval(AT_HWCAP2);
+#else
+    unsigned long hwcap2 = 0;
+#endif
     if (hwcap2 == 0) {
       hwcap2 = crypto_get_arm_hwcap2_from_cpuinfo(&cpuinfo);
       g_needs_hwcap2_workaround = hwcap2 != 0;
--

@justsmth Thanks! The patch looks reasonable to me. Is there a way I can easily test it within the current cargo setup though?

[justsmth]

The change to address this in AWS-LC made it into the release yesterday: https://github.com/aws/aws-lc/releases/tag/v1.31.0

Hopefully, we can get a PR up today for aws-lc-sys to align it with that release.