`kraftld`: Support exposing Kraftkit as linker

[mkroening]

Feature request summary

To create applications from other compilers such as rustc, we need to hide the KraftKit invocation behind a linker facade.

For this purpose, I wrote a shell script (linker.sh). It would be great if this functionality would instead be made available directly from KraftKit via some kraftld in the PATH.

This is closely intertwined with unikraft/unikraft#957.
That PR has been created to smuggle LDFLAGS around KraftKit.
The introduction of UK_RELINK_IF_CHANGED is still relevant, even for a KraftKit-internal solution.

Describe alternatives

We experimented with making Unikraft produce a static library.
Injecting the Unikraft build process into another compiler via this linker-shim, is much more flexible and transparent, though.

Related architectures

None

Related platforms

None

Additional context

Current functionality provided by the current shell script:

Usage:

export PATH=$PATH:path-to-linker.sh

rustc --target x86_64-unikraft-linux-musl -Clinker=linker.sh main.rs

Sample input for linker.sh from rustc:

-m64
/tmp/rustcqNImQS/symbols.o
main.main.50dcbb9212ef9486-cgu.0.rcgu.o
main.v1olu94k1eypyx0.rcgu.o
-Wl,--as-needed
-L
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib
-Wl,-Bstatic
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libstd-26be2b97759b6f23.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libpanic_abort-14cb9590ed4add28.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libobject-eba2c41d8f746f78.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libmemchr-47392a88e333f6c5.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libaddr2line-ed51d5457e8052c7.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libgimli-a33c6392960bc93c.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_demangle-2d8d24d8a79ffbcd.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libstd_detect-ec6ba5d72cb96249.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libhashbrown-5e51b16d5d2cdb0c.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_std_workspace_alloc-0a3d52a89e55d966.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libminiz_oxide-f8e2aa57ca5a463f.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libadler-362befd076ed55a6.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libunwind-56ca020e61338f19.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcfg_if-48c6212d9e9b8fa5.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/liblibc-9d26b9905cf1e604.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/liballoc-a32a9919a0c78ccf.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_std_workspace_core-b2239ec26196b2e7.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcore-3ab1e2a4abbd1396.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcompiler_builtins-28ade668a73fc743.rlib
-Wl,-Bdynamic
-lgcc_s
-lc
-Wl,--eh-frame-hdr
-Wl,-z,noexecstack
-L
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unikraft-linux-musl/lib
-o
main
-Wl,--gc-sections
-no-pie
-nodefaultlibs

Sample input from cargo-driven rustc:

-m64
/tmp/rustc3Jg7xq/symbols.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.10sgik8vww36jtef.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.12n66o84o1yhf6vz.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.1cbftrqr4vajq3ya.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.1flfehpc05ues1gy.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.21p68b6oqa57or0i.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.2johbi6esss7emma.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.2nezplsnpezju8wz.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3dr5i854cmc785jx.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3eb8h6eeocsiwhx7.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3mvqk14hbdlxxxfb.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3sik8pokvkwufyh.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3taw515gz462qgp.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3tq7931f34oznyca.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3utif144y5t37co5.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.3vlr1r1mckzrqf64.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.44i7kqffhk3c4x25.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.4ipjnf0na7a7zigu.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.4numkwppjvyi03gp.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.4svoni65rvydpyey.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.5eqgwbluz6uousuz.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.9xoh71w1124e8hl.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.bdvutzvdk49arjn.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.obuj6bfwlk8b4w9.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.qbv72oqmnj2c70c.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.qcv3h2y7f1rgawe.rcgu.o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0.4jjrgbg7cy6ipqzz.rcgu.o
-Wl,--as-needed
-L
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps
-L
/home/kroening/devel/unikraft-demo/target/debug/deps
-L
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib
-Wl,-Bstatic
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libtiny_http-b656a636e5f0c7ea.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/liblog-c1c2f1e673187e88.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libchunked_transfer-1ab26313756874f8.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libtime-1615c1867e6d5004.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libitoa-ea74d19603951e83.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libtime_core-6e85975127942268.rlib
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/libascii-2e70ef20b31b6dcc.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libstd-37dcd027dd7c05c8.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libpanic_abort-f9acf353e7f84e75.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libobject-0346ffbacec401f9.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libmemchr-39ede8381d5c1a06.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libaddr2line-ecc7e9a8228b47aa.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libgimli-908178188bd84dc2.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_demangle-de616dfe31a0c9d6.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libstd_detect-d74e04ea9d4857f4.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libhashbrown-2eaf8b3ff59d33b3.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_std_workspace_alloc-3a2451b30fb4e2de.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libminiz_oxide-bf34693ae4926fd9.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libadler-3e4613f8f4032fa1.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libunwind-8f4a98b1d905f117.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcfg_if-fbea4c8c0367def7.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/liblibc-b9d311e174fc40e3.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/liballoc-ea956642f680ecad.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/librustc_std_workspace_core-f159b22e2ff6b681.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcore-1b43cc421d48ea7c.rlib
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib/libcompiler_builtins-e67c733ac15be8df.rlib
-Wl,-Bdynamic
-lgcc_s
-lc
-Wl,--eh-frame-hdr
-Wl,-z,noexecstack
-L
/home/kroening/devel/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unikraft-linux-musl/lib
-o
/home/kroening/devel/unikraft-demo/target/x86_64-unikraft-linux-musl/debug/deps/unikraft_demo-384a3381745423d0
-Wl,--gc-sections
-no-pie
-nodefaultlibs

linker.sh

#!/usr/bin/env bash

# Bash strict mode
set -euo pipefail
IFS=$'\n\t'

LDFLAGS="$*"

# Parse arguments
INPUTS=""
while [[ $# -gt 0 ]]; do
    case $1 in
        -o)
            OUTPUT="$2"
            shift
            shift
            ;;
        -L)
            shift
            shift
            ;;
        -*)
            shift
            ;;
        *)
            INPUTS+=" $1"
            shift
            ;;
    esac
done

# Make all relative object paths absolute for make
# rel_path.o -> $PWD/rel_path.o
LDFLAGS=$(echo "$LDFLAGS" | sed -Ee 's,(^[^/].*\.o),'"$PWD"'\/\1,g' )

# Remove -nodefaultlibs
LDFLAGS=${LDFLAGS/-nodefaultlibs/}

export UK_LDFLAGS="$LDFLAGS"

export UK_LDEPS="$INPUTS"

kraft build

# TODO: Don't assume `default_qemu-x86_64` filename
cp .unikraft/build/default_qemu-x86_64 "$OUTPUT"

[mkroening]

@nderjung, I added -o <OUTPUT> parsing and copying to the bash draft. :)

Edit: I also added UK_LDEPS support, so this is working properly now and could be used, aside from the assumption of the output filename default_qemu-x86_64.