Linking dylib fails in some cases, "undefined reference to `rust_begin_unwind"

[roeyskatt]

Here's a minimal code example that reproduces the error:

#![crate_name = "test"]
#![crate_type = "dylib"]

use std::ops::Div;

pub struct Test
{
    pub test:i32,
}

impl Div<Test,Test> for Test { fn div( &self, v:&Test ) -> Test { Test{ test:self.test / v.test } } }

And part of the error:

note: D:\#dev\#compilers\Rust\bin\rustlib\i686-pc-windows-gnu\lib\libcore-4e7c5e5c.rlib(core-4e7c5e5c.o):(.text+0x6b9b): undefined reference to `rust_begin_unwind'

Version:

rustc 0.13.0-nightly (45cbdec41 2014-11-07 00:02:18 +0000)
binary: rustc
commit-hash: 45cbdec4174778bf915f17561ef971c068a7fcbc
commit-date: 2014-11-07 00:02:18 +0000
host: i686-pc-windows-gnu
release: 0.13.0-nightly

This also happens on the x86_64 nightly

[vadimcn]

Duplicating all rlibs on the command line makes the error go away. Which means this is a link ordering issue between Rust rlibs.
We'll probably need to do topological sorting of the libs before passing them to the linker. :(

[alexcrichton]

I believe that this is because the program here isn't actually using any symbols from the standard library. We do actually do a topological sort of the libs on the command line. The linker command line looks like:

gcc ... program.o -lstd ... -lcore

However, this program doesn't actually use any symbols from the standard library's rlib, so the linker actually discards -lstd after it looks at it, moving on to the next library. When it reaches -lcore (which has a reverse dependency back onto libstd), we've stripped libstd, so rust_begin_unwind is lost, and it becomes and undefined symbol.

The best way to solve this... I'm not entirely sure! I've thought in the past that each object generated by rustc needs a symbol which can be referenced by objects to guarantee the linker doesn't strip any of them, which would definitely help here but is slightly heavy-handed...

[vadimcn]

@alexcrichton, if we topologically sort them, shouldn't -lstd have come after -lcore? I thought we simply piled all upstream libraries onto linker command line. Can you point me to the code that does topo-sorting?

[alexcrichton]

My current understanding of linkers leads me to believe that we need a topological sorting with the outermost dependencies at the far left and their own dependencies to the right. For example, if I have a crate c1 that links to c2 which links to c3, we have two options:

gcc c1.o -lc2 -lc3
// or
gcc c1.o -lc3 -lc2

[alexcrichton]

Oops, posted too soon!

Anyway, let's assume that c1 calls c2 functions, but does not call any c3 functions. We do know, however, that c2 calls c3 functions. This means that in the first example, there are outstanding undefined references when -lc2 is encountered, and it resolves some of them, so we keep the lib. This then keeps -lc3 as well.

In the second case, when the linker looks at -lc3 the library doesn't actually satisfy any outstanding undefined references, so the linker forgets about it, includes -lc2 and then errors with undefined references to -lc3 which it already forgot about.

The current topo-sort is here:

rust/src/librustc/metadata/cstore.rs

Lines 156 to 184 in 351f7af

	pub fn get_used_crates(&self, prefer: LinkagePreference)
	-> Vec<(ast::CrateNum, Option<Path>)> {
	let mut ordering = Vec::new();
	fn visit(cstore: &CStore, cnum: ast::CrateNum,
	ordering: &mut Vec<ast::CrateNum>) {
	if ordering.as_slice().contains(&cnum) { return }
	let meta = cstore.get_crate_data(cnum);
	for (_, &dep) in meta.cnum_map.iter() {
	visit(cstore, dep, ordering);
	}
	ordering.push(cnum);
	};
	for (&num, _) in self.metas.borrow().iter() {
	visit(self, num, &mut ordering);
	}
	ordering.as_mut_slice().reverse();
	let ordering = ordering.as_slice();
	let mut libs = self.used_crate_sources.borrow()
	.iter()
	.map(|src| (src.cnum, match prefer {
	RequireDynamic => src.dylib.clone(),
	RequireStatic => src.rlib.clone(),
	}))
	.collect::<Vec<(ast::CrateNum, Option<Path>)>>();
	libs.sort_by(|&(a, _), &(b, _)| {
	ordering.position_elem(&a).cmp(&ordering.position_elem(&b))
	});
	libs
	}

[vadimcn]

I think linker works ever simpler than that: it goes through the libraries on the command line left-to-right, and at each step tries to resolve currently outstanding symbols. It never examines libraries to the left of the current one.
So even if it had already used c2 to resolve some symbols, if there are new dependencies on c2 later down the command line, it wouldn't be able to resolve them.

[vadimcn]

@alexcrichton, I wonder if we have circular dependencies in this case. Does anything in std depend on core?

[alexcrichton]

You can't actually encode a circular dependency in Rust itself, but std/core do have a circular relationship where std depends heavily on core for all its symbols and core depends on the "weak lang items" coming from std (aka rust_begin_unwind). That's the source of the problem here sadly :(

[vadimcn]

If it's a one-off, I suppose it doesn't make sense to build the logic for dealing with circular dependencies into get_used_crates(). Could we simply add #[link_args = "-Wl,-u_rust_begin_unwind"] into libcore to tell the linker that we are going to need this symbol?

[alexcrichton]

Yeah I'm ok with adding a linker flag for specifically symbol for now.

[vadimcn]

No luck, #[link_args] doesn't get encoded into crate metadata :(
Should it have been encoded?

[vadimcn]

@roeyskatt, just make sure that you call panic!() at least once in your code and everything will be fine. :-P

[roeyskatt]

@vadimcn hehe, thanks :)