`musttail` doesn't optimize sibling calls

[brandtbucher]

For example, this code:

int bar(int);

int
foo_tail(int x)
{
    // Correct, compiles to a single jmp:
    return bar(x);
}

int
foo_musttail(int x)
{
    // Buggy, loads and re-spills all parameters:
    __attribute__((musttail)) return bar(x);
}

Compiles to:

_foo_tail:
  jmp _bar

_foo_musttail:
  movl 4(%esp), %eax
  movl %eax, 4(%esp)
  jmp _bar

Compiler Explorer

Adding more parameters to the function (and tail call) increases the number of redundant loads/spills, too.

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: Brandt Bucher (brandtbucher)

For example, this code:

int bar(int);

int
foo_tail(int x)
{
    // Correct, compiles to a single jmp:
    return bar(x);
}

int
foo_musttail(int x)
{
    // Buggy, loads and re-spills all parameters:
    __attribute__((musttail)) return bar(x);
}

Compiles to:

_foo_tail:
  jmp _bar

_foo_musttail:
  movl 4(%esp), %eax
  movl %eax, 4(%esp)
  jmp _bar

Compiler Explorer

Adding more parameters to the function (and tail call) increases the number of redundant loads/spills, too.

[brandtbucher]

Actually, this isn't a 32-bit issue, it's just easier to trigger with fewer parameters there. Here's the same thing on 64-bit: https://godbolt.org/z/1cao7Gxhv

Turns out musttail just doesn't optimize sibling calls. Patch incoming.