Calling Conventions in C, C++, and Rust

A Deep Technical Analysis of Their Impact on the Final ABI

1. Introduction: Calling Convention vs ABI

Before comparing C, C++, and Rust, we must separate two frequently confused concepts:

Calling Convention

Defines:

• How arguments are passed (registers vs stack)

• How return values are delivered

• Which registers are caller/callee saved

• Stack alignment requirements

• Who cleans the stack

ABI (Application Binary Interface)

Much broader. Includes:

• Calling convention

• Type sizes and alignment

• Struct/class layout

• Name mangling rules

• Exception handling model

• Object file format and symbol visibility

• Dynamic linking rules

Calling convention is a component of the ABI — not the whole ABI.

2. The Platform Controls the Real ABI

On modern systems, the platform ABI defines the real low-level rules:

Examples:

• SysV AMD64 ABI (Linux/macOS/BSD on x86-64)

• Windows x64 ABI

• AAPCS64 (AArch64)

C, C++, and Rust compilers generally conform to the platform ABI when generating machine code.

So in practice:

The architecture + operating system define the low-level calling convention. The language defines how much additional ABI surface it adds.

3. C: The Minimal ABI Surface

C is often considered the "binary lingua franca" because:

• No name mangling

• No implicit runtime

• No class layout rules

• No exceptions crossing boundaries

• Simple type system

A C function:

int add(int a, int b);

Will:

• Follow the platform calling convention

• Export a predictable symbol name

• Have stable parameter passing rules

Why C Is Stable in Practice

Because:

• It adds almost nothing beyond the platform ABI.

• All major compilers target the same system ABI contracts.

• Its type system maps cleanly to machine-level constructs.

Where C Still Requires Care

• LP64 vs LLP64 models (long differs)

• Struct padding/alignment

• Bitfields

• Compiler-specific attributes

• Variadic functions

But overall, C offers the narrowest and safest binary interface.

4. C++: Same Calling Convention, Larger ABI Surface

4.1 Function-Level Calling Convention

For simple free functions:

int add(int a, int b);

The actual register usage and stack layout are typically identical to C on the same platform.

So at the lowest level:

C++ often shares the same calling convention as C.

But this is only part of the story.

4.2 What C++ Adds to the ABI

C++ introduces additional binary obligations:

1. Name Mangling

Encodes:

• Function overloading

• Namespaces

• Templates

Different compilers may use different mangling schemes.

2. Class Layout Rules

Includes:

• Padding and alignment

• Multiple inheritance layout

• Empty base optimization

• Virtual base offsets

Layout may differ across:

• Compilers

• Compiler versions

• Flags

3. VTables and RTTI

Virtual functions require:

• vptr placement

• vtable layout rules

• Runtime type information metadata

These are ABI-defined — but not standardized across all toolchains.

4. Exception Handling and Unwinding

C++ requires:

• Stack unwinding metadata

• Personality functions

• Exception object format

Cross-compiler binary compatibility becomes fragile.

4.3 Result

C++ often uses the same low-level calling convention as C, but:

The C++ ABI is much larger and more complex than C’s ABI.

Therefore:

• Stable cross-compiler C++ binary interfaces are difficult.

• Public C++ shared libraries must tightly control toolchains.

5. Rust: Explicit ABI Choice

5.1 Default Rust ABI

Rust functions without an extern declaration use the "Rust" ABI.

Example:

pub fn add(a: u64, b: u64) -> u64 { a + b }

This does NOT guarantee:

• Stable calling convention across compiler versions

• Stable symbol naming

• Stable layout

Rust does not promise a stable native ABI.

5.2 Rust Opt-In ABI

Rust forces you to be explicit:

C ABI

#[no_mangle]
extern "C" fn add(a: u64, b: u64) -> u64 {
    a + b
}

Now:

• Uses platform C calling convention

• Exports unmangled symbol

• Suitable for FFI

C-Compatible Layout

#[repr(C)]
struct Foo {
    x: u32,
    y: u64,
}

Ensures:

• Field ordering matches C

• Padding rules follow C ABI

Unwinding Control

Rust separates:

• "C" ABI (no unwinding expected)

• "C-unwind" ABI (explicit cross-language unwinding)

This is critical for safety.

5.3 Result

Rust does not define a universal stable ABI.

Instead:

Rust delegates stable binary interfaces to the C ABI.

Rust’s strength is explicitness:

• ABI is opt-in.

• Layout is opt-in.

• Unwinding is opt-in.

6. Real-World Failure Modes in Final ABI

Even if you "use C everywhere", ABI mismatches can occur due to:

1. Platform ABI Differences

• Windows x64 vs SysV AMD64

• AArch64 differences

2. Struct Return Rules (sret)

Large struct return conventions vary.

3. Floating-Point and Vector Registers

Incorrect prototypes can corrupt state.

4. Exception / Panic Crossing

• C++ exceptions crossing into Rust

• Rust panic crossing into C

• Undefined behavior if not managed

5. Memory Ownership Contracts

Who allocates? Who frees? Which allocator?

This is part of ABI contract design — not just calling convention.

7. Comparative Summary

8. Engineering Conclusion

C

Smallest ABI surface. Most stable for long-term binary compatibility.

C++

Shares calling convention with C but adds:

• Mangling

• Vtables

• Exceptions

• Template instantiation complexity

Binary stability requires discipline.

Rust

No stable default ABI. Provides explicit, opt-in C ABI compatibility.

Best practice: Expose C ABI. Hide Rust internally.

9. Final Engineering Rule

If you want:

• Cross-language compatibility

• Cross-compiler stability

• Long-term binary durability

Then:

Design a C ABI surface. Implement internally in C, C++, or Rust. Never expose language-specific ABI details.

Calling conventions may match at the CPU level — but ABI stability is determined by how much semantic complexity the language leaks into the binary boundary.