What Breaks First When You PInvoke a C++ Object from a Native AOT Binary

You've gone Native AOT. Startup is snappy. Binary is lean. But then you call into that old C++ library via PInvoke, and things start breaking in weird ways. Not crashes — more like silent data corruption, or a vtable pointer that points to the wrong function. The JIT used to smooth over these edges; now they're razor sharp. Here's what hits first, and how to dodge it.

Why This Topic Matters Now

The rise of Native AOT in .NET

Two years ago, if you shipped a .NET app that talked to a C++ library via PInvoke, you didn't think twice. JIT compilation handled the marshaling dance, resolved function pointers at runtime, and—frankly—let you get away with sloppy interop. That era is ending. Native AOT is the new default for cloud functions, CLI tools, and any binary where cold-start latency matters. The shift changes everything about interop, and most teams only discover this the morning after their first deployment. A customer reports a crash. You reproduce it locally—with JIT—and nothing breaks. That gap between developer machine and production AOT binary is exactly where the pain starts.

PInvoke's hidden assumptions

Standard PInvoke carries silent contracts you never had to honor before. The JIT runtime initializes the marshaling infrastructure lazily—it resolves unmanaged function addresses, validates calling conventions, and even patches up struct layouts that differ between your C# and C++ definitions. Native AOT discards that safety net. It bakes all interop bindings at compile time, which means any mismatch between your managed and unmanaged expectations hardens into a binary fault. Wrong calling convention? Silent stack corruption. Mismatched enum size? Undefined behavior that only surfaces when production traffic hits. The catch is that your unit tests pass—they always pass—because the JIT runtime fills the gaps. That shift from lazy resolution to eager compilation turns warnings into bricks.

'We spent three days chasing a segfault that only appeared in our AOT-compiled Linux container. Turned out our C++ struct had packed alignment, but the C# struct didn't. JIT never complained. AOT built the wrong memory layout into the binary.'

— Infrastructure engineer at a SaaS monitoring startup, 2024 incident postmortem

Real-world pain points from devs

The problems cluster around three areas. First, struct layout assumptions. C++ classes with virtual tables? You can't PInvoke those methods directly—only exported free functions work, and you must handle `this` as an explicit pointer. Second, lifecycle mismatches. If your C++ object allocates memory using `new`, but your C# marshaling code calls `Marshal.FreeHGlobal`, you will corrupt the heap. JIT sometimes masks this with memory padding; AOT doesn't. Third, exception propagation. A C++ exception thrown across PInvoke boundary under AOT is not caught—it becomes a process kill. I have seen teams waste an entire sprint debugging a crash that was simply a constructor throwing `std::bad_alloc`.

Most devs skip testing interop under AOT until staging. That's the mistake. The seam between managed and unmanaged code was always fragile—AOT just stops hiding the cracks. What usually breaks first is the thing you assumed worked because it always had. Wrong. Not anymore. You'll need to validate every struct, every calling convention, every memory ownership rule at compile time. The JIT era let us be lazy. Native AOT demands precision—or it lets you fail loudly in production.

Core Idea: What's Different Under AOT

No JIT to Patch Calling Conventions

When you compile with Native AOT, the safety net disappears. Under the old JIT model, the runtime could inspect your PInvoke declarations at the last moment—rewriting call sites, padding arguments, even inserting marshaling stubs on the fly. That flexibility masked a thousand small sins. I have personally debugged a case where a developer declared a C++ method as stdcall when the library actually exported cdecl, and the JIT silently fixed it by generating an x86 thunk. With Native AOT, there is no thunk. No runtime patch. The calling convention baked into the binary is the one you declared, and if it's wrong—application crash, no warning, no second chance. The catch is that most PInvoke signatures you see online were written for the JIT era; they rely on runtime forgiveness that AOT strips away.

Static Linking vs Dynamic Resolution

The second shift hits harder: Native AOT links statically by default wherever possible, but PInvoke to C++ objects is inherently dynamic. You're not linking a .lib—you're loading a .so or .dll at runtime and hoping the exported symbols match exactly what the AOT compiler baked into your binary's import table. I have seen teams spend two days debugging a segfault only to discover the C++ object layout shifted by two bytes after a compiler flag change. That hurts. Static linking would have caught it at compile time; dynamic resolution punts the error to load time or worse—first method call on the object. What usually breaks first is not the function pointer lookup but the offset calculation inside the object's vtable or data layout. AOT has no JIT to reinterpret those offsets; they're frozen like concrete.

Object Layout Guarantees (or Lack Thereof)

Here is the dirty secret: C++ compilers have never guaranteed binary-compatible object layouts across minor version bumps. Not for std::string, not for your own POD structs if alignment flags change. Under JIT, .NET could rehydrate the managed view of a C++ object on every call—checking sizes, re-marshaling fields. Under AOT, your struct representation is carved into the executable. If the C++ side reorders members, the AOT binary reads garbage. Most teams skip this step: they assume [StructLayout(LayoutKind.Sequential)] matches the C++ #pragma pack(8) layout. It rarely does once optimizations like -O2 -fstrict-aliasing kick in. The symptom is subtle—a string field returns gibberish, an integer reads as the wrong byte order—and the root cause is always the same: the layout you thought was fixed never was.

'We spent a week bisecting commits, convinced the bug was in our marshaling logic. It was a single -fvisibility=hidden flag on the C++ side that shifted the vtable index by one slot.'

— Senior engineer debugging a Native AOT interop crash, private correspondence, 2024

Reality check: name the development owner or stop.

The trade-off is stark: you gain cold-start speed and binary size, but every PInvoke boundary becomes a brittle seam. That sounds fine until you ship to production and discover your C++ dependency updated its build toolchain. The JIT could have absorbed that shock; AOT transmits it directly to the runtime. What usually breaks first is not the obvious stuff—it's the thing you tested locally with Debug builds and the same compiler flags as your CI, but then production uses a different standard library version. Wrong order. Not yet detected. That's the real cost of the AOT interop promise: you own every bit of the contract now, and the compiler won't help you enforce it.

How It Works Under the Hood

Vtable Generation for C++ Objects

The first fracture line hides in the vtable. When you compile a C++ class in a normal managed environment, the runtime can inspect and adjust virtual method tables — it's flexible, almost lazy. Native AOT kills that flexibility. The compiler stamps every vtable at compile time, frozen, immovable. So if your PInvoke signature expects a method at slot 4 but the AOT compiler packed the vtable differently — say, because a base class changed or an optimizer reordered entries — you're calling garbage. Not a crash, not immediately — silent corruption of your object's state.

Most teams skip this: they assume the C++ ABI is stable across toolchains. It's not. Native AOT doesn't use the same layout heuristics as the full runtime. I've seen a project where adding a single virtual destructor upstream renumbered every method slot. The managed side kept calling the old offsets. Debugging that consumed two days. The catch is that you can't rely on the vtable order matching what you tested in debug mode — release builds under AOT reorder aggressively.

'The vtable is a contract written in invisible ink — AOT reads it in a different light.'

— senior engineer, postmortem on a three-day outage

PInvoke Marshaling in AOT Mode

The marshaling stub — that thin translator between managed and native calling conventions — behaves differently under AOT. In the traditional runtime, the JIT generates these stubs on the fly, inspecting types and inserting conversions. Native AOT pre-compiles everything, and that pre-compilation has blind spots. For example, if your C++ method returns a complex struct by value, the AOT linker may not emit the correct copy constructor or may skip padding alignment entirely. The result: you get a struct that's shifted by four bytes on the stack. Wrong order. Fields land in each other's slots.

What usually breaks first is string marshaling. Passing a std::string across the boundary? The AOT binary often assumes a different internal layout — small string optimization buffer sizes vary, allocators differ. I've watched a perfectly valid PInvoke call return corrupted text because the AOT-allocated buffer was 16 bytes and the C++ side wrote 24. The marshaling stub didn't catch it; it just handed the pointer over. That hurts. You can mitigate this by forcing blittable types or custom marshaling, but that kills the convenience of PInvoke entirely.

Memory Allocation and GC Interaction

Here's the sneakiest pitfall: ownership boundaries. When a C++ object allocates memory internally and hands you a pointer, who frees it? In a normal runtime, the GC can pin the buffer and coordinate — awkward but workable. Under Native AOT, the garbage collector is trimmed. No finalizer thread for native handles, no SafeHandle guarantees unless you explicitly register them. The AOT compiler strips out code paths it deems unreachable, and your custom cleanup logic might be one of them.

The tricky bit is that the allocation likely came from a different heap entirely — the C++ side's new versus the runtime's managed allocator. AOT doesn't unify these. So when your managed code finishes and the GC runs, it sees no managed references to the memory and skips it. Memory leak? Yes. Worse — if another thread writes to that freed region later, you get heap corruption. Not yet a crash, but a ticking bomb. The only reliable fix is to expose explicit Create and Destroy functions from the C++ side, called via PInvoke, and never let the GC touch those pointers. It's tedious, but it works.

Walkthrough: A C++ Class Exposed via PInvoke

Setting up the C++ side

Let's build the exact scenario that makes people swear at their monitors. I have a small C++ class—call it FrameBuffer—that holds a raw pointer to heap-allocated pixel data. The constructor calls malloc, the destructor calls free, and there is a method int* Data() that returns that pointer. Exported via extern "C" wrappers: FrameBuffer_Create, FrameBuffer_Destroy, FrameBuffer_GetData. Standard PInvoke fodder. On the C# side you declare [DllImport] for each, hand back an IntPtr, and call it done. Wrong order. The catch is obvious only when you think about who owns the memory—Native AOT compiles your C# into a single native image, no JIT to patch calling conventions later.

PInvoke declarations in C#

The typical declaration looks like this:

Odd bit about development: the dull step fails first.

[DllImport("native.dll", CallingConvention = CallingConvention.Cdecl)] static extern IntPtr FrameBuffer_Create(int width, int height); [DllImport("native.dll", CallingConvention = CallingConvention.Cdecl)] static extern void FrameBuffer_Destroy(IntPtr handle); [DllImport("native.dll", CallingConvention = CallingConvention.Cdecl)] static extern IntPtr FrameBuffer_GetData(IntPtr handle);

Most teams skip this: they forget to check whether the CallingConvention actually matches. Under the JIT that mismatch often works—it's sloppy but the runtime sometimes swallows the stack misalignment. Under Native AOT the seam blows out immediately. But that's not the first break. The first break is subtler.

First run: what breaks and why

I call FrameBuffer_Create, get back an IntPtr, store it. Then FrameBuffer_GetData returns another pointer. I copy the pixel data, call FrameBuffer_Destroy. That usually works. The problem is when FrameBuffer_Destroy runs a second time—double-free crash, but only on the AOT build. Under the JIT the GC heap layout happened to keep the IntPtr alive long enough; under Native AOT the garbage collector collects the SafeHandle wrapper early, triggers a finalizer that calls FrameBuffer_Destroy again, and your console logs a heap corruption. The real fail pattern: you lose a whole day debugging because the finalizer thread races your explicit destroy call. That hurts. The fix is a SafeHandle subclass that disposes exactly once, but I have seen teams patch it by adding a bool flag on the C++ side—a terrible hack that works only if you control the native code.

Under Native AOT, the GC collects earlier and more aggressively. Your PInvoke object lives on borrowed time without a proper SafeHandle.

— real issue filed on a GitHub repo I maintain

What usually breaks first is not the marshalling—it's the lifetime contract. Native AOT erases the JIT's lazy cleanup cushion. You write a destructor on the C++ side, you forget to suppress the finalizer on the C# side, and the binary eats itself. One rhetorical question: has your team audited every IntPtr return value for finalizer safety? No? Then you're one build flag away from a production crash nobody can repro locally.

Edge Cases and Exceptions

Virtual methods and diamond inheritance

Most teams skip this: you export a C++ class with a virtual method, PInvoke it from Native AOT, and it works fine during testing. Until you add a second derived class. Or worse—diamond inheritance. The vtable layout your C++ compiler chose at compile time is baked into the binary. Native AOT doesn't re-derive that layout; it trusts whatever pointer the C++ side hands back. So when you call a virtual method from managed code, you're jumping to an offset that assumes a specific inheritance chain. Wrong order? You'll land in the middle of a different function. I have seen this produce silent data corruption for three weeks before someone spotted the pattern: objects from one derived type called methods from another. The crash, if it happens at all, arrives as an access violation inside calli—no stack trace pointing to the real cause.

The catch is that AOT's type system has zero awareness of C++ vtables. It sees only an IntPtr and a method pointer. Diamond inheritance compounds this: the repeated base class subobject forces two vtable pointers inside one object. PInvoke marshalling maps one pointer to one view. Which vtable does it pick? Undefined. Honestly—the only safe path is a flat C-style interface layer that manually dispatches to virtual methods. That hurts your elegance, but a vtable fault during production doesn't care about elegance.

Stateful static members

Static data members in a C++ class look harmless—they're just globals with class scoping, right? Under Native AOT, that assumption blows up. The AOT compiler may see static int Foo::counter as a symbol it can internalize, or it might redirect the reference through a different module's data segment depending on how the linker resolves it. If your C++ object's static member is defined in a DLL and you call it from an AOT-compiled executable, you get one instance. If the same header is included in two separately compiled translation units without careful __declspec decoration, you get two instances. That isn't a race condition; it's a full duplication of state. We fixed this once by adding a shared atomic counter in a separate C-exported function—but only after a customer's licensing check randomly failed because one side incremented a local copy while the other side read the original. Cross-module static initialization order fiasco, just under a different runtime.

The tricky bit is that Native AOT blurs the line between your managed heap and the C++ runtime's .data section. A static member initialized during DllMain might not be ready when the AOT code's class constructor runs. The result: zero values that aren't zero, or stale pointers. Most teams skip this until they see "impossible" values in a diagnostics log. By then, the pattern is buried under a month's worth of commits.

'We spent two days debugging a static cache that returned different results from managed and native call paths. It was the same variable. It just existed twice.'

— engineer on a mixed C++/Native AOT telemetry pipeline

Cross-module allocation: new on one side, delete on the other

Here's a classic: your C++ factory allocates an object with new, hands it to managed code via an IntPtr, and managed code later calls a C++ destructor exported as a regular function. That works—if both sides link the same CRT. Native AOT doesn't use the CRT the way a classic .NET runtime does. It bundles its own memory allocator, and if your C++ DLL was compiled against a different CRT version or static CRT, the heap regions don't overlap. Calling delete on a pointer from a foreign heap is undefined behavior. Not a crash every time, either—sometimes it silently corrupts an adjacent allocation and you discover it weeks later when a completely unrelated feature breaks. I have personally debugged a case where a std::string destructor tried to free internal buffer memory into the wrong heap manager, and the process only crashed when the GC tried to compact a completely different region. Wrong heap. Wrong lifetime. Wrong everything.

Not every development checklist earns its ink.

The editorial takeaway: never let managed code own the lifetime of a C++ object directly. Provide a Release function exported from the same module that created the object, and call only that. Or better, use a shared allocator—but that ties your build system in knots when you update compiler toolchains. The less glamorous truth is that Native AOT + C++ interop works reliably only when you treat every C++ object as a resource that its own module manages end-to-end. Anything else rolls dice with your production data.

Limits of the Approach

When rewriting in C# is actually safer

Let me save you some pain I've absorbed firsthand: sometimes the right move is to burn the PInvoke bridge entirely. A C++ class that owns internal `std::vector` or `std::map` objects and exposes them via raw pointers? That's a ticking bomb under Native AOT. The GC can't track native heap allocations, so you end up writing manual cleanup routines that look like 1990s C code — and one missed `delete` cascades into a silent heap corruption that surfaces three sprints later. I've seen teams spend two weeks chasing an access violation only to discover they were double-freeing a destructor that the C++ side had already called. Rewriting that 300-line C++ class in C# took one developer five days. Zero marshaling bugs. Zero crash dumps at 2 AM. The trade-off is real: you lose any existing C++ library investment, but you gain a memory model the runtime actually understands. Ask yourself honestly — is that legacy C++ class doing something genuinely uncopyable, or is it just two map lookups and a loop? If it's the latter, port it. Your future self on-call will thank you.

Performance trade-offs of extra marshaling

Everyone assumes PInvoke is free because the syntax looks simple. It's not. Under Native AOT, every call across the boundary carries a covert tax: the runtime must transition from managed to unmanaged code, pin any `string` or `byte[]` parameters, and — worst case — marshal complex structs by value. The cost per call can be 50–200 nanoseconds if you pass integers and blittable structs. Pass a `StringBuilder` or a multidimensional array, and that number jumps to the microsecond range. Not catastrophic for occasional calls, but inside a render loop or a packet processor? You'll watch frame times spike. The painful part is that these costs are invisible under standard JIT — the profiler smooths them over. Under Native AOT, the seams show. We fixed this once by batching fifty small PInvoke calls into one bulk call passing a flat buffer. Performance recovered, but the code got uglier. That's the bargain: you trade readability for latency. If your hot path crosses the boundary more than 10,000 times per second, consider moving that whole loop into native code — or rewriting it in C#. There's no free lunch, only different price tags.

“You can't profiler-steer your way out of a marshaling bottleneck that you refuse to measure.”

— overheard at a .NET internals meetup, painful accuracy included

Tools and diagnostics you can't use anymore

This one hurts the most. Standard .NET developers reach for `dotnet-trace`, EventPipe, or the Visual Studio diagnostic tools to catch memory leaks or thread hangs. Under Native AOT with custom PInvoke, several of those instruments go silent — or worse, misleading. The GC heap no longer aligns with what SOS expects. Native crash dumps from the unmanaged side won't include managed callstacks past the transition point. I once spent a full afternoon debugging a segfault in a C++ destructor only to realize the AOT compiler had inlined a GC-collect at the return site, and the native frame pointer was garbage. Tools that assume a JIT compiler's metadata layout simply fail. Your fallback? Add careful logging around every PInvoke entry and exit, wrap native calls in try-catch with structured exception handling translation, and keep a small WinDbg cheat sheet taped to your monitor. The diagnostics you lose are exactly the ones that save you time when the code is on fire. That's not a reason to abandon PInvoke forever, but it's a reason to keep the surface area small — and to be honest about how much debugging pain you're willing to absorb.

Before you ship, test your crash dumps under AOT. If the callstack stops at `[NativeCode]` and you have no breadcrumbs, you've already lost. Add those breadcrumbs now.

Reader FAQ

Can I use C++/CLI instead?

You could—but you probably shouldn't. C++/CLI works great for full .NET Framework scenarios where the runtime manages the transition. Under Native AOT, there's no JIT and no managed runtime to host the mixed-mode assembly. That means your C++/CLI wrapper becomes a dead weight: it compiles but either crashes at load or silently corrupts the GC heap. I have seen teams waste a week on this before admitting they needed a flat C export. The trade-off is real—C++/CLI gives you convenience when both sides live in a managed world; AOT yanks that bridge out. You're better off writing an explicit C API around your C++ object and pinvoking that.

How do I debug PInvoke calls in AOT?

Debugging these calls in AOT is a different beast than normal interop debugging. The stack frames get trimmed, etw events for PInvoke don't fire the same way, and runtime callable wrappers are compiled static—no stepping through the marshalling glue. What usually breaks first is the calling convention mismatch. Wrong order. cdecl where you wrote stdcall. That hurts. We fixed this by setting 'DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1' and using 'DOTNET_JitDisasm=*' on the host binary to dump the actual assembly prolog. The trick is to compile with debug symbols, then load the native image into WinDbg or lldb—check the `[NativeCallable]` attribute generated the correct frame. Most teams skip this step until they hit a cryptic access violation at midnight. Don't.

What about exception handling across the boundary?

Short answer: you can't throw a C++ exception and catch it as a managed exception. The AOT runtime doesn't register a managed exception handler that unwinds through native frames—there's no managed unwinder for that path. If your C++ object throws std::runtime_error, it either terminates the process or silently corrupts the call stack, depending on the platform. The right approach is to catch all exceptions inside your native function, convert them to HRESULTs or error codes, and return them. I once saw a production service recycle every three hours because a PInvoke call into a C++ library left a thrown exception unhandled—the seam blew out.

“The interop boundary is the most brittle seam in any AOT binary—handle it like a live wire, not a function call.”

— comment from a senior infra engineer during a postmortem

For structured exceptions (SEH) on Windows, you can wrap the native call in [HandleProcessCorruptedStateExceptions]—but that's a last resort. Prefer explicit error propagation.

Should I switch to COM interop?

COM interop might sound like the clean answer—interface contracts, reference counting, language-neutral. But under Native AOT, COM support is partial. The built-in COM callable wrappers rely on runtime reflection and v-table generation that AOT strips away. You can author COM objects manually using [ComVisible] and IUnknown interfaces, but the marshalling layer for things like BSTR or SAFEARRAY is not AOT-optimized—you'll pay a perf penalty or hit missing type converters. The catch is that for simple interfaces (no aggregates, no variant arguments), COM works. For complex C++ objects with inheritance trees and templates? Stick to the flat C export. It's more boilerplate, yes—but it won't surprise you at 3 AM when the paging spikes.