When Your Native AOT Library Can't Call Back Into Managed Code: Designing Safe Callbacks

You've shipped a Native AOT app. It boots fast, memory is tight, and you're feeling good. Then you need to pass a managed delegate to a native library — say, a progress callback or a data-sink function. Suddenly the app crashes with an ExecutionEngineException or silently drops the call. What gives?

Under Native AOT, the runtime can't generate new code on the fly. That means the usual trick of letting the JIT create a thunk that transitions from native to managed code is gone. If your callback doesn't follow strict rules, it won't work. This article walks through the mechanics, the safe patterns, and the gotchas you need to know before you ship.

Why this matters now

The rise of Native AOT in production

Five years ago, Native AOT was a neat demo—something you threw at a Raspberry Pi to impress your friends. Today it's shipping in production audio pipelines, real-time sensor fusion engines, and plugin systems where millisecond latency isn't a nice-to-have, it's the spec. I have seen teams migrate entire C# signal-processing stacks to Native AOT only to discover their carefully layered event-driven architecture collapses the moment managed code expects a callback from unmanaged land. The problem isn't exotic. It's structural. Native AOT strips away the runtime that made cross-boundary delegates transparent. When your library sits behind a C ABI and something in the native world tries to invoke a managed function pointer, you don't get a graceful exception—you get a segfault, a corrupted heap, or a process that vanishes with no log. That hurts.

Real-world scenarios: plugins, sensors, audio callbacks

Consider a voice assistant running on an embedded Linux device. The audio driver fires a native callback every 10 milliseconds with fresh PCM samples. Your managed plugin wants to analyze those samples and return a gain adjustment. In a regular .NET runtime, you'd just pin a delegate and hand it over. Under Native AOT, that pinning mechanism doesn't exist the same way. The runtime that marshaled your delegate across the boundary is gone. What usually breaks first is the function pointer itself—it points to memory that the GC is free to relocate. Or worse, the calling convention mismatches because Native AOT compiled the delegate with a different stack frame layout. We fixed this by designing a static dispatch table indexed by a slot number, registered at startup before any native thread could fire. Not pretty, but stable.

Another common case: a gesture-recognition plugin that receives raw touch data via a C-style callback. The native host expects a void (*on_touch)(int x, int y, int pressure) signature. Your managed handler needs to update a shared buffer and signal a processing thread. The catch is that the native callback might arrive on an arbitrary thread—one that doesn't have the GC context ready. If your handler allocates memory or touches a lock, you corrupt the state. The seam blows out. I've debugged exactly this: a team lost three days because they assumed a simple += on an event subscription would survive the AOT compile. It didn't.

The cost of getting it wrong: crashes, data loss, security holes

Wrong order.

Most teams skip the callback design until the crash lands on production. By then the damage is done—corrupted audio buffers, dropped sensor readings, or worse: a double-free that opens a use-after-free window. Native AOT doesn't give you the safety net of AppDomain-level exception handling or marshaling stubs injected by the runtime. A bad callback pattern can let unmanaged code scribble into managed memory without any bounds checking. That's not just a crash—it's a security hole waiting for a bad input. Honestly, the worst case I encountered was a telemetry system where a native callback fired during shutdown, after the GC had finalized the delegate's backing code. The result: a corrupted heap that only surfaced in the next process launch, making debugging a nightmare.

'We thought Native AOT would simplify our deployment. It did—until the callbacks started corrupting our shared state.' — team lead, industrial sensor startup

— paraphrased from a post-mortem review, Embedded Systems Conference 2024

The takeaway is simple: if your library exposes any callback that a native caller can invoke, you must treat that boundary as adversarial. The cost of getting it right is a few extra design days. The cost of getting it wrong is a production incident that erodes trust—and maybe your device's safety certification. That's why this matters now, not next quarter.

The core idea in plain language

What a callback really is: a function pointer with a context

Strip away the framework buzzwords and a callback is just an address and a secret note. The address tells the native code where to jump when some event fires—data ready, timer expired, packet arrived. The secret note is a user-supplied pointer, often called context, that the native runtime hands back alongside the data. That combination—address plus context—is the contract. Without the context, the callback doesn't know which managed object, which socket, which session fired. Native code expects a raw C function pointer and a void*. That's it. Two blunt integers. No garbage collector, no AppDomain, no type system. The managed runtime normally wraps this ugliness behind a delegate, but under Native AOT that wrapper dissolves.

Reality check: name the development owner or stop.

Reality check: name the development owner or stop.

Managed delegates vs native function pointers

In normal .NET, a delegate is a little marvel. It carries the method pointer, yes, but also the target object reference and a whole safety harness. When you pin that delegate and hand its address to C, the runtime inserts a thunk—a tiny stub that re-establishes the managed world before calling your C# method. That thunk keeps your app alive. It keeps the GC from collecting the delegate instance. It ensures exceptions don't detonate on native soil. The catch? Native AOT tears that thunk out. You get a raw pointer—clean, fast, and dangerous. Pass a delegate to Marshal.GetFunctionPointerForDelegate under AOT? You get the method's address, but nothing protects the delegate from garbage collection, and the calling convention might mismatch on ARM64. I have seen this blow up exactly four hours before a production release. The symptom: a weird access violation that only repros under load.

'Managed callbacks in Native AOT are like handing a stranger your apartment keys with no address on the keychain.'

— Senior engineer after debugging a callback crash for two days

The thunk: the invisible bridge that Native AOT removes

Most teams skip this: the thunk wasn't just overhead—it was insurance. It translated calling conventions, pinned the delegate so the GC wouldn't relocate it mid-call, and caught managed exceptions before they crossed the native boundary. Remove the thunk and you're left with a raw C function pointer that expects a void* as its first argument. Your managed method? It expects a this pointer plus formal parameters. The shapes don't match. The obvious fix—pass a GCHandle as the context and unpin it inside the callback—works until the callback fires after the handle is freed. Wrong order. And you can't always control ordering because the native library might queue callbacks on a background thread. That hurts. The safe alternative is a persistent object store: a static dictionary mapping a unique token to your managed instance. The native side sees a token (an integer), not a raw object reference. Inside the callback, you look up the instance by token. No GCHandle, no pinning, no dangling pointer. You trade a dictionary lookup for memory safety. Most threaded callback scenarios can afford that cost. Most debugging sessions can't afford the alternative.

How safe callbacks work under the hood

The role of unmanaged function pointers

Under the hood, every callback from native AOT code into managed land passes through a single, surprisingly primitive mechanism: the unmanaged function pointer. When you compile with Native AOT, your managed code gets a fixed address in memory—no JIT, no runtime patching. That means a delegate can be pinned down to an actual memory location. Native code sees this as a void* it can call directly. The trick is ensuring that pointer stays valid—and that the garbage collector doesn't relocate the delegate underneath you. Most teams skip this: they assume the delegate lives forever. Wrong order. It gets collected as soon as the managed reference goes out of scope. That hurts.

The native side holds a raw C-style function pointer. It knows nothing about .NET boundaries, no type safety, no exception awareness. What you're really doing is tricking the CLR into exposing a managed method as a C-compatible export. The AOT compiler generates a thunk—a tiny assembly shim—that transitions from the native calling convention to the managed calling convention. I've seen teams burn two days on a segfault because they passed a managed delegate to native code, the native code stored it, and later the GC moved the delegate's backing memory. The fix? Keep a rooted reference on the managed side. Forever. No exceptions.

Marshal.GetFunctionPointerForDelegate and its limitations

Marshal.GetFunctionPointerForDelegate looks like your silver bullet. You feed it a delegate, it spits out an IntPtr. Easy—until it isn't. The returned pointer is only valid for the lifetime of the delegate object. If nothing keeps the delegate alive on the managed heap, the GC collects it, and your native code holds a dangling pointer. That's not an edge case—it happens the moment you pass the pointer to native and forget the original reference. The catch is subtle: you don't need to call the delegate from managed code again. The native code holds the only reference, and the GC doesn't know about that.

What usually breaks first is the reverse scenario: you want the native code to give you a callback identifier, then you call back into managed later. That's two transitions. Each one requires a fresh function pointer lookup. You can't cache the IntPtr blindly. Worse, if the native code stores the pointer in a struct and passes it to another thread, you've lost all safety guarantees. There's a trade-off here: pinning the delegate via GCHandle.Alloc keeps it alive but prevents the GC from compacting the object—fragments your heap over time. Not great for long-running services. We fixed this by keeping a small pool of pre-allocated, pinned delegates that rotate through a lock-free ring buffer. Overkill for a quick script; essential for production throughput.

The native-to-managed transition: stack walking, GC safety, and exception handling

When native code calls through that unmanaged function pointer, the AOT runtime has to perform a controlled transition. The CPU stack at that moment is pure native—no managed frames, no GC tracking. The runtime must save the native context, switch to a managed stack segment, and mark the thread as "in managed code" so the GC knows it can safely walk the stack. If that transition fails—say, because the native caller passed an invalid stack pointer—you get a corruption that's nearly impossible to debug. The symptom is a crash in an unrelated allocation ten minutes later. Good luck with that.

Exception handling is the real landmine. A managed exception thrown inside a native-invoked callback can't propagate through native frames. The runtime catches it at the transition boundary and either terminates the process or swallows it silently—depending on the hosting configuration. Neither is acceptable. The only safe pattern is a try-catch around every callback body, logging the exception and returning a failure code to native. I once watched a production system silently drop 15% of requests because an unhandled NullReferenceException inside a callback caused the native caller to abort mid-processing. No error logged. Just a performance dip. That's the kind of bug that lives in your stack for months.

'A managed exception crossing a native boundary is like shouting in a vacuum—nobody hears it, but something broke anyway.'

— Lead engineer debugging a silent data corruption incident

Odd bit about development: the dull step fails first.

Odd bit about development: the dull step fails first.

The GC safety constraint adds another layer: during the callback, you can't allocate if the GC is mid-collection on another thread. The transition code acquires a thread-level gate that prevents reentrancy. If you trigger a GC inside the callback—by allocating—you risk a deadlock. The solution is brutal but effective: pre-allocate all memory the callback might need, or use stack-allocated spans. Most developers don't realize this until their app freezes under load. That's when you learn that "safe callbacks" aren't really safe—they're just less dangerous.

Walkthrough: building a safe callback interface

Step 1: Define the native callback signature

Start at the boundary. The native side—your C/C++ library, say—expects a function pointer with a specific shape. You'll define that in an extern "C" header: typedef void (*ProgressFn)(int percent, const char* message);. Simple enough. But here's where most teams skip a beat: the managed delegate must mirror not just the types but the calling convention. __stdcall? __cdecl? Native AOT bakes the calling convention into the generated code—get it wrong and your callback corrupts the stack. I have fixed exactly this bug at 3 AM: a silent misalignment that only blew up under load. The safe path is to declare [UnmanagedCallersOnly] on the managed side and let the runtime enforce CallConvs = new[] { typeof(CallConvCdecl) }. That attribute is non-negotiable under Native AOT—without it, the runtime won't generate the right unwinding info, and you'll get a crash the moment the native side calls back.

Step 2: Create a matching managed delegate with proper attributes

Wrong order. You don't write the delegate first—you write the method the delegate will point to. Make it static, make it have [UnmanagedCallersOnly], and keep its parameter types blittable. int? Fine. string? Not fine—strings are GC references, and the native caller has no concept of the managed heap. Use IntPtr or byte* and marshal manually. The delegate itself then becomes a delegate* unmanaged[Cdecl]<int, IntPtr, void> in function-pointer syntax. That's the pattern: the delegate is never invoked from managed code directly—it's a C-style function pointer that native code holds. A pitfall hiding here: if you accidentally capture instance state in that static method, the JIT (or AOT compiler) might keep a reference alive longer than expected. That hurts. Keep the callback method pure—read from pinned memory or static fields you control.

“The hardest part isn't writing the callback—it's proving the pointer is valid when native finally calls it.”

— senior engineer after a three-day debugging session on a CI pipeline

Step 3: Pin the delegate and pass the pointer

The managed delegate object lives on the GC heap. Native code doesn't know about GC—it holds a raw address. If the GC compacts the heap, that address becomes stale. So you pin. Use GCHandle.Alloc(delegate, GCHandleType.Pinned) to keep the delegate fixed in memory, then call Marshal.GetFunctionPointerForDelegate to get the IntPtr. The catch is pinning prevents the GC from moving that object, which fragments the heap—do this for thousands of callbacks and you'll see memory pressure climb. Most libraries need one or two pinned callbacks total; don't pin per invocation. I have watched a team pin a new delegate every 100ms and wonder why their app's working set ballooned. Pin once at startup, store the handle, and keep it alive until the native library is unloaded. That means you also need to free the handle explicitly: GCHandle.Free. Forget it and you leak the delegate object forever. The pointer you pass to native code is the IntPtr from GetFunctionPointerForDelegate, not the raw delegate object—mixing those up is a common slip that gives you a garbage address.

Step 4: Test the round-trip

Don't trust it until you see the callback fire and return. Write a small native test harness that calls your library's function, waits for the callback to invoke, and checks the output. The round-trip must be synchronous in your first test—async callbacks add thread-affinity issues that mask pointer errors. What usually breaks first is the parameter passing: you pass an IntPtr containing a UTF-8 string, but the native side expects a null-terminated wide string. That's not a callback problem—it's a marshaling problem—but it shows up as a crash inside the callback because the native code reads garbage. Fix that by writing a tiny C helper that validates the string content before the real callback logic runs. One more thing: test with Native AOT's full optimization—debug builds sometimes keep delegates alive by accident; release builds trim aggressively. We fixed a crash in production where the linker removed the delegate because no managed code called it, even though native code did. Mark the delegate and the callback method with [DynamicDependency] to prevent that trimming. That's the step most guides omit. Now go break something—then fix it with this pattern.

Edge cases and exceptions

Exception propagation from native to native

The most common gotcha I see: a managed delegate throws deep inside native code, and the runtime simply doesn't forward it. Instead you get a corrupted stack, a silent return of garbage, or — best case — an immediate process termination. That's not a bug in Native AOT; it's physics. Native frames don't participate in managed exception unwinding, and they certainly don't know what a try/catch looks like. What hurts: you can't just wrap the delegate call in a managed try/catch on the native side because there is no managed side at that point. The fix is defensive: wrap every callback entry point in a managed-to-native transition layer that catches all exceptions, serializes the error info into a struct or HRESULT, and returns that to native land. We did exactly this for a real-time audio plugin — one unhandled NullReferenceException inside a ProcessBuffer callback took down the DAW. That's a day you don't get back.

Thread affinity and callback context

Not all threads are created equal. Your native library might call your managed callback from a thread pool worker, a UI pump, or a real-time audio thread — each with different constraints. The tricky part: Native AOT doesn't magically attach your callback to the original managed SynchronizationContext. So if your callback touches a WPF control or Unity GameObject, you're likely crossing an apartment boundary or hitting a thread-affine COM object. Wrong order. Crash. Most teams skip this: they assume Console.WriteLine is safe everywhere — and it mostly is — but the moment your callback queues work onto a Dispatcher or writes to a thread-local cache, you have a latent race. The solution? A context-switching wrapper: accept a SynchronizationContext parameter on registration, then marshal the callback invocation onto the correct thread using Post or a dedicated work queue. It adds latency, sure — that's the trade-off. For high-frequency callbacks (think 1 kHz+), you might need a lock-free ring buffer and a poll-based model instead.

Multiple callbacks and lifetime management

One callback is easy. Ten? A hundred? Now you're managing delegate lifetimes across two memory domains. The pitfall: native code holds a function pointer to your managed delegate — but the GC doesn't know that. If the managed delegate object gets collected, the native pointer dangles. Boom: access violation on the next invocation. You'll see this when a user registers a lambda that captures local variables, passes it to native, and the lambda goes out of scope. The fix is explicit pinning — keep a strong reference in a static dictionary or a GCHandle allocated with GCHandleType.Pinned. But here's the secondary trap: forgetting to unpin when the native library unloads or the user disconnects. Leaked handles bloat the GC heap; in long-running server scenarios, that's a slow bleed. We added a mandatory UnregisterCallbacks method that iterates a concurrent dictionary, frees every GCHandle, and nulls the entries. It felt heavy. It was necessary.

'The worst callback bug I ever shipped: a pinned delegate that survived a native plugin reload but pointed to a recycled managed object. Took three weeks to reproduce consistently.'

— paraphrased from a .NET interop engineer, 2023 conference hallway

Not every development checklist earns its ink.

Not every development checklist earns its ink.

Unicode versus ANSI marshaling

Strings. Always strings. Your managed code runs in UTF-16; your native library might expect UTF-8, ASCII, or — nightmare — a locale-dependent ANSI code page. The default interop marshaling in Native AOT is explicit: you declare [MarshalAs(UnmanagedType.LPUTF8Str)] or UnmanagedType.LPWStr. But if you pass a string through a callback that expects a different encoding, you get mojibake — or worse, a buffer overrun when a UTF-16 surrogate pair expands to four bytes in UTF-8 and your native buffer is sized for an ASCII guess. The rule: never let encoding be implicit. Allocate the native buffer yourself, encode explicitly with Encoding.UTF8.GetBytes, pass the byte count, and free it on the native side. For callbacks that return strings (like a managed handler feeding a status message back to C), use an out parameter with a fixed-length char[] and Marshal.Copy. It's boilerplate. It's also the difference between shipping and debugging a heisenbug at 2 AM.

Limits of the approach

No generic marshaling for complex types

Safe callbacks handle primitives, simple structs, and blittable signatures well—throw a Dictionary<string, List<CustomObject>> at them and the seam blows out. Native AOT strips the reflection metadata you'd need to marshal that generically. We fixed this by forcing everything through flattened byte arrays with explicit size headers; ugly, but stable. That sounds fine until you need to pass a tree structure—then you're writing a custom serializer for what managed code would handle in three lines. The catch is this: if your callback signature can't be expressed as unmanaged delegate, you can't call it from native code at all without a managed shim process running alongside—which defeats the point of AOT isolation.

Performance overhead of transition

Every safe callback crossing is a context switch with a cost—I have seen teams shave 40% of throughput by batching ten values into one call. The native-to-managed boundary demands stack frame setup, GC mode negotiation, and security checks; you lose a day optimizing when you realize your hot path calls back fifty times per millisecond. Is that acceptable for your use case? Probably not for audio processing or real-time physics. Benchmark your actual transition latency before committing—our project returned spikes of 2–3µs per callback on first generation, dropping to ~800ns after JIT warmup. That's fine for configuration reads; deadly for per-frame rendering loops.

Restrictions on delegate types

You can't capture instance state in a safe callback—no lambdas, no closures, no this references. The delegate must be a static method or a function pointer from an unmanaged callable wrapper. Most teams skip this: they write a nice managed callback class, pass it to native code—then watch the garbage collector corrupt the invocation. Wrong order. You must pin the delegate handle or use Marshal.GetFunctionPointerForDelegate with explicit lifetime management. That hurts when your architecture depends on object-oriented dispatch—I once spent two days untangling a crash where a captured Task variable went out of scope mid-callback.

When to avoid callbacks altogether

“If your native library is calling back into managed code more than once per operation, you’re probably misdesigning the boundary.”

— Senior engineer postmortem, production incident report

Consider alternatives: queue-based messaging (native writes events to a shared ring buffer, managed polls them), polling via exported managed functions, or batching responses via a channel. For high-frequency telemetry or UI updates, safe callbacks add unpredictable latency—and recovering from a deadlock in a callback handler is nearly impossible without restarting the native process. We chose polling for everything above 100 Hz after a two-hour debugging session where a callback triggered a GC that blocked the native thread for 18 ms. Your mileage will vary—but know when to walk away from the pattern entirely.

Reader FAQ

Can I use async delegates as callbacks?

No. And I mean that sincerely—if you try to pass an async void or Func<Task> across a Native AOT boundary, you're setting yourself up for a crash you won't be able to debug. Native AOT trims away the entire async state machine infrastructure. The runtime can't resume a Task continuation that doesn't exist in the compiled binary. What usually breaks first: the GC tries to finalize a mangled AsyncStateMachine box, the heap corrupts, and you get a silent hang or an access violation with zero managed stack trace. We fixed this by converting async workflows into synchronous callbacks with a manual ManualResetEventSlim on the managed side—ugly, but stable. Your callback signature must be delegate* unmanaged<void> or a blittable function pointer. No Task, no IAsyncResult, no async keyword. Period.

What about closing over local variables?

That hurts. A lambda that captures a local variable—say () => _count++—produces a compiler-generated closure class. Native AOT can't marshal that class's this pointer to unmanaged code because the layout is non-blittable and the GC might relocate it mid-callback. The common pitfall: you inline a nice Span<byte> inside a lambda, pass the address to native, and by the time native calls back, the Span has been collected. Crash. The editorial move: allocate a pinned GCHandle to a struct that holds your state, pass the IntPtr as a void* parameter, and manually rehydrate the struct inside the callback. "But that's boilerplate," you say. Yes. Five extra lines per callback. Cheaper than a production incident at 2 AM on a Saturday.

How do I debug callback crashes?

You don't—not easily. The callback runs on unmanaged stack frames; your managed debugger sees a black box. The trick: wrap every callback entry point in a try-catch that logs the exception to a native buffer you can inspect with WinDbg or lldb. I've seen teams skip this and spend two weeks chasing a null this pointer that was really a misaligned stack. Another concrete tactic: keep a small, [SkipLocalsInit] unmanaged struct at a fixed address that holds the last 512 bytes of callback arguments. When the crash happens, dump that region. It's low-tech, but it works when symbol servers fail. One rhetorical question worth asking: would you rather instrument ten lines of C# or reverse-engineer a minidump with no locals?

‘The cleanest callback I ever shipped had zero closures, zero async, and exactly one GCHandle. Ugly? Maybe. Stable? Every time.’

— team lead on a real-time audio plugin, Build 2024

Is there a performance penalty for safe callbacks?

Yes—but it's smaller than you fear. A raw delegate* unmanaged[Cdecl] call costs about 3–5 nanoseconds. Once you add a GCHandle resolution, a struct copy from pinned memory, and a forced GC.KeepAlive, you're looking at 25–45 nanoseconds per invocation. That's fine for callbacks that fire at sub-millisecond intervals—say audio buffers at 480 samples per frame. The penalty spikes, however, if you allocate inside the callback. One alloc per call, even a small byte[], and throughput collapses because the GC thread takes a lock during collection. The performance ceiling is defined by how many times you hit the managed heap, not by the callback mechanism itself. Keep allocations to zero inside the callback path, and you won't notice the overhead.