Operating systems

DLL injection, and why antivirus cares

Loading code into a process you did not write is how profilers, screen readers and malware all work. The technique does not know which one you are.

Every process gets its own virtual address space. That isolation is the point: a pointer in one process means nothing in another, and the memory manager enforces it in hardware. "Injecting a DLL" means arranging for a second process to map your library into its address space and run your initialisation code there, after which your code is inside the isolation boundary rather than outside it.

Once inside, you share everything — the heap, the handle table, the thread pool, the security token. You are not talking to the process; you are part of it.

Scope: this is a description of how a class of operating-system mechanism works, aimed at people who ship instrumentation, debug crash dumps full of unfamiliar modules, or need to understand why their EDR agent is arguing with their profiler. It is not a guide to loading code into software you have no right to modify, which is at minimum a licence violation and in many jurisdictions an offence.

What "loading a library" already does

The mechanism is less exotic than it sounds, because the operating system already does exactly this on every process start. The loader maps the executable, walks its import table, maps each dependency, resolves each imported function to an address, and calls each library's entry point. A DLL's initialisation routine runs inside the process, with full access to it, as a matter of ordinary operation.

Injection is therefore not about inventing a capability. It is about triggering the loader from outside, for a library the target never asked for.

The mechanisms, conceptually

The remote thread. The textbook approach: allocate memory in the target, write the path of your library into it, then create a thread in the target whose start address is the loader's own library-loading function and whose argument is that path. The loader does the rest. It is elegant because it uses the target's own machinery — but it is also the most recognisable pattern in the field, because it requires a cross-process allocation, a cross-process write, and a cross-process thread creation in quick succession. Security products watch for exactly that sequence.

Hook installation. Windows lets you install a hook procedure that lives in a DLL and applies to threads other than your own; the operating system maps that DLL into every process with a matching thread as a supported part of the mechanism. This is not a trick — it is the documented way global hooks have always worked, and it is why accessibility software and input method editors have historically ended up in every process on the machine.

Loader-supported paths. Registry keys that name libraries to load into processes (largely disabled by default on modern Windows), shell extensions, COM in-process servers, and audio and print drivers all provide sanctioned ways to get a library loaded into someone else's process. These are extension points rather than attacks, and they are also the routes with the longest history of abuse, which is why several have been progressively locked down and now require signing.

Thread hijacking and APCs. Rather than creating a thread, suspend an existing one, change its saved instruction pointer to code you wrote, and resume it — or queue an asynchronous procedure call that runs the next time the thread enters an alertable wait. Both avoid the conspicuous thread-creation event, which is precisely why detection has moved to watching thread context modification too.

Elsewhere. The Unix equivalent of the easy case is LD_PRELOAD, which asks the dynamic linker to load a library ahead of the others and lets it interpose on any symbol; the hard case uses ptrace. On macOS DYLD_INSERT_LIBRARIES is the analogue, and is heavily restricted by the hardened runtime, System Integrity Protection and code-signing entitlements — Apple has effectively closed the general form, which is why so much macOS tooling needs explicit user permission and its own entitlement.

Hooking, which is what people actually want

Injection is rarely the goal. Being inside the process so you can intercept a function is the goal.

Inline hooking overwrites the first few bytes of the target function with a jump to your replacement. Because you have destroyed real instructions, you must first copy them somewhere and append a jump back, so the original behaviour remains callable — which means you need a length-disassembler to know where the instruction boundaries are, since x86 instructions vary in length and you cannot cut one in half. Microsoft's Detours is the canonical documented implementation of this and is worth reading for the edge cases alone.

Import table hooking is gentler: rewrite the entry in the target module's import address table so calls to an imported function go to you instead. Nothing is patched, but it only catches calls made through the import table, so anything resolving the function dynamically slips past.

Both are fragile in the same specific way: when two products hook the same function, the outcome depends on the order they arrived. An anti-cheat, an EDR agent and a graphics overlay all hooking the same system call is a common configuration, and the resulting crashes are famously difficult to attribute because each vendor's stack looks correct in isolation.

Why the CLR complicates all of this

A managed assembly is not a library you can simply load. Several things break at once.

There is no native entry point to call. A .NET assembly is a portable executable containing intermediate language, not machine code. Handing its path to the loader does not produce running code; it produces a module the loader cannot execute. Something must first start a runtime.

So you need a host. The supported route is a small native shim that loads and initialises the CLR, then asks it to load your assembly and invoke a method with a specific signature. The API for this changed completely between .NET Framework and .NET Core: the framework used a metahost and runtime host interfaces with application domains, whereas modern .NET uses the nethost/hostfxr libraries and AssemblyLoadContext, and application domains no longer exist. Code written against the old interfaces does not port.

Native addresses are not stable. Managed methods are compiled just-in-time on first call, so there is no fixed address to hook before then. Worse, tiered compilation means a method that has been called enough times is recompiled at a different address with different optimisations, so a hook placed on the first version quietly stops being reached.

The garbage collector moves things. Managed objects are relocated during collection. A raw pointer to one is valid until the next GC and not after, so anything native holding a reference must pin it, and pinning fragments the heap.

The loader lock will deadlock you. A library's entry point runs while the loader holds a process-wide lock, and the set of things you may safely do there is very small — no allocation from another library, no waiting on another thread, and certainly nothing that starts a runtime. Doing real work in an injected library's entry point is the single most common cause of a target that hangs on start with no error.

For managed processes there is a better answer than any of this. The runtime exposes a profiling API designed for exactly this purpose: a native component registers as a profiler, and the runtime calls it on module loads, JIT compilation, exceptions and garbage collection, and permits IL rewriting at JIT time. Every commercial application performance monitoring agent — the ones that instrument your code without you changing it — works this way, because it is supported, survives runtime upgrades, and does not fight the JIT. If you are instrumenting managed code, this is the interface, and inline hooking is the thing you do when no interface exists.

The legitimate half of the ledger

The technique's reputation comes entirely from its other users, so it is worth listing what depends on it:

  • Debuggers and profilers, including every language runtime's own.
  • Application performance monitoring agents, which instrument production code with no source changes.
  • Screen readers and accessibility tools, which must observe and drive UI in processes they did not write.
  • Input method editors, which insert themselves into text input everywhere.
  • Graphics overlays — chat, streaming, screen capture, frame counters — which hook the presentation call to draw on top of a game's own output.
  • Test automation and UI robotics.
  • Compatibility shims, where the operating system itself injects fixes into old applications so they keep working across OS versions.
  • Antivirus and EDR products, which inject in order to observe.

That last one is the irony worth sitting with: the security software flagging injection as suspicious is itself injecting into everything on the machine. It is not hypocrisy so much as an accurate reflection of the fact that the mechanism is neutral.

What detection actually looks for

Because the technique cannot be distinguished from itself, detection is behavioural and contextual rather than signature-based:

  • cross-process memory allocation and write followed by remote execution, especially into a process with no relationship to the writer
  • memory regions that are both writable and executable, which normal code does not need
  • modules loaded from unusual paths, or unsigned modules resident in a signed process
  • modifications to another thread's saved context
  • hooks in the system call layer that were not there at process start
  • a mismatch between a module on disk and the copy mapped in memory

Which yields the practical guidance for anyone shipping legitimate instrumentation. Sign your binaries. Use the supported extension point if one exists — the profiling API, an accessibility API, a documented plugin interface — rather than the general mechanism. Do as little as possible in the entry point. Expect to be flagged anyway, and expect to spend real effort on vendor allow-listing, because from the outside your profiler and a credential stealer perform the same three system calls in the same order.

Seeing what is already inside your process

The diagnostic version of this is worth knowing even if you never inject anything. When a process crashes in a module you have never heard of, the loaded-module list is the first thing to read: lm in WinDbg, the module view in Process Explorer, or the module list in any crash dump. An unexpected DLL in your process is usually a security product, an overlay or a compatibility shim, and identifying it converts an inexplicable crash into a known interaction — which is very often the entire investigation.