Inside EDR Unhooking: How Security Researchers Map EDR Behavioral Patterns

Hook

At least 15 major endpoint security products that enterprises trust to protect their networks can be fingerprinted and analyzed through their API hooking patterns—and this repository shows you exactly how they do it.

Context

Modern Endpoint Detection and Response (EDR) systems stand between attackers and enterprise networks, monitoring billions of API calls to detect malicious behavior. Most EDRs operate by injecting themselves into every process, hooking critical Windows APIs in ntdll.dll to intercept system calls before they reach the kernel. When your program calls NtCreateFile or NtAllocateVirtualMemory, the EDR's hook executes first, logging parameters and making allow/block decisions.

For red teamers conducting authorized penetration tests and security researchers studying defensive technologies, understanding these hooks is essential. The Mr-Un1k0d3r/EDRs repository emerged from this need, providing both intelligence (which APIs each vendor hooks) and proof-of-concept techniques for detecting and bypassing userland hooks. It represents a community-driven effort to document the cat-and-mouse game between offensive security practitioners and EDR vendors, offering transparency into how billion-dollar security products actually work under the hood.

Technical Insight

The repository's core unhooking technique exploits a fundamental architectural constraint: userland hooks must modify code in the process's own memory space, which means they can be detected and overwritten. The approach involves mapping a clean copy of ntdll.dll directly from disk, extracting the original syscall stubs, and patching them over the hooked versions in memory.

The patch_syscall.c implementation demonstrates this elegantly. It starts by loading a fresh ntdll.dll copy using CreateFileMapping and MapViewOfFile, then parses the PE (Portable Executable) structure to locate the export address table. For each exported function, it calculates the relative virtual address (RVA) and reads the original bytes:

HANDLE hFile = CreateFileA("C:\\Windows\\System32\\ntdll.dll", 
    GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
HANDLE hMapping = CreateFileMappingA(hFile, NULL, PAGE_READONLY, 0, 0, NULL);
LPVOID pMapping = MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);

// Parse PE headers
PIMAGE_DOS_HEADER pDosHeader = (PIMAGE_DOS_HEADER)pMapping;
PIMAGE_NT_HEADERS pNtHeaders = (PIMAGE_NT_HEADERS)(
    (BYTE*)pMapping + pDosHeader->e_lfanew);
PIMAGE_EXPORT_DIRECTORY pExportDir = (PIMAGE_EXPORT_DIRECTORY)(
    (BYTE*)pMapping + pNtHeaders->OptionalHeader.DataDirectory[0].VirtualAddress);

// For each export, copy clean bytes to hooked ntdll
for (DWORD i = 0; i < pExportDir->NumberOfNames; i++) {
    DWORD nameRVA = ((DWORD*)((BYTE*)pMapping + pExportDir->AddressOfNames))[i];
    char* functionName = (char*)((BYTE*)pMapping + nameRVA);
    
    if (strncmp(functionName, "Nt", 2) == 0) {
        // Calculate address in both clean and hooked versions
        LPVOID cleanAddr = (BYTE*)pMapping + functionRVA;
        LPVOID hookedAddr = GetProcAddress(GetModuleHandleA("ntdll.dll"), functionName);
        
        // Overwrite hooked version with clean bytes
        DWORD oldProtect;
        VirtualProtect(hookedAddr, 32, PAGE_EXECUTE_READWRITE, &oldProtect);
        memcpy(hookedAddr, cleanAddr, 32);
        VirtualProtect(hookedAddr, 32, oldProtect, &oldProtect);
    }
}

This code circumvents userland hooks by restoring the original syscall stubs, effectively removing the EDR's visibility into those API calls. The technique works because Windows loads ntdll.dll from C:\Windows\System32 at process startup, and EDRs hook it afterward—so the disk version remains pristine.

The repository also includes hook_finder64.exe, which detects hooks by comparing the in-memory ntdll against the disk version. When it finds discrepancies in the first 32 bytes of any Nt* or Zw* function (where syscall stubs reside), it flags them as hooked. This produces the intelligence data documenting which APIs each EDR monitors—CrowdStrike hooks 330+ functions, SentinelOne focuses on 50+ high-value APIs, while Cortex XDR uses kernel callbacks instead of userland hooks entirely.

The unhookIAT.c variant takes a different approach by manipulating the Import Address Table (IAT) rather than patching syscall stubs directly. It replaces hooked function pointers in the IAT with addresses from a clean ntdll mapping, achieving similar results with less memory modification.

What makes this particularly clever is the dynamic resolution strategy. Rather than hardcoding syscall numbers (which change between Windows versions), the code reads them from the clean ntdll.dll's syscall stubs at runtime. A typical syscall stub looks like this in assembly: mov r10, rcx; mov eax, <syscall_number>; syscall; ret. By parsing these bytes, the tools work across Windows 7 through Windows 11 without modification.

The repository's real-world intelligence shows fascinating patterns. Microsoft Defender hooks broadly (200+ functions), likely because it's integrated into the OS and has performance headroom. Third-party EDRs like Carbon Black and CylancePROTECT are more selective, focusing on file operations, process creation, and memory manipulation APIs. Cortex XDR and modern CrowdStrike versions have abandoned userland hooking entirely, using kernel callbacks that these techniques cannot defeat.

Gotcha

The elephant in the room is kernel-mode detection. As the repository explicitly notes for Cortex XDR and CrowdStrike Falcon (newer versions), these techniques are useless against EDRs that operate primarily through kernel callbacks like PsSetCreateProcessNotifyRoutine, ObRegisterCallbacks, and minifilter drivers. When an EDR registers a kernel callback, it receives notifications directly from the Windows kernel before any userland code executes—no amount of ntdll.dll patching can bypass that.

The hook lists also suffer from staleness. EDR vendors continuously update their products, adding new hooks, removing ineffective ones, and shifting detection strategies. The repository relies on community contributions to stay current, but there's no automated testing infrastructure. A hook list from 2021 may not reflect how that same EDR behaves in 2024. Additionally, these are proof-of-concept tools meant for learning and research, not production red team operations. They lack error handling, obfuscation, and the operational security features needed for real engagements. The code is intentionally simple and readable for educational purposes, which means it's also easily signatured by the very EDRs it targets.

Verdict

Use if: You're a security researcher studying EDR internals, a red teamer learning evasion fundamentals for authorized penetration testing, or a defensive engineer trying to understand what attackers see when analyzing your security stack. The intelligence data alone is worth the star for planning detection strategies. Skip if: You need production-ready offensive tooling (look at SysWhispers2 or commercial frameworks instead), you're facing modern EDRs that use kernel-mode detection primarily, or you want comprehensive bypass solutions beyond API unhooking. This repository excels as educational material and reconnaissance intelligence, but it's deliberately positioned as research code rather than operational malware. The value is in understanding the techniques and patterns, not copying the code into your implant.