kevoreilly/capemon/capemon-developer

Capemon Skills

capemon is a monitoring and instrumentation engine designed for malware analysis, configuration extraction, and payload recovery. It acts as the core injection component for the CAPEv2 sandbox.

Core Capabilities

1. API Hooking & Monitoring

capemon implements an extensive hooking engine derived from cuckoomon-modified, providing deep visibility into application behavior across multiple subsystems:

• Process & Thread Management: Monitoring creation, termination, and manipulation of processes and threads.
• File System Operations: Tracking file creation, deletion, reading, and writing.
• Registry Activity: Capturing configuration changes and persistence mechanisms.
• Network Communication: Intercepting socket operations, DNS queries, and high-level protocol activity (HTTP, etc.).
• Cryptography: Extracting keys and monitoring encryption/decryption routines.
• Synchronization & Services: Monitoring mutexes, events, and Windows Service interactions.
• Windows Management Instrumentation (WMI): Intercepting WMI queries used for anti-analysis or reconnaissance.
• Scripting Engines: Specific hooks for VBScript and other language runtimes.

2. Debugging & Tracing

capemon implements an in-process debugger independent of Windows debugging interfaces, but harnessing the capabilities of the processor:

• Hardware breakpoints: Four breakpoints bp0-bp3 that can be set on execute, read or write
• Software breakpoints: Unlimited INT3 or 'CC' breakpoints overwriting instruction byte
• Single-step: Tracing allows instruction-level capture enhanced with configurable step-over, trace-length, register changes, function names, strings & more
• Actions: Configurable actions allow control flow manipulation with skipped or taken jumps, arbitrary register changes or jumps, string capture, dumps, scans & more
• Programmable: Debugger configurable either on submission with simple text options or via dynamic YARA signature scans during unpacking or detonation
• Integration: Hooking engine integrated with optional behavior log output & breakpoints set on return from hooked APIs (break-on-return)
• Stealth: Debugger does not rely upon Windows interface and thus evades detection by a slew of interface-related indicators, with additional stealth from hook-based protections

3. Automated Unpacking

'capemon' implements an unpacking engine using a combination of techniques

• Memory region tracking: Regions of memory revealed through indicators of execution, allocation or protection are tracked
• Early capture: Multiple possible triggers allow payload capture at earliest moment often resulting in working unpacked samples
• Injection capture: Strong coverage of injection techniques for inter-process payload capture
• PE unmapping: Integrated Scylla engine allows capture of memory or file-mapped PE images in memory
• Shellcode dumping: Shellcode & non-PE regions equally captured as payloads
• Import Reconstruction: Repairing Import Address Tables (IAT) to create functional dumped executables.
• AMSI Dumping: Intercepting and dumping buffers passed to the Antimalware Scan Interface (AMSI).

4. Config Extraction

Automated Static & Dynamic malware configuration extraction relies on 'capemon' capabilities

• Static extraction: Typically reliant upon capemon's unpacking or process dump capture before static parsing
• Dynamic extraction: When parser implementation is onerous, dynamic capture of decrypted configs can be performed by debugger via YARA signature

5. YARA integration

Integration of YARA for in-memory scanning

• Dynamic configuration: Sandbox configuration such as hooking exclusions or options implemented during detonation
• Debugger programming: Precise dynamic breakpoint address resolution using YARA signatures & cape-specific metadata
• Unpacking engine integration: Dynamic scanning of all memory regions prior to unpacking capture
• Function resolution: Allows dynamic address resolution for APIs or functions for hooking or other purposes

Technical Foundations

• Platform: Windows (x86 and x64).
• Hooking Method: Inline hooking of Win32 and Native APIs (NTAPI).
• Debugger: Native in-process 'self' debugging utilising minimal OS interfaces & hardware capabilities (breakpoint, single-step)
• Dependencies:
• distorm for instruction decoding.
• libyara for pattern matching.
• Scylla for PE reconstruction.
• bson for data serialization.

Architectural Idioms (MANDATORY - read before designing any feature)

capemon has one established way of doing each of the following. New code MUST plug into these paths; parallel mechanisms will be rejected in review.

A. Detection = YARA, not C scanners

• Byte-pattern, magic-value or header detection is a YARA rule. Monitor-wide detections go in the built-in InternalYara[] string (CAPE/YaraHarness.c); family/config detections go in the analyzer yara directory.
• Do NOT write for (p = start; p < end; p++) if (*(DWORD*)p == MAGIC) loops. Express header validation in the rule condition (uint32(@s[i] + N), for any i in (1..#s) : (...)).

B. Where detection runs = the unpacking engine's region scans

• CAPE_post_init() runs BEFORE unpacking. Anything packed (UPX, crypters, shellcode loaders) is not yet visible there.
• YaraScan(Address, Size) is already invoked on the initial image (CAPE.c), on tracked regions when they are executed or change protection (ProcessTrackedRegion etc. in CAPE.c), and on debugger-driven scans (Trace.c). A rule added to the rule set fires on all of these automatically.
• Never tie a feature to the ImageBase global or GetModuleHandle(NULL) alone.

C. Action dispatch = cape_options metadata

• A rule triggers behaviour via meta: cape_options = "...". YaraCallback parses it with user_data = base of the scanned region, and ParseOptionLine resolves $string references to match addresses.
• A new action is a new keyword handled in YaraCallback, which calls the feature with an explicit address (region base and/or base + Match->offset).
• Features take the base address as a parameter: no globals, and no re-discovery of what YARA has already located.
• Regions are re-scanned; expect repeated hits on the same region and make features idempotent (dedup by address).

D. Debugger ownership

• Each breakpoint owns its handler: SetBreakpoint(..., Callback) for hardware breakpoints, SetSoftwareBreakpoint(BPs, Address, Callback) for INT3.
• NEVER modify SoftwareBreakpointHandler, SoftwareBreakpointCallback, SingleStepHandler or CAPEExceptionFilter to dispatch a feature. Doing so hijacks every other debugger consumer (traces, YARA bp options, syscall breakpoints).
• Single-step state is per thread; do not overwrite the global SingleStepHandler to service a feature.

E. Behaviour log: one LOQ_* call per API call or breakpoint hit

• Every hook or breakpoint emits at most ONE LOQ_* record per hit. Gather every relevant field first (function name, arguments, buffers, resolved names), then emit them together in a single call with a multi-field format string (e.g. "sSS").
• Do NOT log a generic "function called" record and then one record per parameter. Unreadable arguments are logged as empty or zero-length values inside the same record.
• A call that spans entry and return (e.g. output buffers) logs once, at the point where the data is available (usually on return). Stay silent at entry.
• Bulk metadata (module info, file lists, dependency trees) is one record per object, not one per item. Filter out noise (stdlib, dependencies already listed elsewhere) and cap the size.
• DebugOutput goes to the debug log, not the behaviour log, but it should not emit per-item floods either.

F. Design review checklist (answer before writing code)

1. Does an existing mechanism already do this? Check YaraScan callers, YaraCallback options, SetBreakpoint*, tracked regions, DumpRegion.
2. Does it work on a UPX-packed sample? On a non-PE (shellcode) region?
3. Are addresses passed as arguments rather than read from globals?
4. Does it modify a shared handler? If yes, redesign.
5. Is it gated by a config option and documented in docs/configuration.md?
6. Does each hit produce at most one behaviour log record?

G. Target Runtime Internals & Re-entrancy Audit (MANDATORY before hooking a runtime or subsystem)

Before writing any hook, callback, or instrumentation for a target runtime (Win32/NT loader, .NET CLR/CoreCLR, Go runtime, scripting engines, thread pools), research and verify the runtime's internal execution invariants along five axes:

1. Lock Hierarchy & Hook Re-entrancy (`hook_disable` / `enter_hook`):

• Every hooked Win32/NT API (VirtualAlloc, VirtualProtect, VirtualQuery, CreateFile, etc.) called while hook_info()->disable_count == 0 enters enter_hook() -> operate_on_backtrace() -> our_stackwalk() -> RtlLookupFunctionEntry() (acquires the Windows loader lock / LdrpInvertedFunctionTableSRWLock on x64) and invokes unpacker handlers (AllocationHandler, ProtectionHandler).
• Any non-HOOKDEF callback (CLR compileMethod, ICorProfilerCallback, VEH CAPEExceptionFilter / SoftwareBreakpointHandler / RestoreSoftwareBreakpoint, Go inline/breakpoint dispatch, APC callbacks) MUST wrap internal Win32/NT/dump calls in hook_disable() / hook_enable() and preserve lasterror_t (get_lasterrors / set_lasterrors, with lasterror.Eflags = 0 inside exception filters).
• Critical `loq()` caveat: On upstream/capemon, loq() ends with hook_enable(), which resets hook_info()->disable_count = 0 (until PR #181's refcounted disable_count++/-- lands upstream). Any LOQ_* call inside a hook_disable() region MUST be followed by hook_disable() if subsequent code calls hooked APIs (DumpMemory, DumpPesInRange, VirtualQuery), and exit must restore the caller's saved disable_count.

1. No `CRITICAL_SECTION` Inside Hooks or Runtime Callbacks (`lookup.h` + Atomics):

• Never introduce EnterCriticalSection inside hooks, JIT callbacks, or VEH handlers: a thread holding a custom lock while calling a Win32 API (or triggering RtlLookupFunctionEntry / LdrLoadDll) causes lock-order inversion deadlocks against the OS loader lock, CLR JIT locks (JitListLock), or GC stop-the-world suspension.
• Replace locks with lookup.h (lookup_t) and atomic primitives:
• Per-thread state: LOOKUP_THREAD(&table, type) (lookup_get_or_create keyed by GetCurrentThreadId()). Never use TlsAlloc/TlsGetValue (clobbers LastError = 0) and never add fields to hook_info_t (clobbered by tmphookinfo and memcpy in hook_file.c/hook_special.c).
• Deduplication sets: LOOKUP_MARK_SEEN(&table, key) or lookup_get + lookup_add.
• Counters & budgets: InterlockedIncrement / InterlockedCompareExchange on volatile LONG.
• `lookup_get_or_create` shared-key trap: lookup_get_or_create is ONLY safe for per-thread (TID) keys. On shared keys across threads (LOOKUP_SHARED), concurrent misses both call lookup_add and insert duplicate nodes. For shared keys, fast-path lookup_get, and on miss guard lookup_get + lookup_add with a short InterlockedCompareExchange spinlock.