sentinel-one/ai-siem/sdl-log-parser

SentinelOne SDL Log Parser Authoring

<!-- CONFIG-FILE-ADDRESSING v1 -->

SDL config files: address by `udoId`, and do not trust a REST listing.

REST listFiles / getFile cannot see udoId-addressed /dashboards/ files, so getFile

returns 404 on a dashboard the console is displaying and the listing under-reports (measured on

one tenant: REST 8, GraphQL 17, console 48 files). **If a listing disagrees with what the UI

shows, the listing is wrong until proven otherwise**, change read path before concluding the

object is missing or the token lacks scope. Use the GraphQL configFiles / configFile surface.

A name-addressed addConfigFile to /dashboards/ creates a duplicate instead of updating;

address dashboards by udoId with expectedVersion. content is HJSON, not JSON. S1-Scope

changes which files exist as far as the caller can tell.

Full detail: `sdl-api/references/config-file-graphql.md`

This skill turns raw log samples into deployed, validated SDL parser definitions. A parser is an augmented-JSON file at /logParsers/<name> on the SDL tenant that extracts fields from each ingested line. The parser editor and the Test Parser button in the console run the parser client-side in JavaScript; this skill mirrors that workflow programmatically and finishes by ingesting a sample through the deployed parser to confirm the actual ingest path works.

Deploy and validate through the `s1-secops-mcp` MCP tools. sdl_get_file / sdl_put_file read and deploy the parser, powerquery_run validates on the live stream, and hec_ingest runs the synthetic-ingest fallback. The server runs on the user's machine, so it reaches *.sentinelone.net where the Cowork sandbox cannot, and it reads credentials from environment variables or the OS keychain. If the tools are missing, point the user to the s1-secops-mcp README (s1-secops-mcp/README.md in the s1-secops-skills source repo, mcp/s1-secops-mcp/README.md in ai-siem; not shipped inside the plugin).

Gate 0: can an SDL parser run on this source at all? (check this first)

The SDL parser sees exactly two attributes on an event: message and parser. Every other attribute on the event is invisible to it. The value of the parser attribute names which parser to apply, and that parser is applied to the content of message. That one fact decides whether writing a parser is the right move, so settle it before you invest in authoring.

An SDL parser cannot reach fields that live outside `message`. If the fields you want to transform are already promoted to top level on the event, an SDL parser is the wrong tool however it is written: it has no way to read them. That work belongs in DPM.

Determine which case a source is in by inspecting a sample event for the presence of message and parser. The PowerQuery probes for that, plus how to set the parser attribute from DPM, are in references/parser-eligibility.md.

Two hard rules (every parser, every time)

These two rules apply to every parser you write or edit, regardless of source, regardless of whether you're starting from scratch or copying from the catalog. Violating either produces a non-conformant parser even if extraction works.

Rule 1: The 4 mandatory attributes are always present.

attributes: {
  "dataSource.category": "security",   // hardcoded constant, never any other value
  "dataSource.name":     "Corelight",  // specific product/service name
  "dataSource.vendor":   "Corelight",  // parent vendor / company name
  "metadata.version":    "1.0.0"       // semver; bump on substantive parser changes
}

dataSource.category is fixed at "security". Not application, not network, not system, not audit. The catalog has parsers that violate this; you do not.

metadata.version is mandatory and follows semver (MAJOR.MINOR.PATCH, with optional pre-release suffix like -rc1 or -beta1). Increment it on every build. Patch bump for fixes, minor for additive changes, major for breaking schema changes. It can also be set inside mappings via a constant op when the source itself carries a vendor schema version you'd rather propagate per-event:

{ constant: { field: "metadata.version", value: "<your-current-version>" } }

A constant in mappings overrides the parser-root attributes: value when it fires, which is useful when one parser handles multiple vendor schema versions and you want each event tagged with the right one.

Rule 2: Every OCSF field name comes from `references/ocsf-schema-documentation.md`.

Before emitting any OCSF dotted path (src_endpoint.ip, actor.user.name, metadata.product.vendor_name, etc.), grep references/ocsf-schema-documentation.md for the exact string. Do not invent field names. Do not copy from a catalog parser without verifying. Do not rely on memory. The reference is the single source of truth for the ~25,759 documented OCSF field names across the 7 categories.

The rest of this skill assumes both rules are upheld. If either is unclear for a given parser, stop and resolve before continuing.

Critical pitfalls (read before authoring)

These are the DSL traps that cost the most iteration cycles when authoring parsers on the current S1 tenant. They are repeated in the long gotcha list below with more context, but if you're about to write a new parser, internalize these first. Items 1-9 are tenant-validated as of 2026-05-26; items 10-12 as of 2026-06-22.

1. Key=value catch-all REQUIRES leading `.*`. The canonical form is ".*$_=identifier$=$_=quoteOrSpace$" with repeat: true. Without the .* the format anchors at position 0 of the message and fires zero times on any line starting with a syslog priority <NNN>, a CEF header, an XML envelope, or a timestamp, i.e. virtually every real log source. The leading .* lets the engine scan forward to the first key=value pair and then repeat: true walks the rest. This is the same pattern the built-in keyValue parser uses. Also: `{parse=keyValue}` is NOT a valid sub-parser directive; built-in parsers like keyValue are only usable via root-level aliasTo:.
2. v1 mappings are FIRST-MATCH-WINS, which means renames must be duplicated into every per-subtype block alongside the constants. A catch-all predicate: "true" rename block placed at the end NEVER fires for events that matched an earlier per-subtype block, only the truly-unmatched events get those renames. If you have per-subtype constant blocks for OCSF class/activity fan-out (e.g. subtype=='forward' → activity_id=6), you must put the full rename list inside EACH of those blocks. Verbose but correct.
3. v0 constants take `field`, not `output`. A v0 constants: entry keyed output (the key v0 renames: and copies: use) is rejected by putFile ("Expected a field", surfaced by the MCP as "Content is invalid json"). Keyed field ({ field: "class_uid", value: 4001, type: "int" }) it deploys and applies, which is the shape every marketplace v0 parser uses. Live-validated 2026-10-13; this corrects an earlier note that all v0 constants: were rejected. For new parsers prefer v1 ({ constant: { field, value } }) anyway.
4. Same-name v1 rename silently NULLS the field (a same-name v0 renames entry keeps the value, live-validated 2026-10-13). Writing { rename: { from: "action", to: "action" } } to "keep" a captured raw field under its own name actually deletes the value. The KV catch-all already extracts raw vendor field names directly into the event, to "keep" one, OMIT the rename. Only write a rename when the target name is genuinely different from the source.
5. Unquoted numerics from `quoteOrSpace` are auto-typed as NUMBER columns. A FortiGate srcport=44321 sentbyte=8421 line going through the KV catch-all produces srcport: 44321 (number), sentbyte: 8421 (number), queryable with avg() / sum() / > N without number() wrapping. Quoted values stay as strings even if they look numeric (level="6" → string). Caveat: SDL column type-locking still applies, so wrap numerics with number() in dashboard panels and STAR rule bodies if the column has ever been written as string in this tenant's history.

Two related queryability rules (not DSL traps, but they matter for any consumer of the parser):