prowler-cloud/prowler/prowler-compliance

When to Use

Use this skill when:

• Creating a new compliance framework for any provider — decide universal vs legacy first (see below)
• Syncing an existing framework with an upstream source of truth (CIS, FINOS CCC, CSA CCM, NIST, ENS, etc.)
• Adding requirements to existing frameworks, or extending a universal framework to a new provider
• Mapping checks to compliance controls
• Adding `ConfigRequirements` guardrails so configurable checks can't silently satisfy a requirement with a loosened config
• Auditing existing check mappings as a cloud auditor ("are these mappings correct?", "which checks apply?", "review the mappings")
• Adding a new legacy output formatter (table dispatcher + per-provider classes + CSV models)
• Fixing JSON bugs: duplicate IDs, empty Version, wrong Section, stale check refs, inconsistent FamilyName, padded tangential check mappings
• Investigating why a finding/check isn't showing under the expected compliance framework in the UI
• Understanding compliance framework structures and attributes

The authoritative contributor doc is docs/developer-guide/security-compliance-framework.mdx — keep this skill and that doc consistent when either changes. For reviewing a compliance PR, use the sister skill prowler-compliance-review instead.

Universal vs Legacy: The First Decision

Prowler supports two JSON schemas. Choosing wrong means unnecessary Python code, so decide this before anything else. At load time both converge: legacy files are adapted into the universal ComplianceFramework model (adapt_legacy_to_universal()), so the difference is about authoring cost and capabilities, not about what the rest of Prowler sees.

Side-by-side comparison

When to use which

Use universal when (any of these):

• The framework is new to Prowler — no existing attribute class, no

existing formatter. This is the default: zero Python changes needed.

• The framework spans (or will span) more than one provider — DORA, CSA

CCM, CIS Controls. One file covers all providers; extending to a new provider is a one-line checks edit.

• The attribute shape is unique to this framework — declare it in

attributes_metadata instead of adding a Pydantic class to the Union.

Use legacy only when extending an existing legacy family:

• A new version of a shipped legacy framework (CIS 8.0 for AWS → new

cis_8.0_aws.json, same CIS_Requirement_Attribute, same cis/ formatter).

• An existing legacy framework for a new provider (ENS for m365 → new

ens_rd2022_m365.json + ens_m365.py transformer).

• Consistency with the family matters more than the universal benefits — a

lone cis_8.0_aws in universal format while 20+ CIS files stay legacy would fragment the family.

Never: start a brand-new single-provider framework as legacy "because it's only AWS today". Universal handles single-provider fine (the checks dict just has one key) and you skip 3 output files + 3 registrations.

The same requirement in both schemas

Universal (prowler/compliance/my_framework_1.0.json):

{
  "framework": "My-Framework",
  "name": "My Framework 1.0",
  "version": "1.0",
  "description": "...",
  "attributes_metadata": [
    {"key": "Section", "type": "str", "required": true},
    {"key": "Service", "type": "str"}
  ],
  "outputs": {"table_config": {"group_by": "Section"}},
  "requirements": [
    {
      "id": "MF-1.1",
      "name": "Root MFA",
      "description": "Root account must have MFA enabled.",
      "attributes": {"Section": "IAM", "Service": "iam"},
      "checks": {
        "aws": ["iam_root_mfa_enabled"],
        "azure": []
      }
    }
  ]
}

Legacy (prowler/compliance/aws/my_framework_1.0_aws.json — plus a second file per extra provider, plus formatter + registrations):

{
  "Framework": "My-Framework",
  "Name": "My Framework 1.0 for AWS",
  "Version": "1.0",
  "Provider": "AWS",
  "Description": "...",
  "Requirements": [
    {
      "Id": "MF-1.1",
      "Name": "Root MFA",
      "Description": "Root account must have MFA enabled.",
      "Attributes": [
        {"ItemId": "MF-1.1", "Section": "IAM", "Service": "iam"}
      ],
      "Checks": ["iam_root_mfa_enabled"]
    }
  ]
}

Same control, but the universal file already covers Azure, validates its own attribute schema, and renders table/CSV/OCSF with no code. Field-by-field references for each schema follow below.

Architecture (Mental Model)

Prowler compliance is a four-layer system. Bugs usually happen where one layer doesn't match another, so know all four before touching anything.

Layer 1: SDK / Core Models — prowler/lib/check/

All in Pydantic v1 (from pydantic.v1 import ...). Three model groups live in compliance_models.py:

Legacy tree — Compliance → Compliance_Requirement / Mitre_Requirement:

• One *_Requirement_Attribute class per framework family. Registered today (Union order matters):

ASDEssentialEight, CIS, ENS, ISO27001_2013, AWS_Well_Architected, KISA_ISMSP, Prowler_ThreatScore, CCC, C5Germany, CSA_CCM, STIG (Okta IDaaS), and Generic_Compliance_Requirement_Attribute as fallback.

• Generic MUST stay LAST in Compliance_Requirement.Attributes: list[Union[...]] —

Pydantic v1 tries union members in order; Generic first would swallow every framework-specific attribute. NIST 800-53/CSF, PCI DSS, GDPR, HIPAA, SOC2, FedRAMP, SecNumCloud etc. intentionally use Generic.

• A root_validator rejects empty Framework, Provider or Name.
• MITRE uses the separate Mitre_Requirement model (Tactics, SubTechniques,

Platforms, TechniqueURL at requirement top level, per-provider Mitre_Requirement_Attribute_{AWS,Azure,GCP}).

Universal tree — ComplianceFramework → UniversalComplianceRequirement:

• Flat attributes: dict per requirement, schema declared in

attributes_metadata (key, label, type, enum, required, enum_display, enum_order, output_formats). A root_validator rejects missing required keys, unknown keys (drift guard), enum violations, and int/float/bool type mismatches. If attributes_metadata is omitted, no validation runs.

• checks: dict[provider, list[check_id]] — the provider list of the framework

is derived from these keys (get_providers() / supports_provider()); the top-level provider field is only a fallback.

• outputs.table_config (group_by, split_by, scoring, labels) drives the CLI

table; outputs.pdf_config exists in the model but is not consumed by the API PDF pipeline yet (see Layer 4).

Guardrails — Compliance_Requirement_ConfigConstraint:

• Fields Check, ConfigKey, Operator (lte|gte|eq|in|subset|superset),

Value, optional Provider (required in universal multi-provider files).

• A root_validator rejects Value/Operator type mismatches at load time.
• Evaluation is centralized in prowler/lib/check/compliance_config_eval.py

(evaluate_config_constraints, apply_config_status, get_effective_status, CONFIG_NOT_VALID_PREFIX = "Configuration not valid for this requirement."), shared by CSV/OCSF/table outputs and the API backend. A violated constraint forces the requirement to FAIL and prepends the reason to status_extended. Constraints whose ConfigKey is absent from audit_config are skipped (defaults assumed compliant).

Loaders:

• Compliance.get_bulk(provider) — legacy: scans only

prowler/compliance/{provider}/ (+ external JSONs via the prowler.compliance entry-point group). Does NOT see top-level universal files.

• get_bulk_compliance_frameworks_universal(provider) — scans both the

top-level prowler/compliance/ and every provider subdirectory, adapting legacy files via adapt_legacy_to_universal() (flattens Attributes[0] to a dict, wraps Checks as {provider: [...]}, infers attributes_metadata). Also loads external universal frameworks via the prowler.compliance.universal entry-point group (built-ins win collisions).

• get_check_compliance(finding, provider_type, bulk_checks_metadata) lives in

`prowler/lib/outputs/compliance/compliance_check.py` (not in lib/check/compliance.py). It builds the per-finding dict keyed f"{Framework}-{Version}" only when Version is non-empty — an empty Version silently produces the key "{Framework}" and breaks downstream filters and tests.

• prowler/lib/check/compliance.py now contains only

update_checks_metadata_with_compliance().

Layer 2: JSON Catalogs — prowler/compliance/

See "Compliance Catalog Coverage" below.

Layer 3: Output Formatters — prowler/lib/outputs/compliance/

Universal path (no Python needed per framework):

• universal/universal_table.py — get_universal_table(), renders the CLI

table from outputs.table_config + attributes_metadata.

• universal/universal_output.py — UniversalComplianceOutput, builds the CSV

Pydantic model dynamically from attributes_metadata.

• universal/ocsf_compliance.py — OCSFComplianceOutput; OCSF output is

always generated for universal frameworks regardless of --output-formats.

• Orchestrated by process_universal_compliance_frameworks() in

compliance.py, which runs before any legacy dispatch and removes the processed frameworks from the set.

Legacy path — per-framework directory, usually:

{framework}/
├── __init__.py
├── {framework}.py            # get_{framework}_table() summary-table function
├── {framework}_{provider}.py # One ComplianceOutput subclass per provider
└── models.py                 # One Pydantic CSV row model per provider

Directories today: asd_essential_eight, aws_well_architected, c5, ccc, cis, cisa_scuba, ens, generic, iso27001, kisa_ismsp, mitre_attack, okta_idaas_stig, prowler_threatscore, universal. Known deviations (don't "fix" them without a reason): iso27001/ has no table file (falls to the generic table), aws_well_architected/ has no per-provider files, cisa_scuba/ only ships googleworkspace.