konflux-ci/clair-in-ci-db/ci-pipeline-debugging

CI Pipeline Debugging

Overview

This repo has two Tekton PipelineRun definitions in .tekton/: one for pull requests and one for pushes to main. Both are hermetic builds using trusted artifacts (OCI TA) with Cachi2 prefetching. The most critical and failure-prone step is fetch-db-data, which downloads the Clair vulnerability database and needs 16-32 GiB of memory.

When to Use

• A Tekton pipeline task failed and you need to understand what it does
• You need to understand the task execution order
• The fetch-db-data step is failing (OOM, timeout, network)
• A security scan is blocking the pipeline
• You need to understand PR vs push pipeline differences

Task Execution Order

init
  └─ clone-repository
       └─ fetch-db-data (non-hermetic, downloads matcher.db)
            └─ prefetch-dependencies (RPMs + generic artifacts via Cachi2)
                 └─ build-container (buildah, hermetic)
                      └─ build-image-index
                           ├─ clair-scan
                           ├─ clamav-scan
                           ├─ sast-snyk-check
                           ├─ sast-shell-check
                           ├─ sast-unicode-check
                           ├─ coverity-availability-check → sast-coverity-check
                           ├─ ecosystem-cert-preflight-checks
                           ├─ deprecated-base-image-check
                           ├─ rpms-signature-scan
                           ├─ apply-tags
                           ├─ push-dockerfile
                           └─ build-source-image (push only)
finally:
  └─ show-sbom

PR vs Push Pipeline Differences

fetch-db-data Step (Most Common Failure Point)

This step downloads the latest Clair vulnerability database. It is the most resource-intensive and failure-prone task.

The output artifact (SCRIPT_ARTIFACT) is passed to prefetch-dependencies, which means the matcher.db file is included in the source artifact chain and eventually available during the build-container step where the Dockerfile COPY matcher.db /tmp/matcher.db picks it up.

Resource Requirements

Most tasks request 10 GiB memory. Key allocations:

Security Scan Tasks

All scans run in parallel after build-image-index and can be skipped with skip-checks: "true":

Common Failures

Trusted Artifacts (OCI TA)

This pipeline uses OCI-based trusted artifacts instead of PVCs for sharing data between tasks:

• Source code is stored as OCI artifacts in the output image registry
• SOURCE_ARTIFACT results chain through: clone → fetch-db-data → prefetch → build
• CACHI2_ARTIFACT carries prefetched dependencies from prefetch → build
• Artifacts expire based on ociArtifactExpiresAfter (PR: 5 days, push: never)