version: "1.0.1" name: ci-cd-quirks description: Use when CI checks fail unexpectedly, when preparing code for CI, or when encountering non-obvious build and pipeline behavior. Covers vendoring, hermetic builds, security scans, code generation checks, coverage, and webhook validation gotchas.

CI/CD Quirks and Gotchas

Overview

Integration-service CI runs on Konflux (Tekton-based) for builds and GitHub Actions for linting/testing. Several non-obvious requirements catch developers off guard.

When to Use

• CI failed and you don't understand why
• Preparing a change that touches dependencies, CRDs, or RBAC
• Understanding what the build pipeline actually does
• Debugging webhook validation behavior

Must-Run Commands Before Push

CI Checks That Surprise People

Vendoring (most common failure)

Go vendoring prunes non-Go files. After go mod vendor, CRD YAML files from dependencies are gone. make download-crds re-downloads them for tests. CI runs this automatically, but if you updated deps and didn't vendor, CI fails.

RBAC Wildcard Check

CI greps config/ for RBAC wildcards (*) and fails if found. Never use wildcard verbs or resource names in +kubebuilder:rbac markers.

Code Generation Drift

CI runs make generate manifests and diffs the result. Any uncommitted generated files = failure. Always commit generated output.

Cached Client list/watch RBAC Requirement

The controller-runtime cached client requires list and watch verbs on every resource type served from the informer cache — omitting them causes a reflector retry storm. Resources in Client.Cache.DisableFor (currently Secret and konfluxv1alpha1.Component — see cmd/main.go) bypass the cache and do not need these verbs; the old appstudio.redhat.com Component type is not in DisableFor and still uses the cache. CI will not catch this: envtest runs as cluster-admin and the RBAC check only looks for wildcards, so the bug surfaces only under real RBAC in a production cluster.

go mod tidy Drift

CI runs go mod tidy and checks for changes. If your go.mod/go.sum differ after tidy, CI fails.

Build Pipeline Details

PR Pipeline (.tekton/integration-service-pull-request.yaml)

• Injects ENABLE_COVERAGE=true — binary built with -cover -covermode=atomic -tags=coverage
• Hermetic build (network-isolated), deps prefetched via Cachi2
• Multi-platform: x86_64 + arm64
• Image tag: on-pr-{{revision}}
• Expires after 5 days, max 3 kept
• Security scans: Clair, Roxctl, Snyk SAST, ClamAV, ShellCheck, Unicode, RPM signature

Push Pipeline (.tekton/integration-service-push.yaml)

• Regular build (no coverage) + extra build-instrumented-image task (with coverage)
• Image tag: {{revision}}
• Same security scans as PR

Dockerfile Gotchas

• Builds two binaries: manager (controller) and snapshotgc (garbage collector)
• Both ship in the same image, entrypoint is /manager
• ENABLE_COVERAGE build arg switches between instrumented and production builds
• Base image: UBI9 minimal, runs as user 65532 (non-root)

Webhook Validation Gotchas

Coverage

• Tool: codecov, flag: unit-tests
• Patch target: 85% for changed coverable lines
• Ignores: **/zz_generated*, e2e-tests/**, vendor
• Patch coverage: informational only (non-blocking)

Common Mistakes