Automating security scans in CI for Linux servers using OpenSCAP and Lynis

Operators who run Linux-servers today know the fundamental problem: security and compliance requirements are rarely one-off projects, but recurring attestations. Yet checks are often performed „by feel“ or only before audits. This is exactly where automating security scans in CI comes in: they make hardening and compliance measurable, repeatable and, above all, visible early – before an image is rolled out or a configuration change goes into production.

This practical article explains how to integrate OpenSCAP and Lynis into a CI pipeline. OpenSCAP stands for „Security Content Automation Protocol“ and, using standardized content (e.g. XCCDF/OVAL), systematically checks configurations against benchmarks such as CIS or STIG. Lynis is a proven Linux audit tool that aggregates hardening guidance, vulnerability indicators and operational risks from the perspective of an admin audit. The two tools complement each other: OpenSCAP provides structured compliance results, Lynis provides pragmatic remediation steps and „operator knowledge“. The goal is a setup that works in everyday operations: clear baselines, clean artifacts, traceable thresholds, and a fallback strategy if a gate suddenly proves too strict.

Why security scans in CI for Linux servers deliver more than „security once a quarter“

CI (Continuous Integration) is often understood in an infrastructure context as a pipeline around images, IaC (Infrastructure as Code) or configuration management. The benefit is not „more scans“, but shorter feedback cycles and less drift (deviation from the desired configuration over time).

• Early detection: misconfigurations (e.g. SSH policy, sysctl, package levels) become visible during the build or before merge, not only after rollout.
• Traceability: scan reports are pipeline artifacts. They can be versioned, compared and used as evidence for audits.
• Standardization: a security gate in CI enforces baselines, exception processes and clear responsibilities.
• Scalability: once established, the same logic checks hundreds of hosts/images without additional effort.

Important: CI scans do not replace ongoing monitoring, patch management or incident response. They are a quality filter for changes to images and configurations – and reduce the likelihood that you bake technical debt into production.

OpenSCAP and Lynis: roles, strengths and common misunderstandings

OpenSCAP in two sentences

OpenSCAP is a toolchain that processes SCAP content. SCAP content consists, among other things, of XCCDF (checklist and evaluation logic) and OVAL (test definitions). Practically this means: you select a profile (e.g. CIS Level 1) and OpenSCAP evaluates the target state. The result is a structured report (XML/HTML) suitable for compliance and comparison.

Lynis in two sentences

Lynis runs a local audit and provides findings, guidance and hardening recommendations. It is less „benchmark-driven“ and more practical: file permissions, services, kernel settings, logging, authentication, integrity checks, bootloader aspects. The result is a text report plus metrics that can be interpreted as a gate.

Common operational misunderstandings

• ‚One tool is enough‘: in practice OpenSCAP and Lynis cover different perspectives. Combined they are more robust against blind spots.
• „CI scans production“: CI should primarily scan artifacts (Images, Golden AMIs, container bases, VM templates). Production scans belong more in scheduled jobs (e.g., centrally orchestrated) with change windows.
• „Everything must be 100%“: Benchmarks contain requirements that do not fit every operating model (e.g., strict password rules for pure SSH key auth). You need baselines and documented exceptions.

Architecture: Where do scans make sense in a CI/CD pipeline?

For Linux-servers in cloud or virtualization environments a pattern has proven effective: scan the image and the configuration change, not the „running server“. That reduces side effects and makes results more reproducible.

Proven pipeline pattern (image-oriented)

• Build: build the image/template (Packer, Image Builder, custom build scripts).
• Provisioning: apply hardening (Ansible, Salt, Chef, Cloud-init modules). This is where the baseline is established.
• Scan: run OpenSCAP and Lynis against the built artifact (e.g., in a VM, via chroot, via a container approach depending on tool capability).
• Gate: evaluate results against thresholds (e.g., „no high findings“, „Compliance >= X%“).
• Publish: only publish to the registry/template repository on success.

If you still scan a „running“ target host (e.g., staging), do so in a controlled way: dedicated staging instance, fixed data, no production secrets and clear runtime limits. Otherwise scans produce unclear findings (for example due to temporary debug packages or changing mounts).

Prerequisites: What you should clarify before automating

Automated scans rarely fail because of the tool; they fail because of unclear framework conditions. Clarify in advance:

1) Target systems and benchmark reference

• Distributions and versions (RHEL/Alma/Rocky, Debian/Ubuntu, SLES).
• Role classes (web, DB, jump host, bastion, Kubernetes node).
• Relevant benchmarks (CIS, DISA STIG, internal policies). „Benchmark“ here means: a defined desired state, not „best practice by feel“.

2) Trust model and privileges

OpenSCAP and Lynis require elevated privileges (root) for many checks because they read system files, kernel parameters or service configurations. In CI this is sensitive: you do not want to run arbitrary code as root. Typical countermeasures:

• Run scans in isolated runners (dedicated VM, ephemeral container/VM, no shared runners).
• Only signed/trusted pipeline sources may trigger scan jobs (branch protection, code owner, merge gates).
• No production secrets in the scan job. Scans rarely need application secrets—if they do, that is a warning sign.

3) Ergebnisformat und Aufbewahrung

Decide early which artifacts you will store: HTML report (human-readable), XML/JSON (machine-readable), and a short summary for the gate. Plan retention (retention period) and access (audit, security, operations).

Security scans in CI: implementation with OpenSCAP and Lynis as a repeatable job

Below is a practical approach that can be mapped easily in GitLab CI or similar systems. The goal is not a “perfect” YAML for every platform, but a pattern: install tools, run the scan, archive artifacts, evaluate thresholds.

Step 1: Install tooling (observe distribution)

OpenSCAP packages have different names depending on the distribution. On RHEL-like systems these are typically openscap-scanner and scap-security-guide (SSG, a common content package). On Debian/Ubuntu they are openscap-scanner and possibly separate content packages. Lynis is often available as a package or included as a verified download. For CI it is advisable to use official repos where possible, otherwise pin versions and checksums.

#!/usr/bin/env bash
set -euo pipefail

# Beispiel: RHEL/Alma/Rocky
sudo dnf -y install openscap-scanner scap-security-guide lynis

# Beispiel: Debian/Ubuntu (Paketnamen können je Release variieren)
# sudo apt-get update
# sudo apt-get -y install openscap-scanner lynis
# Content (SSG) kann je nach Repo-Lage separat sein

Why this matters: many „pipeline breaks“ stem from content mismatches (profiles do not exist) or from missing dependencies (e.g. Python modules for individual checks). Keep tool and content versions under control, otherwise results will change without an intentional update to your baseline.

Step 2: OpenSCAP scan with profile and report artifacts

OpenSCAP typically uses the oscap CLI. Key elements are: the content file (e.g. SSG), the profile ID (e.g. CIS Level 1) and the output. For CI it makes sense to produce both a result XML (machine-readable) and an HTML report (for humans).

#!/usr/bin/env bash
set -euo pipefail

OUTDIR="artifacts/openscap"
mkdir -p "$OUTDIR"

# Beispielpfad für SSG auf RHEL-artigen Systemen (je nach Distro/Version prüfen)
SSG_DS="/usr/share/xml/scap/ssg/content/ssg-almalinux9-ds.xml"
PROFILE="xccdf_org.ssgproject.content_profile_cis"

# Scan gegen das lokale System (typisch in einer ephemeral VM/Build-Umgebung)
# --results-arf erzeugt ein ARF (Asset Reporting Format), gut für Weiterverarbeitung
sudo oscap xccdf eval 
  --profile "$PROFILE" 
  --results-arf "$OUTDIR/results.arf.xml" 
  --report "$OUTDIR/report.html" 
  "$SSG_DS"

When this fails:

• Mismatched content: the SSG file does not match the distribution/version. An AlmaLinux-content on Ubuntu will produce invalid results.
• Profile not found: Profile ID is incorrect. Check available profiles beforehand.
• „Not applicable“ flood: Many rules are not applicable because role/packages are missing. That indicates the profile is too generic or you are scanning the wrong artifact.

Check step for the content (display profiles):

#!/usr/bin/env bash
set -euo pipefail

SSG_DS="/usr/share/xml/scap/ssg/content/ssg-almaLinux9-ds.xml"
oscap info "$SSG_DS" | sed -n '1,200p'

Step 3: Run the Lynis audit and make it scoreable

Lynis produces reports under /var/log/lynis-report.dat and /var/log/lynis.log. For CI, copy the files into an artifact directory. Additionally, you need a small evaluation that extracts a metric from the report (e.g. hardening index) or counts high-risk warnings.

#!/usr/bin/env bash
set -euo pipefail

OUTDIR="artifacts/lynis"
mkdir -p "$OUTDIR"

sudo lynis audit system --quick --no-colors || true

# Reports in Artefakte kopieren
sudo cp -a /var/log/lynis-report.dat "$OUTDIR/" || true
sudo cp -a /var/log/lynis.log "$OUTDIR/" || true

# Beispiel: Hardening-Index aus report.dat extrahieren
# (Format kann je Version variieren; daher defensiv parsen)
HARDENING_INDEX=$(awk -F= '/^hardening_index=/{print $2}' "$OUTDIR/lynis-report.dat" | tail -n1)
HARDENING_INDEX=${HARDENING_INDEX:-0}

echo "Lynis hardening_index=$HARDENING_INDEX" | tee "$OUTDIR/summary.txt"

Why || true appears here: Lynis does not always use exit codes the way CI gates expect. It is better to let Lynis run, secure artifacts and implement the gate logic yourself based on clear criteria (e.g. minimum index or number of certain warning categories). This prevents “false negatives” where the job fails before reports are secured.

Step 4: Gate logic with thresholds (and why you should start small)

A security gate is only useful if it is stable. Start with conservative rules:

• Gate only fails on critical findings (e.g. specific OpenSCAP rules that you define as „must pass“).
• Everything else is reported as a warning and moved into tickets/backlog.
• Thresholds are tightened deliberately after the baseline has been established.

Example: simple gate on the Lynis hardening index (as a starting point, not as the sole source of truth):

#!/usr/bin/env bash
set -euo pipefail

MIN_INDEX=${MIN_INDEX:-70}
REPORT="artifacts/lynis/lynis-report.dat"

IDX=$(awk -F= '/^hardening_index=/{print $2}' "$REPORT" | tail -n1)
IDX=${IDX:-0}

if [ "$IDX" -lt "$MIN_INDEX" ]; then
  echo "FAIL: Lynis hardening_index $IDX ist kleiner als Mindestwert $MIN_INDEX"
  exit 2
fi

echo "OK: Lynis hardening_index $IDX (>= $MIN_INDEX)"

Pitfall: A single index can mask improvements in one area while another area deteriorates. Use the index as an „early warning“, but define concrete must-have criteria in the medium term (e.g. „Root login via SSH disabled“, „Auditd active“, „critical file permissions corrected“).

Example: GitLab CI job structure with artifacts

The following example shows a rough structure that you can adapt to your environment. It is about the principles: isolated runner, artifacts, clear stages, and a gate that does not lose reports.

stages:
  - build
  - scan

variables:
  MIN_INDEX: "70"

scan_security:
  stage: scan
  image: almaLinux:9
  tags:
    - isolated-runner
  script:
    - bash ci/install-tools.sh
    - bash ci/run-openscap.sh
    - bash ci/run-lynis.sh
    - bash ci/gate-lynis.sh
  artifacts:
    when: always
    expire_in: 30 days
    paths:
      - artifacts/openscap/
      - artifacts/lynis/

Important for operation: the runner must be built so that root actions are possible (or you perform scans inside a VM that the job starts). On shared runners this is often not permitted — and from a security perspective not recommended.

Cloud and image workflows: what changes compared to bare metal

In cloud environments (IaaS, VM templates, golden images) two effects are relevant:

Ephemeral hosts and baseline drift

If instances are rebuilt regularly, CI is the right place to ensure new images do not regress. Drift then typically arises from “Day-2” changes (hotfixes on running hosts). In that case teams combine CI scans (before release) with periodic compliance jobs (e.g. monthly) on representative hosts.

Cloud-init, agents and provider-specific defaults

Cloud-init can modify SSH settings, users, host keys or package sources. Provider images also often include their own agents (e.g. for monitoring, guest tools). This leads to findings that are not “insecure” but are divergent. Practical rule: scan the artifact after your provisioning steps and with the agents that will actually be present later. Otherwise you get a baseline that is never achieved in reality.

Troubleshooting: common failure patterns and how to resolve them systematically

Problem 1: OpenSCAP cannot find profiles or produces empty reports

• Cause: incorrect DataStream file (SSG) or wrong profile ID.
• Check: oscap info on the DS file, compare profiles, verify paths per distribution.
• Solution: Do not hardcode the content path in the pipeline; set it per OS matrix (e.g. a variable per job).

Problem 2: Many “fail” results due to CI environment instead of the target baseline

• Cause: You are scanning a build environment that does not match the later server role (missing mounts, different kernel parameters, temporary packages).
• Check: Verify role and package lists, run the scan in a VM that is closer to production.
• Solution: Place the scan stage at the end of the image build and keep environment variations minimal.

Problem 3: Lynis reports “skipped tests” or contradictory messages

• Cause: missing tools (e.g. netstat/ss), limited privileges, container environment without systemd, read-only FS.
• Check: read Lynis log, selectively install the missing dependencies.
• Solution: do not run scans in overly restricted containers; run them in a VM/privileged context or inside the image itself.

Problem 4: Gate is unstable due to fluctuating results

• Cause: unpinned package versions, changing content, non-deterministic environment.
• Check: log tool and content versions, pin build inputs (repos, mirrors, version states).
• Solution: „Policy as Code“: version baselines and exceptions, plan updates deliberately (e.g., monthly content-refresh pipeline).

Checklist: From „first scan“ to a reliable CI security gate

• Scope: Which OS versions and roles are covered? Which benchmarks are relevant?
• Isolation: Dedicated runners/VMs, no shared runners with root.
• Determinism: control tool/content versions, keep the scan environment stable.
• Artifacts: HTML for humans, XML/ARF for machines, retention defined.
• Baseline: measure first, then set thresholds. Document exceptions.
• Gate design: start with a few must-have criteria; tighten later.
• Operationalization: findings into tickets/backlog, responsibilities clear.

Fallback strategy: What to do if the gate suddenly blocks everything?

A security gate becomes dangerous when it stops deployments uncontrollably without an available process to proceed. Therefore plan a fallback strategy that balances security and delivery capability:

1) „Soft fail“ as a transition

In early phases: the job may fail but should not block the entire release (e.g., only a warning). Once baselines are stable, switch to „hard fail“ for clearly defined criteria.

2) Break-glass with documentation

If an urgent fix must be deployed, you need a controlled exception: e.g., merge approval by Security/Operations and automatic generation of an exception log (ticket ID, expiry date). The important factor is not the tooling but that exceptions are visible and time-limited.

3) Baseline versioning and rollback

Version profiles, tailoring (customized rules) and thresholds. If a content update suddenly generates new failures, you can revert to the last working baseline. That’s the difference between „CI blocks us“ and „we control changes“.

Conclusion: OpenSCAP and Lynis in CI bring stability to compliance and hardening

Automated security scans in CI are not an end in themselves. Properly implemented they provide exactly what admin teams need in daily operations: reproducible checks, traceable reports and early alerts before misconfigurations creep into new images and rollouts. OpenSCAP provides the structured benchmark perspective, Lynis the pragmatic audit and hardening view. The key lies in stable baselines, clean isolation of runners, clear artifacts and a gate that is tightened step by step. Following this approach reduces drift, lowers audit stress and enables operationalizing security requirements without blocking delivery.

For this topic Linux Hardening and Compliance Scans are also important. The article places these aspects into context and shows what matters in day-to-day operations.