SIEM for small teams: Elastic Stack vs. Splunk Light – architecture, use cases, rule design and alert tuning

Introduction

For many small IT teams, expectations of a SIEM (Security Information and Event Management — central collection, correlation and alerting of log data) are high while available resources are limited. This article shows practical steps to build a SIEM for small teams, decide between the Elastic Stack (Elasticsearch, Beats/Logstash, Kibana) and a leaner Splunk variant (Splunk Light / Single-Instance), prioritize use cases, approach rule and alert design systematically, and tune alerts efficiently. The goal is a maintainable, scalable setup that becomes productive in weeks rather than months and incurs manageable operational effort.

When does a small team actually need a SIEM?

Before investing time and budget, evaluate concrete requirements and expected benefits. A SIEM is appropriate when audit, correlation or automated alerting requirements exist. For occasional ad-hoc analyses, centralized log aggregation is often sufficient.

SIEM for small teams: decision criteria

The focus criterion is operations: who will patch, scale and monitor the solution? Smaller teams prefer solutions with clear operational maturity and low update overhead. Important comparison axes are:

• Operational effort (monitoring, JVM/DB tuning)
• Cost model (license vs. infrastructure costs)
• Flexibility (mapping, enrichment, export)
• Ecosystem (content packs, integrations, community)

Short overview: Architecture of Elastic Stack vs. Splunk-Light

Both approaches follow the same basic principle: agents collect logs, a transport/queue decouples agents from indexers, indexers store data and a search/visualization component enables analysis.

Elastic Stack — recommended minimal architecture for small teams

Core components: Filebeat (agent), optional Logstash (parsing/enrichment), Elasticsearch (index/store), Kibana (dashboard). On very small teams, Elasticsearch and Kibana can run on a single VM; for production a clear separation of roles is advisable. Elastic offers control over mappings and lifecycles but requires JVM, heap and I/O fine-tuning.

Splunk Light / Single-Instance

Core components: Universal Forwarder (agent), Splunk Indexer/SearchHead (often combined). Splunk Light enables a fast start and includes ready-made content packs, but becomes more expensive as ingest grows and is less open in index structure.

Setup: Practical steps for a quick proof-of-concept

Plan a two-stage approach: PoC (2–4 weeks) and stabilization (monitoring, retention, hardening).

1) Base: set up log forwarding

Recommendation: Start with host agents for native logs. For Linux: Filebeat. For Windows: Winlogbeat or Splunk Universal Forwarder. Filebeat records offsets, delivers efficiently and scales easily; verify timezone and timestamps.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/auth.log
    - /var/log/syslog
output.elasticsearch:
  hosts: ["https://es01.example.local:9200"]
  username: "filebeat"
  password: "changeme"

2) Parser & Enrichment

Logstash or Elasticsearch ingest pipelines structure data and add fields (e.g., GeoIP, asset tags). Without a clean parser, rules are often triggered incorrectly.

3) Index and lifecycle strategy

Define index naming scheme and ILM (Index Lifecycle Management) for automatic transitions to warm/cold. Example: daily indices, hot phase 7 days, warm 30 days, cold 90 days, snapshots to object storage.

Prioritize and specify use cases

Small teams should focus on use cases with high ROI, e.g. authentication anomalies, privileged account changes, network exfiltration, web app attacks, and ransomware indicators. Define required log sources, thresholds, and response steps for each use case.

Rule design: methods and examples

Rules are hypotheses: „If X and Y occur within Z minutes, suspicion A is likely.“ Good rules are precise, robust against noise, and explainable. Building blocks: source/fields, baseline/whitelist, time window, enrichment, and performance budget.

Example rule: brute-force detection

{
  "query": "event.action:authentication_failed",
  "group_by": ["user.name"],
  "threshold": 10,
  "time_window": "5m",
  "condition": "count(distinct source.ip) > 3"
}

The combination of a volumetric threshold and distinct checks reduces false positives. Error sources are missing IP fields or shared accounts.

Alert tuning: systematically reduce, better prioritize

Alert fatigue is the greatest risk. Goal: maximize the number of manageable alerts. Steps: filtering, enrichment, aggregation/suppression, prioritization via a scoring model.

Suppression & throttling

Throttling prevents alert flooding; escalation rules must still keep persistent incidents visible.

# Pseudo Watcher‑Konzept
watch:
  trigger: { schedule: { interval: "1m" }}
  input: { search: { request: { indices: ["logs-*"], body: { query: {...} } } } }
  condition: { compare: { "ctx.payload.hits.total": { "gt": 0 } } }
  actions:
    email_action:
      throttling:
        period: 10m

SIEM for small teams: WordPress specifics

WordPress installations are common SIEM use cases: wp-login attacks, plugin exploit patterns, unusual admin changes, and PHP errors. Typical log sources: web server access/errors, PHP-FPM logs, WordPress audit plugins (if installed), and database logs for suspicious queries.

Detect wp-login attacks via patterns in access logs, e.g. many POSTs to /wp-login.php or /xmlrpc.php. Structured fields (request, status, source.ip, user_agent) make correlation with EDR traces or firewall logs easier.

Filebeat provides modules for nginx/apache. Enable these modules and add a processor set for user and request fields:

# Beispiel: Filebeat Module aktivieren
filebeat modules enable nginx
filebeat setup --dashboards
sudo systemctl RESTart filebeat

If you want to produce specific WordPress indicators (e.g. many wp-login POSTs), you can use a simple Logstash grok rule:

grok {
  match => { "message" => "%{IPORHOST:clientip} - - [%{HTTPDATE:timestamp}] "%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} %{NUMBER:bytes} "%{DATA:referrer}" "%{DATA:useragent}"" }
}
if [request] =~ "/wp-login.php" and [verb] == "POST" {
  mutate { add_tag => ["wordpress_login_attempt"] }
}

Why this matters: early tagging simplifies later rules and dashboards. Error sources: caching layer or reverse proxy modifies request fields; verify header forwarding.

Rule backtesting and quality metrics

Before rules go into production, you must backtest them against historical data. Backtesting reveals typical False‑Positive sources and performance costs and allows you to determine metrics such as Precision, Recall and average handling time per alert.

Practical approach:

• Choose a data period (e.g. 30 days) and run the same queries against the historical index set.
• Manually evaluate 50–100 alerts, mark False‑Positives, refine the rule.
• Define metrics: target Precision ≥ 80% with acceptable Recall per use case.

Operational and security hardening

Security and operational hardening encompasses access control, roles, encryption and patch management. Practical points:

• Service accounts with minimal privileges; no shared admin accounts.
• TLS for agent-to-indexer traffic; mutually authenticated TLS when possible.
• Enable audit logging on the SIEM itself (e.g. Elastic Security Audit; use internal audit logs for Splunk).
• Distribute endpoint integrations only via verified signatures/deployment mechanisms.

Example: rsyslog forwarding with TLS has already been demonstrated; additionally check certificate rotation and CRL/OCSP processes.

Cost and resource planning brief (Sizing rules of thumb)

For small teams a simple calculation helps: expected ingest rate (GB/day) × retention (days) × compression factor (0.4–0.6) ≈ raw data requirement. Account for copies for snapshots and replication. Elastic requires I/O capacity (SSD) and sufficient RAM for heap/file system cache; Splunk processes ingest efficiently but incurs licensing costs per GB.

Example: 20 GB/day × 30 days × 0.5 = 300 GB usable index data plus snapshots and replicas → plan 1–1.5 TB provisioned storage for headroom.

SOAR, automation and playbooks

Automation reduces MTTR (Mean Time To Respond). For small teams a lean SOAR integration is often sufficient: automatic enrichment (Threat‑Intel Lookup), automatic IP blocking in firewall/proxy and ticket creation. Choose simple, reliable actions.

# Example playbook (pseudo‑YAML) - on alert: suspicious wp-login flood
name: wp_login_flood_response
triggers:
  - alert_type: wordpress_login_flood
steps:
  - name: enrich_with_threatintel
    action: lookup_threatintel
    params: { ip: "{{source.ip}}" }
  - name: create_ticket
    action: create_ticket
    params: { queue: "security", summary: "WP login flood from {{source.ip}}" }
  - name: block_ip_temporarily
    action: firewall_block
    params: { ip: "{{source.ip}}", duration: 3600 }
  - name: notify_oncall
    action: notify
    params: { channel: "#secops", message: "WP flood blocked: {{source.ip}}" }

Why this works: automatic enrichment provides context, ticketing creates traceability, firewalls immediately stop further damage. When it fails: when enrichment produces false positives or firewall rules are deployed inconsistently.

Metrics, reporting and KPIs

Measurable metrics help small teams set priorities. Important KPIs:

• Alerts per day (by priority)
• Median MTTR per priority level
• Precision/False‑Positive‑Rate per rule set
• Storage costs per GB/month

Regular reports (weekly for Ops, monthly for management) show trends and enable budget forecasting.

Typical pitfalls and checklist

Common pitfalls:

• Introducing too many rules at once → Alert‑Flooding.
• Dynamic mapping without limits → field explosion in Elasticsearch.
• Unchecked timestamp formats → incorrect correlation.
• Missing snapshot RESTore tests → false sense of backup security.

Quick checklist before production deployment:

• Timestamps consistent (prefer UTC)
• Index templates defined (mapping limits)
• ILM/Retention enabled
• Snapshot plan documented and RESTore tested
• Playbooks for top-3 alerts ready for deployment

Config example: Minimal index template snippet (Elasticsearch)

{
  "index_patterns": ["logs-*"],
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": { "type": "keyword" }
        }
      }
    ]
  }
}

Why: Prevents field explosion caused by the default-text MAPPING and makes many fields searchable/aggregatable without expensive full-text analysis.

Conclusion

A SIEM for small teams works best when you take a pragmatic approach: start lean with 3–5 prioritized use cases, use robust agents (Filebeat/Universal Forwarder), define an index/retention strategy, perform early rule backtesting and continuously tune alerts. Elastic Stack offers more long-term flexibility and cost control at the expense of higher operational effort; Splunk Light delivers results faster but can become more costly during growth phases. Complement your SIEM with clear playbooks, simple automations and regular RESTore tests so it reliably supports you in an incident.

Extended checklist to take away

• Start with 3–5 prioritized use cases.
• Configure agents uniformly (timestamps, host metadata).
• Define index lifecycle and snapshots before going live.
• Equip rules with whitelists, enrichment and suppression.
• Write and test playbooks for the most important alerts.
• Plan monthly snapshot RESTore tests and continuous heap/disk monitoring.
• Integrate rule backtesting into a CI/CD-like process (Rules as Code).

SIEM for small teams: resilience, backpressure and audit trails

For small teams, the decisive factor is not only feature functionality but above all operational resilience. Design the architecture so that short-term peaks, ingest spikes and faulty parsers do not immediately take down the entire system.

Decoupling and backpressure

A simple but effective approach is a buffer layer (e.g., Kafka, RabbitMQ or a cloud-based queueing). Advantages: agents write locally to a resilient buffer that the indexer can consume at its own pace. Risks: additional operational effort and latency. Recommendation for small teams: a lightweight queue (single-node Kafka or S3-staged files) only for critical sources; measure latency and backlog size before expanding the component.

Observability of the SIEM stack itself

Monitor these metrics per indexer/node: ingest rate (GB/min), index lag, JVM heap utilisation, GC pauses, merge/segment count, disk watermark and refresh latencies. Set alerts at Heap > 75 %, merge-queue growth or Disk usage > 70 %. Early warnings allow controlled actions (e.g., throttle index refresh or temporarily disable parsers).

Safely introduce parser and rule changes

Changes to Grok/ingest pipelines are a frequent source of errors. Use a „Rules as Code“ pipeline: Git → CI → Staging. Practically: dual-write for 24–72 hours (old + new) and comparison reports on matches and false positives. For critical production sources, a canary stream (1–5 % of traffic) can reveal early issues without risking overall operation.

Privacy and correlation: consider trade-offs

Field masking or hashing reduce PII risks but limit correlations (e.g. in user investigations). Decide per use case: pseudonymize for trend detection, but retain unaltered raw data in a secured, short-lived WORM archive for forensic cases.

Rapid rollback strategy

Define a rollback path before changes: switch agent endpoints, disable new index templates, restore snapshots. Test the route at least once per quarter. Small teams benefit from simple automations (Ansible/PowerShell playbooks) for switches instead of manual steps.

• Quick check: Buffer available? Observability alerts defined? Dual-write planned?
• Patch/Upgrade: snapshot before every major change.
• Privacy: document masking policy, secure recovery path.

Rule design is also important for this topic. The article places these aspects in a comprehensible context and shows what matters in day-to-day operations.