IT-Admin.tech

Configure Link Aggregation (LACP) reliably and detect deadlocks

Diagramm einer LACP Link-Aggregation mit gebündelten Links, Peer-Link und markierten VLAN/MTU-Pfaden
Visualisierung einer LACP‑LAG mit Peer‑Link und gekennzeichneten VLAN/MTU‑Pfaden zur Fehleranalyse und Architekturplanung.

Link aggregation (LACP) is a fundamental operational mechanism to achieve redundancy and aggregated throughput. The focus keyword Link-Aggregation (LACP) is intentionally placed at the beginning: LACP bundles multiple physical Ethernet links into a logical connection (LAG, Link Aggregation Group) and negotiates this using LACPDU frames. In practice, issues arise less from the protocol itself than from inconsistent ancillary settings: VLAN tagging, MTU, hashing, LACP timers, multi-chassis setups (MLAG/vPC) or loop protection. This guide presents a compact verification sequence, typical failure patterns, concrete tests and a pragmatic fallback strategy for production operation.

What LACP negotiates — and what you must clarify separately

LACP (IEEE 802.3ad / 802.1AX) ensures that two sides recognize member ports as part of an aggregation and agree which physical links become active. LACP governs member assignment, but not VLAN tagging, MTU or forwarding decisions (hashing). This separation is central: a LAG can be formally “up” while specific VLANs, large transfers or firewall sessions fail.

Decisions before configuration

Topology: Single-Chassis vs. Multi-Chassis

Single-chassis LAG terminates on a single switch/stack. Multi-chassis LAG (MLAG/vPC) distributes members across two physical switches, increasing availability but creating an additional fault domain: peer-link, state-sync and split-brain protection. Many apparent LACP deadlocks are actually MLAG synchronization issues.

Mode and timers

Choose the LACP mode (active/passive) deliberately. Active/active is usually more reliable in heterogeneous environments because both sides send LACPDUs. Set the LACP rate (fast/slow) on both sides: differing timer values can lead to unexpected re-selections.

Failure strategy: Min-Links, fallback, degradation

Define Min-Links (minimum number of active members) to avoid a single link carrying traffic alone as the „bundle.“ Avoid automatic fallback to static Port-Channels where possible; that increases loop risk if the remote endpoints are not configured identically. For MLAG, plan clear behavioral rules for peer-link disruptions.

Typical pitfalls — practical causes

VLAN/Trunk inconsistency

Most often LACP does not fail at negotiation but because VLANs are missing or trunk/access settings differ on member ports. Symptom: the LAG is green, but individual VLANs or services are unreachable. Always check the trunk configuration on the LAG interface, not on individual ports.

MTU and jumbo-frame mismatch

MTU is path-dependent. If a link or an intermediate point has a smaller MTU, the path will fragment or drop packets. Typical result: small packets work, large transfers fail. PMTUD breaks when ICMP „Fragmentation needed“ is filtered.

Hashing does not match the load

LACP distributes flows via hashing (e.g. L2/L3/L4 fields). An „elephant flow“ can saturate a single member while others remain idle. This is often operationally relevant for firewalls, backup streams or storage replication.

Physical issues and link flapping

Faulty transceivers, fiber breaks, DAC issues, autoneg mismatches or power-saving mechanisms cause flaps and constant renegotiations. LACP may remain in the up state, yet STP, MAC learning and forwarding fall into continuous fluctuation.

STP and Loop‑Protection

STP treats a correctly formed LAG as a logical port. On mismatch (e.g., static vs. LACP) STP may see and block member ports individually — this causes apparent deadlocks. Loop-protection features (BPDU Guard, UDLD) can set ports to err-disable and thus partially disable the bundle.

Deadlock-like patterns and technical causes

Deadlock is not an LACP state, but describes operational situations where control and protection mechanisms block each other. Four recurring patterns:

  • LAG up, but no or not the expected traffic: VLAN/MTU/hashing or a unidirectional link fault.
  • Periodic outages: timer mismatch (fast/slow) or flapping rhythms.
  • MAC flapping / ARP issues: loop, MLAG split-brain or host teaming mismatch.
  • Member selected, but forwarding blocked: STP/Loop-Protect or UDLD err-disable.

Investigation path in the incident — quick, reproducible steps

Proceed in a structured way: from simple configuration checks to Layer-1/2 tests.

1) Narrow down: What exactly is affected?

Questions: Only one VLAN? Only large transfers? Only one direction path? Only one member? Were there recent changes (firmware, cables, configuration)?

2) Check LACP status on both sides

Compare Actor/Partner IDs, member list, aggregator ID and whether ports are in the collecting/distributing state.

Shell
# Beispiel: Linux Bonding-Status und Link-Stats
cat /proc/net/bonding/bond0
for i in eth0 eth1; do
  echo "=== $i ===";
  ethtool $i;
  ethtool -S $i | egrep -i "err|drop|crc|discard|timeout" || true;
done
lldpcli show neighbors

3) Check VLAN/trunk consistency

Check tags via a capture on a member port and test reachability per VLAN. If tags are missing or appear only on individual members, the switch configuration is inconsistent.

Shell
# VLAN-Tag-Paket sichtbar machen
tcpdump -eni eth0 -c 50 vlan
# Beispiel: VLAN-Subinterface prüfen
ip -d link show bond0.100

4) Test MTU path

Use DF pings to verify path MTU. Note: ICMP filtering can disrupt PMTUD.

Shell
# IPv4: DF-Tests
ping -M do -s 1472 -c 3 198.51.100.10  # MTU 1500
ping -M do -s 8972 -c 3 198.51.100.10  # MTU 9000 (Jumbo)

5) Error counters and unidirectional faults

CRC-Errors, FEC corrections or input errors indicate Layer-1/2 issues. UDLD can detect unidirectional faults, but enable it only after the behavior has been documented over time.

6) Check STP/loop protection

Do STP views match? Is the LAG treated as a single interface, or do you see individual members in the bridge? BPDU Guard events or topology changes often correlate with LACP problems.

7) MLAG/vPC: Peer-Link as its own fault domain

Check peer-link stability, VLAN allow-list on the peer link, keepalive path and consistency checks. Many protection mechanisms disable ports if the peer link has problems.

Targeted experiments in operation (quickly falsify hypotheses)

Disable member links one at a time

Temporarily disable individual members to see whether the problem disappears or moves. If so, the path or device for that link is the cause.

MTU and PMTUD experiments

Test large transfers with DF pings and controlled file transfers. If fragmentation messages are missing (due to ICMP filtering), you will only detect PMTUD issues by observing TCP retransmits.

Intentionally provoke/analyze MAC flapping

Determine whether hosts are using incorrect teaming modes. On Linux /proc/net/bonding shows the mode; mismatches between switch LACP and host teaming are classic sources of error.

Concrete configuration examples and verification commands

The following examples help identify common configuration patterns and validate them. Adapt syntax to your vendor and OS version.

Cisco IOS / IOS-XE: Port-Channel with LACP

Shell
interface Port-channel10
 description LAG to Firewall-Cluster
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 10,20,30
 ip mtu 9000
!
interface GigabitEthernet1/0/1
 switchport mode trunk
 switchport trunk allowed vlan 10,20,30
 channel-group 10 mode active
!
interface GigabitEthernet1/0/2
 switchport mode trunk
 switchport trunk allowed vlan 10,20,30
 channel-group 10 mode active

Important: trunk and MTU settings are effective on the Port-Channel. Individual members should not have differing VLAN or MTU settings.

Linux Bonding (active-backup vs. 802.3ad)

Shell
# /etc/network/interfaces (Debian example)
auto bond0
iface bond0 inet static
  address 10.0.0.10/24
  bond-mode 802.3ad
  bond-miimon 100
  bond-lacp-rate fast
  bond-slaves eth0 eth1
  mtu 9000

Note: bond-lacp-rate fast corresponds to the fast LACP timer. Use the same setting on the switch.

Firewall-specific how-tos and troubleshooting

Firewalls are particularly critical because they maintain stateful sessions. A single packet loss or an asymmetric path can break sessions.

1) Check session and cluster health

Inspect session counters, drops and cluster sync errors on the firewall. For clusters, check replication or heartbeat errors—these often correlate with LACP issues.

2) Asymmetric routing checks

Ensure return path and ingress hashing are consistent. If symmetry is missing, run packet traces (tcpdump) on both sides and compare TCP and IP headers.

Shell
# Example: packet trace on firewall interface
tcpdump -ni eth0 -w /tmp/fw_eth0.pcap 'tcp and host 10.0.0.5'
# in parallel on the far side
tcpdump -ni eth1 -w /tmp/switch_eth1.pcap 'tcp and host 10.0.0.5'

3) Check PMTUD policy

Firewalls must not block ICMP wholesale. Check ICMP filter rules and, if necessary, temporarily enable logs for ICMP „Fragmentation Needed“.

4) Hashing policy for stateful services

For VPN, IPSec, or stateful-proxy workloads, an L3/L4 hash that includes source and destination ports is recommended. Avoid pure L2-based hashing when many sessions originate from the same IP range.

Operational Runbook: Incident playbook for LACP failures

A concise runbook increases the chance of rapid recovery. Important: clear responsibilities, communicable steps, secure telemetry.

Emergency steps (short version)

  1. Confirm the alarm and document consequences (affected VLANs, services).
  2. Export snapshots of configurations and relevant logs (switch, firewall, host).
  3. Disable and observe one member at a time in a controlled manner (max 30s between changes).
  4. If there is a clear indication of a member fault: shut down the port, activate replacement LAN/transceiver.
  5. If MLAG is affected: check the peer-link, if necessary temporarily isolate the peer and establish single-chassis operation.
  • After stabilization: RCA (Root Cause Analysis) within 24–48 hours and update of the runbooks.
  • Config/Log export examples

    Shell
    # Switch: back up configuration (example SSH session)
    show running-config | redirect flash:running-config-$(date +%F).txt
    show logging | redirect flash:logs-$(date +%F).txt
    # Firewall: sessions and cluster status
    show session summary
    show cluster status
    

    Monitoring and preventive checks

    Automate the following checks to get early alerts:

    • Member count per LAG and unexpected changes
    • LACP state changes (partner ID change, suspended)
    • PMTU error rate and ICMP fragmentation messages
    • MAC flapping rate per VLAN
    • Peer-link latency, drops and keepalive errors (for MLAG/vPC)

    Metrics can be collected via SNMP, telemetry (gNMI/streaming) or via syslog/collectd. Establish baselines and alert on deviations rather than only on static thresholds.

    Best practices summarized

    • Configuration symmetry: Trunk/VLAN/MTU/speed/auto-negotiation identical on both sides.
    • Timing: align LACP rate and timeouts on both sides.
    • Min-Links: define the minimum number of links and test degradation scenarios.
    • MLAG: monitor peer link & keepalive as separate services.
    • Firewall: actively monitor PMTUD and session health; adjust hashing for stateful workloads.
    • Documentation: define runbooks, config backups, test scenarios and the RCA process.

    Conclusion

    Stable link aggregation is achieved through consistency: same port parameters, consistent VLAN and MTU design, an appropriate hashing policy and, for multi-chassis setups, a healthy peer link with a clear split-brain strategy. Deadlock-like symptoms are usually the result of multiple protective mechanisms overlapping (LACP, STP, loop protection, MLAG/keepalive). With a clear verification sequence, precise experiments (disable member, test MTU, correlate flap/MAC events) and a pragmatic fallback strategy you reduce downtime and establish the basis for resilient operation.

    Link aggregation (LACP) in operation, automation and system integration

    Beyond pure network configuration, operations determine reliability: How are LACP changes tracked, how does monitoring react and how does the network layer integrate with your digital enterprise solutions and CMDB? Often the greatest risks lie here — not because LACP fails, but because processes, telemetry and vendor quirks are not reconciled automatically.

    Key operational aspects and recommendations:

    • Detect config drift automatically: Continuously export LAG and port configs into a version repository. This allows timely rollbacks and makes changes auditable.
    • Document vendor quirks: ASIC-based hash algorithms, CPU offload, or differing default LACP timers (Cisco vs. Arista vs. Broadcom silicon) lead to inconsistent behavior. Record these deviations in the inventory and check firmware compatibility before rollouts.
    • Control-plane protection: Many LACP flaps generate high CPU load on switch controllers. Limits, rate-limiting for LACPDU and targeted alerting prevent control-plane degeneration.
    • Integration interfaces: Send LACP‑state changes into your logging/telemetry (gNMI, SNMP‑traps, syslog). Link alerts to ticketing and your inventory so that configuration changes and physical spare parts (Transceiver) are assigned automatically.

    Practical automation check: A simple SSH‑loop collects LACP partner IDs from multiple devices and exposes deviations. Adapt vendor commands to your CLI.

    Shell
    #!/usr/bin/env bash
    # hosts.txt enthält eine Zeile pro Switch
    while read host; do
      echo "== $host ==";
      ssh admin@${host} "show lacp neighbor || show etherchannel summary" 2>/dev/null | sed -n '1,120p'
    done < hosts.txt
    

    Use this output as the basis for an automated diff against the most recently persisted Config‑Snapshot. If a deviation is detected, trigger a canary test: temporarily disabling a member in a lab VLAN, automatic health check and, on success, a controlled rollout.

    Finally: Link monitoring events to your operational processes. An LACP change should not only generate an alert but automatically provide context (last Config‑commit, firmware version, assigned firewall cluster). This reduces MTTR and prevents infrastructure issues from causing disruptions to your business software and process‑centric software solutions.

    Weiterfuehrend

    Passende weitere Inhalte