Skip to content

HA: Corrupted config files with null bytes on slave failover #1771

Description

@gsanchietti

Problem

Recurring issue during HA failover: slave node corruption when rsync syncs from primary.

Config files become corrupted with null bytes, breaking services. Examples:

  • /etc/config/firewall → fw4 fails, VRRP reports "Operation not permitted"
  • /etc/nginx/mime.types → nginx won't start, shows binary data
  • Files in rsync temp dir: /usr/share/keepalived/rsync/etc/nginx/mime.types

Symptoms

  • Slave behaves incorrectly: activates interfaces incorrectly
  • VRRP can't communicate with master (packets visible in tcpdump, but kernel blocks)
  • Secondary filesystem reports improper unmount: "FAT-fs: Volume was not properly unmounted"

Root Cause

Overlayfs/tmpfs corruption during concurrent rsync writes. File system secondary didn't unmount cleanly.

Proposed Fix

  • Reduce rsync update frequency from current to 5-10 minute intervals
  • Move temporary sync files from disk to tmpfs
  • Investigate overlayfs mount state during failover

Metadata

Metadata

Assignees

No one assigned

    Labels

    verifiedAll test cases were verified successfully

    Type

    Projects

    Status
    Done ✅

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions