DNSSEC Key Rotation Guide

This document describes procedures for rotating DNSSEC keys in OpenGSLB.

Overview

DNSSEC uses cryptographic keys to sign DNS records. Key rotation is necessary for:

• Security best practices (limit key exposure time)

• Algorithm upgrades

• Compromised key response

• Compliance requirements

Key Types

OpenGSLB uses a single key signing model:

Key	Purpose	Algorithm	Recommended Rotation
ZSK (Zone Signing Key)	Signs DNS records	ECDSAP256SHA256	Annually

Note: OpenGSLB currently uses a combined KSK/ZSK model with a single key.

Pre-Rotation Checklist

• Notify stakeholders (DNS admins, security team)

• Verify access to parent zone for DS record update

• Backup current DNSSEC keys

• Schedule maintenance window

• Test procedure in staging environment

• Ensure all Overwatches are healthy

Standard Key Rotation Procedure

Step 1: Backup Current Keys

# Backup DNSSEC keys
sudo cp -r /var/lib/opengslb/dnssec /var/lib/opengslb/dnssec.backup.$(date +%Y%m%d)

# Or encrypted backup
tar -czf - /var/lib/opengslb/dnssec | \
    gpg --symmetric --cipher-algo AES256 \
    > /backup/dnssec-backup-$(date +%Y%m%d).tar.gz.gpg

Step 2: Document Current Key Tag

# Get current key tag
curl http://localhost:9090/api/v1/dnssec/status | jq '.keys[0].key_tag'

# Save for reference
OLD_KEY_TAG=$(curl -s http://localhost:9090/api/v1/dnssec/status | jq -r '.keys[0].key_tag')
echo "Old key tag: $OLD_KEY_TAG"

Step 3: Generate New Keys (Primary Overwatch)

# Stop Overwatch
sudo systemctl stop opengslb-overwatch

# Remove old keys (new keys generated on start)
sudo rm -rf /var/lib/opengslb/dnssec/*

# Start Overwatch (generates new keys)
sudo systemctl start opengslb-overwatch

# Wait for startup
sleep 10

Step 4: Verify New Keys Generated

# Get new key tag
curl http://localhost:9090/api/v1/dnssec/status | jq '.keys[0].key_tag'

# Get new DS record
curl http://localhost:9090/api/v1/dnssec/ds | jq '.ds_records[0]'

Step 5: Sync Keys to Other Overwatches

Force immediate sync:

# On each secondary Overwatch
curl -X POST http://overwatch-2:9090/api/v1/dnssec/sync
curl -X POST http://overwatch-3:9090/api/v1/dnssec/sync

Verify sync completed:

# Check all Overwatches have same key
for ow in overwatch-{1,2,3}; do
    echo "=== $ow ==="
    curl -s http://${ow}:9090/api/v1/dnssec/status | jq '.keys[0].key_tag'
done

Step 6: Update DS Record in Parent Zone

This is the critical step - update depends on your DNS infrastructure:

Option A: Contact Domain Registrar

1. Log in to registrar

2. Navigate to DNSSEC settings

3. Add new DS record (keeping old one temporarily)

4. Wait for propagation

Option B: Update Parent Zone Directly

If you control the parent zone:

# Get DS record
DS_RECORD=$(curl -s http://localhost:9090/api/v1/dnssec/ds | jq -r '.ds_records[0].ds_record')

# Add to parent zone file
# gslb.example.com. IN DS 12345 13 2 abc123...

DS Record Double-Signing Period

Keep both old and new DS records in parent zone for propagation:

; Parent zone during transition
gslb    IN  DS  OLD_KEY_TAG 13 2 old_digest...   ; Old key
gslb    IN  DS  NEW_KEY_TAG 13 2 new_digest...   ; New key

Step 7: Verify DNSSEC Chain

# Wait for propagation (can take hours)
# Then verify
dig +dnssec myapp.gslb.example.com @8.8.8.8

# Use DNSSEC debugger
# https://dnsviz.net/d/myapp.gslb.example.com/dnssec/

Step 8: Remove Old DS Record

After successful propagation (24-48 hours):

1. Remove old DS record from parent zone

2. Verify resolution still works:

   delv myapp.gslb.example.com
   

Step 9: Cleanup

# Remove old key backups after successful rotation (keep one recent backup)
find /backup -name "dnssec-backup-*.tar.gz.gpg" -mtime +30 -delete

Emergency Key Rotation (Compromised Key)

If keys are compromised, immediate rotation is required:

Step 1: Generate New Keys Immediately

# On primary Overwatch
sudo systemctl stop opengslb-overwatch
sudo rm -rf /var/lib/opengslb/dnssec/*
sudo systemctl start opengslb-overwatch

Step 2: Force Sync to All Overwatches

for ow in overwatch-{1,2,3}; do
    ssh $ow "sudo systemctl stop opengslb-overwatch && sudo rm -rf /var/lib/opengslb/dnssec/*"
done

# Start primary first
sudo systemctl start opengslb-overwatch

# Then secondaries (they'll sync from primary)
for ow in overwatch-{2,3}; do
    ssh $ow "sudo systemctl start opengslb-overwatch"
done

# Force sync
for ow in overwatch-{2,3}; do
    curl -X POST http://${ow}:9090/api/v1/dnssec/sync
done

Step 3: Update DS Record Urgently

Contact registrar/parent zone admin immediately.

Note: During DS update propagation, DNSSEC validation may fail for some resolvers. This is unavoidable in emergency rotation.

Step 4: Document Incident

• Time of compromise detection

• Actions taken

• Impact assessment

Scheduled Key Rotation

Recommended Schedule

Scenario	Rotation Frequency
Normal operations	Annually
High-security environment	Quarterly
After security incident	Immediately

Automation

Create a rotation reminder:

# Add to cron or calendar
# Annual reminder on January 1
0 9 1 1 * /usr/local/bin/notify-team.sh "DNSSEC key rotation due"

Troubleshooting

DS Record Update Not Propagating

# Check DS record at authoritative servers
dig DS gslb.example.com @ns1.parent-zone.com

# Check propagation
dig +trace DS gslb.example.com

DNSSEC Validation Failures After Rotation

Common causes:

1. DS record not updated: Update parent zone

2. Propagation delay: Wait 24-48 hours

3. Wrong DS record: Verify key tag and digest match

4. Key sync failed: Force sync between Overwatches

# Verify key tags match across Overwatches
for ow in overwatch-{1,2,3}; do
    curl -s http://${ow}:9090/api/v1/dnssec/status | jq '.keys[0].key_tag'
done

Rollback to Previous Keys

If rotation fails:

# Stop all Overwatches
for ow in overwatch-{1,2,3}; do
    ssh $ow "sudo systemctl stop opengslb-overwatch"
done

# Restore backup on primary
sudo rm -rf /var/lib/opengslb/dnssec
sudo tar -xzf /backup/dnssec-backup-YYYYMMDD.tar.gz -C /

# Start primary
sudo systemctl start opengslb-overwatch

# Start secondaries (they'll sync)
for ow in overwatch-{2,3}; do
    ssh $ow "sudo systemctl start opengslb-overwatch"
done

Monitoring

Key Age Metric

# Alert if key older than 400 days
opengslb_dnssec_key_age_seconds > 34560000

Alert Example

- alert: DNSSECKeyOld
  expr: opengslb_dnssec_key_age_seconds > 31536000  # 1 year
  for: 1d
  labels:
    severity: warning
  annotations:
    summary: "DNSSEC key is older than 1 year - consider rotation"

Related Documentation

• Security Hardening

• Backup and Restore

• DNSSEC Issues

---

Document Version: 1.0 Last Updated: December 2025