Websites frequently change domains, rebrand, or expire. Automated link health sweeps purge dead external links, preserving directory crawl efficiency and user trust.
1. Link Status Code Action Protocol
- HTTP 200 OK: Verified live; updates timestamp in MongoDB.
- HTTP 301 Moved Permanently: Follows redirect; automatically updates target URL to destination canonical.
- HTTP 404 / 410 Gone: Flags listing for re-check after 72 hours; unindexes after 14 days of downtime.
- HTTP 5xx Server Error: Retries with exponential backoff before generating webmaster alert.