Tagged: algorithms

Bloom filters and the URL-seen problem in web-scale crawling

A primary-source walk through the URL-seen problem in large crawlers: why naive dedup fails at scale, how Bloom filters answer it, the false-positive math, and the counting, scalable, blocked, and cuckoo variants that followed.

crawling distributed-systems algorithms

Thu, March 26, 2026 · 23 min read

Consistent hashing: the algorithm behind every modern load balancer and cache

Traces consistent hashing from Karger's 1997 ring to virtual nodes, jump hash, Maglev tables, and the bounded-load variant that Vimeo shipped in HAProxy, with the minimal-remapping math that ties them together.

infrastructure algorithms distributed-systems

Tue, December 30, 2025 · 21 min read