Skip to content

The history of CDNs: from Akamai's 1998 MIT spinout to the edge era

· 20 min read
Copyright: MIT
history cdn infrastructure
Wordmark reading CDN with a map of distributed edge points and one orange node

In February 1999, Victoria’s Secret bought a Super Bowl ad to promote a live webcast of its fashion show. The ad worked too well. Somewhere north of a million people hit the site in the first few minutes, the Broadcast.com servers behind it folded, and most viewers got a stuttering mess of buffering spinners instead of a runway. The retailer got more press from crashing the internet than it would have got from a clean stream. The engineers got a lesson the whole industry was about to learn: a single origin server, or even a rack of them in one building, cannot absorb a crowd that arrives all at once.

That problem already had a name in the research literature. A sudden, correlated surge of requests for the same content is a flash crowd, and in the late 1990s it was killing popular sites on a regular basis. Slashdot links would melt a small server in minutes. A breaking news story would take down a newspaper. The web had no mechanism to spread a hot object across many machines and many networks fast enough to matter. The fix that emerged is the content delivery network, and the company that turned the fix into an industry was spun out of an MIT math department.

This post follows that arc. It starts with the algorithm that made distributed caching tractable, walks through Akamai’s 1998 founding and the events that proved the model, covers the competitors that turned delivery into a commodity, looks at how Amazon and then Cloudflare changed the pricing, and ends with the shift from caching bytes to running code at the edge. The through-line is a single tension: every generation of CDN solved the previous generation’s bottleneck and exposed the next one.

1997: the algorithm before the company

The technical idea came first. In 1997 a group at MIT published Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web at the ACM Symposium on Theory of Computing. The authors were David Karger, Eric Lehman, Tom Leighton, Rina Panigrahy, Matthew Levine, and Danny Lewin. The paper’s title names the exact problem the industry would spend the next decade attacking: hot spots on the web.

The core move is consistent hashing. Imagine a hash table that has to grow and shrink as servers come and go. In a naive scheme, hashing a key with key mod N servers means that changing N remaps almost every key. That is fine for an in-memory table you rebuild on a whim. It is a disaster for a cache spread across hundreds of machines, because remapping a key means a cache miss, and a wave of simultaneous cache misses is the flash crowd you were trying to prevent. Consistent hashing maps both objects and servers onto the same circular keyspace and assigns each object to the next server going clockwise. When a server joins or leaves, only the keys in its arc move. The expected fraction remapped is roughly the inverse of the server count, not the whole table.

S1 S2 S3 S4 Object hashes to a point on the ring, then walks clockwise to the next server. Remove S2 and only its arc of keys moves to S3. The rest of the table is untouched. *Consistent hashing places servers and objects on one ring so that adding or removing a server only disturbs a small slice of the cache.*

That property is what makes a large, churning cache cluster behave well under load. Tim Berners-Lee, who had been warning his MIT colleagues about coming congestion since the mid-1990s, later credited the consistent hashing work, and Lewin specifically, with solving the slashdotting problem that plagued the early web. The algorithm has since spread far beyond CDNs. It sits inside distributed databases, key-value stores, and load balancers, and it is worth its own treatment, which the companion piece on consistent hashing gives it. For the CDN story, the relevant fact is that an academic caching protocol had a direct commercial application, and the people who wrote the protocol went and built the company.

1998: Akamai incorporates

The path from paper to company ran through a business plan contest. In 1997 Leighton and Lewin entered the MIT $50K Entrepreneurship Competition, run by the Sloan School. Their entry was one of six finalists out of roughly a hundred. It did not win. It attracted enough venture interest to start a company anyway, and Akamai was incorporated on August 20, 1998. The founding group was Leighton and Lewin plus three Sloan-connected people: Preetish Nijhawan, Jonathan Seelig, and Randall Kaplan. George Conrades came in as chairman and CEO in April 1999. The name “Akamai” is Hawaiian for clever or cool.

The product idea was deceptively simple to state. Instead of asking users to fetch a popular file from one origin server, Akamai placed copies of that file on thousands of servers scattered across many networks, then used DNS to send each user to a nearby copy. The clever part is the steering. When a browser resolves a hostname that Akamai serves, Akamai’s DNS does not hand back a fixed address. It picks an edge server based on where the user appears to be, how loaded each candidate is, and the state of the network paths between them. Consistent hashing decides which server within a cluster holds which object, so that a cluster can lose or gain machines without a thundering herd of cache misses. A separate mapping system, which Akamai has described as using a stable-marriage style assignment, balances load across clusters. The exact production layout has changed many times and much of it has never been published in detail; what is public is the shape, not every field.

Origin Edge cluster A Edge cluster B Edge cluster C User fill on miss nearest, least loaded DNS picked cluster B *The steering decision happens at DNS resolution time: the user is mapped to a healthy nearby cluster, which serves from cache and only contacts the origin on a miss.*

The model got proved fast, and in public. Akamai delivered its first live traffic in February 1999, a single Disney pixel as a test. The next month it carried the ESPN site through the March Madness college basketball tournament, and that was the moment the technology stopped being a thesis. Reports from the period describe Akamai absorbing thousands of requests per second and keeping ESPN responsive on a day that would otherwise have buried it. Yahoo became a charter customer in April 1999. The pattern that the Victoria’s Secret webcast had exposed as a failure, Akamai had turned into a service you could buy.

For the deeper mechanics of how anycast, points of presence, and the cache hierarchy actually move a byte from origin to browser, the companion post on how a CDN works goes layer by layer. The history here cares about why those mechanics got built and who built them.

The Lewin loss

Danny Lewin was Akamai’s CTO and, by every account, its technical center of gravity. He was born in Denver in 1970, grew up partly in Israel, served four years in the Israel Defense Forces, and came back to MIT for a PhD in 1996. He was 31 when he was killed on American Airlines Flight 11 on September 11, 2001, seated near the hijackers. He is widely identified as the first person to die in the attacks. The square at Main and Vassar in Cambridge, near MIT, was later renamed Danny Lewin Square.

The loss landed at the worst possible moment for the company. Akamai’s stock had ridden the dot-com bubble up and was riding it down, the post-9/11 economy gutted ad and media budgets, and the man who understood the system best was gone. The company survived, kept its patents, kept its customer base, and by the 2010s was carrying a double-digit percentage of all web traffic on a normal day. That it came through at all is part of why the CDN became permanent infrastructure rather than a bubble-era curiosity.

2001 to 2008: the commodity years

Akamai had the patents and the head start, but a CDN is, at bottom, servers in data centers plus software to steer traffic, and that is a thing competitors can build. The 2000s were the decade when delivery turned into a commodity.

Speedera launched in 1999 as an aggressive price competitor and fought Akamai hard enough that the two ended up in years of litigation; Akamai acquired Speedera in 2005, which removed one rival and added patents. Digital Island, Mirror Image, and a handful of others came and went. The competitor that mattered most for the media business was Limelight Networks, founded in 2001 and built specifically for companies pushing large volumes of video and software. Limelight carried marquee events through the decade and became the natural second source that big media buyers wanted so they were not captive to Akamai’s pricing.

EdgeCast arrived in 2006, founded in Los Angeles by Alex Kazerani, James Segil, and others, with backing from Disney’s Steamboat Ventures. Its pitch was a cleaner, more developer-friendly platform with real-time configuration and analytics at a time when Akamai’s product felt heavy and enterprise-sales-driven. EdgeCast is the thread to pull on later, because its corporate life ran through Verizon and then back out again, and its ending is the cautionary tale of the whole CDN commodity business.

What every one of these companies was selling, underneath the branding, was the same thing: cache a customer’s bytes on our distributed servers, steer users to the nearest healthy copy, charge by the gigabyte delivered or by committed capacity. Margins came from utilization and from network peering deals. The technology differentiators were real but narrow, things like faster cache purges, better video-specific handling, tighter origin-shielding. None of them changed the basic unit of sale. Bandwidth got cheaper every year, so the price per delivered gigabyte fell every year, and the only way to grow revenue was to move more bytes or to sell something other than bytes. That squeeze is the backdrop for everything that came next.

2008: Amazon changes the billing

Amazon launched CloudFront in November 2008 as part of AWS, and the important thing about it was not the technology. It was the invoice. Traditional CDNs sold through enterprise contracts: you talked to a salesperson, committed to a volume, signed a term, and got onboarded by hand. CloudFront let you turn on a distribution from a web console, pay per gigabyte and per request with no commitment, and tear it down when you were done. It plugged directly into S3 and the rest of AWS, so for anyone already on Amazon’s stack it was the path of least resistance.

This did to CDN procurement what AWS had already done to servers. It removed the salesperson from the small and medium customer. A developer with a credit card could put a global cache in front of an app in an afternoon. The incumbents still had advantages at the high end, with more points of presence, deeper peering, and account teams who understood a media company’s traffic. But the floor of the market, the long tail of sites that would never have negotiated an Akamai contract, now had a self-serve option. CloudFront did not have to win the enterprise to reshape the business. It just had to make the entry-level case trivial, and it did.

The deeper effect was on expectations. Once one major provider offered metered, self-serve, no-commitment delivery, every provider eventually had to. The contract-and-onboarding model did not vanish at the top of the market, but it stopped being the only way in. And the same cloud platforms that commoditized delivery were starting to think about what else lived in those data centers near the user, which is the seed of the edge-compute story.

2009: Cloudflare makes it free

If CloudFront removed the salesperson, Cloudflare removed the bill entirely for the bottom of the market. The company was founded in 2009 by Matthew Prince, Lee Holloway, and Michelle Zatlyn, growing out of Project Honey Pot, an earlier spam-tracking effort Prince and Holloway had run. It launched publicly at TechCrunch Disrupt in September 2010. The offer was the thing that got attention: a free plan that put your site behind Cloudflare’s network, gave you CDN caching and DDoS mitigation, and later a web application firewall, at no cost, for anyone with a domain.

1998 Akamai contracts 2008 CloudFront pay per GB 2009 Cloudflare free tier 2017 Workers edge code 2019 Compute @Edge Each step changed the unit of sale, not just the price. *The business model moved from negotiated contracts to metered self-serve to a free tier, and then sideways into selling compute rather than bytes.*

The economics only work because of how a CDN’s costs scale. The expensive parts, the network buildout, the peering relationships, the points of presence, are mostly fixed. The marginal cost of caching one more small site is close to nothing, especially when that site’s traffic is small and cacheable. Cloudflare could give the free tier away because the free users cost little, generated enormous reach and threat intelligence, and converted some fraction to paid plans. By the time it went public in 2019, it was sitting in front of a large share of the web. The free tier had turned what was once an enterprise-only service into something a hobbyist would switch on without thinking.

That reach is exactly what made Cloudflare a security company as much as a delivery company. Sitting in front of millions of sites means seeing the attacks against all of them, and that vantage point fed the bot-management and WAF products that became a large part of the revenue. The fuller arc of that company, from free CDN to security platform, is in the dedicated history of Cloudflare, and the threat side shows up across the history of the WAF and the history of bot mitigation. For this story, the point is narrower: Cloudflare proved that the delivery layer could be a loss leader, and that whoever controlled it controlled a position with value far beyond moving bytes.

2011 to 2019: programmable edges

The next move was visible from a distance. If you already run servers in hundreds of locations near users, and the byte-delivery business is a race to the bottom on price, the obvious thing to sell is computation in those same locations. Two companies pushed this hard from different starting points.

Fastly, founded in 2011 by Artur Bergman, came at it from the configuration side. Bergman had been CTO at Wikia and had wanted his wikis fast worldwide; when existing CDNs could not purge their caches fast enough for a wiki’s edit-heavy traffic, he built his own. Fastly’s network ran on Varnish, and its differentiator was letting customers write real logic at the edge in VCL, the Varnish Configuration Language, with cache invalidation measured in tens of milliseconds rather than minutes. That made Fastly the CDN of choice for sites that treated the edge as part of their application rather than a dumb cache. It went public in May 2019.

Cloudflare came at it from the cache side. By 2017 it had spent years bolting features onto a fixed-function HTTP proxy, and it concluded that it could never anticipate every customer need that way. The answer, announced on September 29, 2017, was Cloudflare Workers: make the edge programmable and let customers write the logic themselves. The technical choice that made it work is the isolate. Rather than give each customer a container or a virtual machine in every location, which would be far too heavy to run in hundreds of points of presence, Workers runs customer JavaScript in V8 isolates, the same lightweight sandboxing mechanism a browser uses to keep tabs apart. Many isolates share one process, spin up in single-digit milliseconds, and carry almost none of the per-tenant overhead a container does.

Container per tenant OS + runtime tenant A OS + runtime tenant B heavy, slow cold start Isolates in one process A B C D light, millisecond start One V8 process holds many tenants. That is what lets edge compute run in hundreds of locations without a VM per customer per site. *The isolate model is the trick that makes per-request compute economical at edge scale: many tenants share a process instead of each getting a container.*

The cold-start gap is the number that sells the model. Cloudflare has put Workers cold starts under five milliseconds and compared that to Lambda cold starts that could run into seconds, and later pushed the figure toward zero by warming isolates ahead of the request. Fastly went a different route and reached for WebAssembly, building the Lucet compiler from 2017 and shipping Compute@Edge so customers could run code compiled from Rust and other languages with startup times it measured in tens of microseconds. Amazon answered with Lambda@Edge, running functions in its CloudFront locations, though with the heavier per-invocation model that the isolate approach was designed to avoid. The detailed comparison of these platforms lives in edge compute compared; the historical point is that by the end of the 2010s the CDN had stopped being only a cache. It was a place to run your code.

EdgeCast’s long ending

The fate of EdgeCast is the commodity-business cautionary tale, and it took fifteen years to play out. Verizon acquired EdgeCast in December 2013 in a deal reported at more than $350 million, folding it into Verizon Digital Media Services. The asset then rode every twist of Verizon’s media strategy. It passed through the Oath umbrella, got rebranded as Verizon Media, and after Apollo Global Management bought most of Verizon Media in 2021, the CDN piece was spun back out under the EdgeCast name again.

Then the two threads converged. In June 2022 Limelight Networks acquired EdgeCast for around $300 million and rebranded the combined company as Edgio. Two of the more established names in delivery, one from 2001 and one from 2006, became one company. It did not save either. Edgio filed for Chapter 11 in late 2024 and shut down operations in January 2025, ending what one industry chronicler counted as roughly 23 years of continuous CDN history between the two brands, and made it something like the 25th CDN vendor to fold over the previous three decades. Parler Cloud Technologies later picked up some of the EdgeCast technology assets.

The lesson in that arc is the one the whole post keeps circling. Selling delivery by the gigabyte is a business with falling prices and rising fixed costs, and a pure-play CDN that does not move up the stack into security or compute gets caught between hyperscalers that bundle delivery into a broader cloud and a free-tier player that gives the entry level away. EdgeCast and Limelight had good technology and real customers and still could not escape the squeeze. The CDNs that thrived were the ones that stopped selling only bytes.

How the CDN reshaped everything around it

A CDN is not an island. Putting a cache and a proxy in front of most of the web changed the protocols and the operational practices on both sides of it. The steering layer depends on DNS, and the demands CDNs placed on DNS, fast geo-aware answers, health-based responses, low TTLs, pushed the whole history of DNS toward the programmable, latency-sensitive resolvers we have now. The reachability of a single anycast address from everywhere depends on the routing layer, which is why CDN operations and the history of BGP are so tangled; a CDN lives or dies on how cleanly its prefixes propagate and how fast it can shift traffic between locations.

Sitting in the request path also made the CDN the natural home for defense. A network that already terminates TLS for millions of sites and sees traffic to all of them is the obvious place to absorb a volumetric DDoS, run a WAF, and score bots. That position is why the same companies in this post show up in the security stories, from Akamai’s bot management to Cloudflare’s, and why a crawler today spends as much effort on the CDN’s defenses as on the origin behind them. The reverse-proxy posture a CDN adopts is the same one analyzed in the proxy taxonomy, just operated at planetary scale.

What the arc actually shows

The CDN began as a fix for one narrow failure mode: the flash crowd that flattened a single origin. Consistent hashing made distributed caching stable enough to sell, Akamai turned it into a service, and a public sporting event proved it inside the first year. Everything after that is the story of an industry working out that the fix it sold was easy to copy, and that the only durable position was to keep changing what was being sold. Delivery by contract gave way to delivery by the gigabyte, then to delivery for free, then to selling compute and security in the same boxes that did the delivering.

The companies that read the arc correctly are the ones still standing. Akamai survived the loss of its CTO and the dot-com crash and moved into security. Cloudflare gave the cache away and built a platform on top of it. Fastly bet on programmability early. The ones that kept selling bytes by the gigabyte, EdgeCast and Limelight among them, ran a good business into a wall that bandwidth pricing had been building toward for two decades. The edge that started as a place to stash a copy of a JPEG is now a place to run a function, terminate a TLS handshake, score a request for fraud, and decide whether a visitor is a person. Danny Lewin’s ring of servers turned out to be the most valuable real estate on the internet, and the fight over what runs on it is nowhere near settled.

Sources & further reading

Further reading