Asynchronous Multicast

Multicast is widely regarded as an essential part of live (linear) content delivery, but there is an intriguing alternative. The case for multicast is straightforward—content delivered down a multicast tree traverses internal network links only once. But a second “content distribution tree” is rapidly becoming available in operator networks—the caching hierarchy at the core of the operator’s CDN. Instead of pushing content down a multicast tree to end-users, those same end-users pull content down the caching hierarchy, filling the caches along the way so that subsequent requests can be serviced from caches near the edge of the hierarchy rather than near the top of the hierarchy. Like a multicast tree, a caching hierarchy delivers each piece of content only once across each internal link, but since the content remains in the cache, it is also available to satisfy subsequent user requests. In other words, we can think of a caching hierarchy as supporting a form of asynchronous multicast.

The caching hierarchies offered by CDNs are widely understood to work well for video-on-demand applications, but not necessarily for linear applications, such as emerging “TV Everywhere” offerings. That situation is changing. The key technical advance is low-latency cut-through proxy support, so that the delay pulling content through a caching node is significantly lower than a typical store-and-forward proxy. In the case of caching, this delay is essentially the ingest latency—how long it takes to ingest content from upstream so that it is available to serve downstream. Low-latency cut-through is not only possible (we have demonstrated sub-10ms latencies with the Verivue HyperCache), but also a reality in Verivue deployed CDN solutions for live delivery using HTTP adaptive streaming.

There are several keys to achieving such performance. The first is obvious, but bears mentioning because so many CDNs running today are based on caches that were originally web servers rather than reverse proxies: don’t store the ingested content to disk before delivering it to the downstream users. A Store-to-Disk-and-Forward approach is a non-starter. The second is to begin delivering bytes downstream before the entire object is received into the node. This must happen at the granularity of packets rather than whole objects. Third, a viable approach to low-latency cut-through proxy has to efficiently support many concurrent user requests waiting for the object to be delivered from an upstream cache. Serializing requests introduces unacceptable latency into the cut-through mechanism and is particularly problematic in the delivery of live content where clients are naturally synchronized to request the same content objects.

A caching hierarchy that is as efficient delivering live content as a multicast tree gives operators a powerful new tool. Operators can deliver both live and VoD content on the same caching substrate, and as a bonus, the live content isn’t limited to being strictly live. Because content is cached throughout the hierarchy, it is readily available for time-shifting (e.g., catch-up TV). In other words, the distinction between live, live-with-catch-up and VoD is blurred. (The distinction is blurred at the CDN level, independent of whether the service provider continues to offer multiple services to customers.) Moreover, at the live end of the spectrum, the latency is so low that concurrent cut-through makes Internet-based CDNs a viable option for even the “first screen” (the home television), which has fast-channel-change expectations.

That a single technology can seamlessly serve such a range of requirements is an important technical breakthrough, but the frosting on the cake is that it does so without introducing additional complexity, but rather, by using a simpler mechanism. Instead of the complexity of multicast tree maintenance—managing user joins and departures across the set of multicast channels carrying different bit-rate streams—the entire flow of content (at different bit-rates) comes for free as end-users issue HTTP GET requests for the content they want to see. Content is pulled down the caching hierarchy, on-demand, in response to those requests. From an operational perspective, the choice between multicast tree maintenance and leveraging the RESTful HTTP protocol is an easy one.