Introduction
NKN (New Kind of Network) was founded in January 2018 by industry veterans from Google, Nokia, and Qualcomm with the goal of building the world’s largest shared network. In order to do this, NKN developed its own technology innovations that combine massively P2P architecture, intelligent routing, and blockchain distributed ledger to create a software overlay network designed for scale. While NKN has offices in Silicon Valley and Beijing, the company has a global presence through its active open-source developer community and more than 30,000 followers worldwide. Today, NKN’s network includes up to 20,000 nodes across 40+ countries and growing.
Business Imperative
With the increase in popularity of online video streaming services such as Netflix, Hulu, Disney+ and others, comes a greater demand for content delivery network (CDN) services in order to serve the billions of hours of online viewing. In addition, video streaming has become a competitive business and service providers need to closely control the cost per bit delivered. Therefore, network operators need to find a way to expand their content delivery capability, ensuring it is flexible and expandable enough to meet the demand of new subscribers. It also has to offer the performance needed to provide a good customer experience allowing streaming services the ability to manage their costs at scale, as expanded from the three attributes below. are three attributes.
Flexible and Expandable
Video Service Providers typically deploy a multi-CDN architecture that offers flexibility to add or remove additional network hosting resources to meet the demand from new subscribers. Any new solution would need to be complementary to their existing architecture while being able to expand to add new resources in the future. Video streaming demand can be highly dynamic as well, so the new solution needs to be able to scale up and down rapidly in response to customer demand.
Performance and Customer Experience
Consumers have grown used to instant gratification. A few seconds delay in starting a video can cause the video to be skipped or abandoned resulting in potential lost revenue. Therefore, any solution would need to meet or exceed the latency and bandwidth requirements needed to provide a good customer experience for video delivery.
Manage Cost at Scale
Traditional CDN solutions can be costly as these solutions require a large hardware and data centre infrastructure that must be built and maintained to support their CDN service. At the same time, CDN is often the lion’s share of the overall operational cost of video delivery for service providers. Therefore, lowering the cost of CDN service can have a significant impact on helping a streaming service stay competitive.
Operator and the Solution
Background
China Mobile is the largest mobile carrier in the world with more than 900 million mobile subscribers. With the roll-out of 5G service in 2019, the company is charging ahead with its 5G enterprise strategy to offer cloud services, leveraging its 5G network, data centres, and edge compute nodes. With this initiative, China Mobile was interested in exploring new innovative approaches to using edge compute resources for enterprise customers.
At the same time China Mobile was rolling out its 5G service, NKN began engaging with one of the largest Video Service Providers in China that have been experiencing phenomenal customer growth and looking for new ways to meet the growing demand. The Video Service Provider quickly realised that NKN’s decentralised shared network was the perfect fit for its CDN software and their scalability needs with benefits that include:
- NKN meeting the scalability needs – thanks to its horizontally scalable architecture and convenient self-service deployment, the NKN Network can grow to millions of nodes to meet future requirements.
- Flexibility and expandability in deploying network resources for their CDN – Nodes can join or leave the network according to periodic demand dynamics.
- Good video experience – NKN incorporates optimal routing algorithm that ensures latency and bandwidth requirements are met.
- Manage CDN cost at scale for Video Service Providers – The NKN network is comprised of a crowd-sourced pool of network resources that tap into the unused and under-utilised market for the most cost-effective options, reducing overall cost, and unlocking more bandwidth for CDN services without installing new hardware or using additional power which in essence benefits the environment.
NKN was also looking for a reliable infrastructure partner that could help the network scale to meet the needs of video service providers and other enterprise customers, and as a result, partnered with China Mobile. China Mobile has more than 31 provincial data centres in China with additional nodes abroad as well as access to more than 60,000 cloud server resources. At the same time, the collaboration provides China Mobile with healthy new sources of revenue from NKN’s CDN services.
The Solution
In order to provide video service providers with a CDN as a Service offering (CDNaaS), NKN combined its own distributed micro hosting network software together with CDN caching capabilities all running on idle resources available on China Mobile’s edge cloud. The resulting product called New Kind of CDN or nCDN offers the scalability, bandwidth performance, and cost management capabilities that fit the needs of today’s video service provider customers.
NKN’s micro hosting capabilities were developed on top of its own open-source peer to peer network architecture. This architecture offers the ability to scale to millions of nodes and has already seen a growth of up to 20,000 nodes globally. Integrated with its micro hosting is network management for deploying and managing services such as CDN running on the network. The nodes in this architecture also have a built-in payment channel with the ability to reward network operators in real-time. All together, NKN software offers the micro hosting platform for CDN caching.
For CDN caching software, NKN not only has its own solution but can also host third party caching applications in order to integrate into any CDN expansion or multi-CDN configuration. This offers great flexibility for the video service provider while at the same time removing complexity for the operator in supporting multiple caching solutions.
The solution uses the idle resources available on China Mobile’s edge cloud infrastructure. By doing so, China Mobile is maximising the efficiency of the network while benefiting for the additional revenue streams. This is further supported by the flexibility in nCDN software, which allows nodes to join or leave the network in order to manage available resources as well as meet customer demand.
NKN together with China Mobile and one of the largest Video Service Providers in China successfully completed a commercial trial for nCDN in the second half of 2019. The commercial trial included the delivery of video content from the Video Service Providers catalogue of TV shows and movies to a portion of its nearly 100 million subscribers. In doing so, the trial met or exceeded all of the technical challenges of delivery quality CDN service in a demanding large scale video streaming environment. Figure 1 below shows the network setup for the commercial trial and the interactions between China Mobile’s edge infrastructure, NKN’s node/content delivery software and the video service providers media store in order to serve content subscribers.
Figure 1: Commercial Trial Network Architecture
Economic Benefits
Incremental Revenue
NKN’s CDN as a Service (CDNaaS) solution offers economic benefits for both for the enterprise customer and for the mobile operator.
Since nCDN is purely a SaaS model with pay-as-you-go pricing, there is no CAPEX cost for the enterprise customer or the mobile operator. The enterprise can take advantage of the lower barrier to entry and scale faster without any large network expenses. For Mobile Operators, NKN’s solution runs on existing edge servers and connectivity and offers incremental revenue to operators hosting the service.
Opex Benefits
There are also additional OpEx benefits to mobile operators when offering NKN’s solution to enterprise customers. These benefits include both mobile backhaul savings as well as customer experience.
There are also additional OpEx benefits to mobile operators when offering NKN’s solution to enterprise customers. This is due to the fact that nCDN uses caching to store content such as videos or movies at the edge of the network. According to the article Mobile Backhaul: An Overview by GSMA, “Caching reduces the load on mobile backhaul and enhances the customer experience by storing frequently accessed contents in the edge network.” These same benefits apply to nCDN and will quantify these improvements below.
In relation to the Mobile Backhaul Saving, contents are downloaded via mobile backhaul and then cached at an edge server as part of a content delivery solution such as nCDN. The content can be served to many subscribers without the need to access the original content again over backhaul, meaning it’s possible to download the content once and serve it to many customers. During tests carried out by NKN, it was identified that popular contents, resulted in 1:10 savings. Meaning for every 1MB we cached, NKN can serve 10MB of data on average, resulting in a 90% backhaul savings as compared to not having access to the cached content. However, for non-popular content that ratio could be as low as 1:1 meaning, there were no backhaul savings related to caching. For cases of non-popular content, though, there are caching benefits related to customer experience to consider.
By offering this content at the edge of the network rather than concentrated in data centres that could be hundreds or even thousands of miles away, customers can achieve lower latency, faster response time, and overall better customer experience. According to the article ‘Find out how you stack up to new industry benchmarks for mobile page speed’ from Google; as page loads increase from 1 second to 3 seconds the probability that someone abandons the page increases by 32% which directly correlates to lost views and subsequent lost revenue. NKN carried out further tests with NKN’s very own nCDN software, which ran on China Mobile’s network. The test resulted in over 40% faster page-loading than NKN’s nearest competition in the China region bringing about increased customer experience. Figure 2 below compares page load times across China using nCDN with China Mobile Infrastructure compared with the competition.
Figure 2. nCDN + China Mobile vs Competition Page Load Times China Region
Implementing the Solution
nCDN is deployed and managed by NKN making it easy for enterprises to implement. NKN offers a self-service portal that allows enterprises to sign up and track their content delivery usage. The enterprise customer only needs to update their DNS record to direct traffic to NKN’s DNS for delivery and the enterprise will be ready to go.
Deploying nCDN in an operator network is a two-step process. The first step is to launch the NKN nodes that connect and join the network. The second is to deploy CDN application software to those nodes to deliver service.
An NKN node can run on most Linux based machines and can be deployed on China Mobile’s edge cloud, as well as bare metal, VM, or as a docker container. The software is installed via a rapid deployment process with a single line script that will download and install NKN’s node software in a few minutes. Once this is completed, the nCDN application software will need to be deployed using a subscription-based model. NKN’s nodes must first subscribe to a group using a Docker Swarm, all nodes are then subscribed to the group that received the CDN software downloaded before installation can take place. While the subscription process will only take a few minutes, it may take up to a few hours for all nodes to receive the CDN software, install, and configure. This is still significantly faster than traditional “Bring Your Own Server” type of CDN deployment, which could take weeks if not months of project work.
Challenges / Lessons Learned
As per our deployment, there are several lessons that can be learned and shared, especially with the commercial trial that took place with China’s leading Video Service Provider and China Mobile, some of the challenges faced in this trial are:
Heterogeneous hardware, software, networking and configurations
- In Phase 1 of the commercial trial, there was a plan to use a more diverse collection of infrastructure services in order to host NKN’s software. It was learned early in the process that it would be difficult to manage multiple configurations and still be able to meet the trial goals in the allotted time. Instead, a decision was made to focus on a primary infra platform by China Mobile as the basis of our deployment. The result allowed NKN to quickly iterate on China Mobile’s platform which provided a reliable foundation to successfully complete the commercial trial.
Deployment at scale
- In order to deploy 100s of nodes for testing, a trusted tool was needed that would help orchestrate and deploy multiple instances across many host machines. While the ability to execute a rapid deployment script to bring up new CDN nodes was available, it was discovered that a containerised mechanism such as docker container was a better choice at scale. For this purpose, it was decided to set up and integrate the development operations tool, docker swarm. In doing so, NKN was able to deploy and manage CDN nodes at scale to successfully support the commercial traffic demands during the trial testing.
Conclusion
This case study introduces a new model in providing carrier-grade content delivery services for Video Service Providers. Using China Mobile’s edge compute infrastructure together with NKN’s distributed micro hosting software, the solution was able to meet all of the technical hurdles in delivery commercial content delivery services for one of the largest Video Service Providers in China. This solution also showcases the ability for Mobile Operators to create new revenue streams by offering edge infrastructure and idle resources to a new breed of edge applications that rival or even better the performance of large centralised data centre solutions. As 5G and Edge Compute infrastructure continue to be deployed and video is increasingly consumed by mobile, Video Service Providers should consider moving toward a more distributed edge content delivery solution for performance and lower cost.