Blogs

Internet Peering, Simplified!

Today’s digital world rests on the fundamental premise of connectivity. According to Statista, global IP data traffic was at a staggering rate of 121,694 petabytes per month in 2017, further it is expected to reach 278,108 petabytes per month. This is the reality of connected global economy itself catering to a digital mobilized consumer that wants access to content and interact with each other, applications that interconnect at every layer and IoT devices that are extending our perimeter and reach.

As of today this traffic, or most of it say over 98% of internet traffic getting routed via subsea cables and the creation, consumption of content is doubling it’s footprint posing the challenge of clogging the super highways of the Internet connectivity. Today the sum total of transmission capacity of these submarine cables is over 7.1 Tbps and the level of speed has only increased with time reducing the headroom and alternate paths of our connected world. Now these cables are able to transmit 160 terabits per second via over 1.1 million kilometers of subsea cable in service globally and over 35 new subsea systems are planned for completion before 2022, this is just the beginning of the era of data explosion and IP traffic across the globe..

The reason of this data explosion is triggered by OTT ( Read, Over-The Top) content providers and consumers are the main reasons of this congestion if we can say… Today, Google has investments, or plans to invest in, no less than five undersea cables; Microsoft, four; and Facebook two; according to the telecommunications research firm TeleGeography, yet the content created locally but travels across shores and comes back to consumers. Typically, majority of the consumers if localized in a particular city or country the content may travel around the globe and comes back just after switching but not actually terminating at that end.

In some worst-case scenarios, traffic from one side of a city is brought all the way to a distant exchange point in a remote foreign country, exchanged, and then returned to another side of the same city creating what we call a hair-pining traffic that should and must really be avoided for better user experience and containing the local traffic locally. In 2013, internet traffic was five gigabytes per capita and this number reached 14 gigabytes per capita by 2018. Such an increase poses a potential capacity challenge and requires more frequent cable upgrades, and expansions of existing networks which is costly and runs into millions of dollar worth of sustenance.

The traditional transit routes are hierarchical and often end up carrying the entire traffic across multiple networks or transit point till the destination routes are available causing delays and bad user experience even though sufficient bandwidth is available. Moreover, the proven concepts of data compression and localization / Caching (Read, CDNs) might help reduce the congestion but by all means will not be sufficient unless innovative ideas are implemented quickly. One of such idea worth exploring is the Internet peering, the concept that ensures the creation and consumption of content locally.

So how does Peering Work?

Peering is a process by which multiple Internet networks connect and exchange traffic to directly hand off traffic between their customers, without having to pay transit charges to any third-party upstream ISP to carry that traffic across the Internet for them since peering with each other gives greater control over their traffic flows or allows them to better serve local populations / consumers.

Interestingly peering can happen between two entities via simple cross connect cable adding routes to each other on their routers or via an internet switch connecting multiple entities that peer with many other networks through a single port allocated for each entity on that switch, also known as multilateral peering. Typically, such exchange points are terms as An Internet Exchange Point (IXP) is where networks connect, and exchange regional traffic co-located in the same facility.

Peering agreements aren't just for the ISP-to-ISP BGP sessions, they are also used to negotiate company to company traffic flows that produce and consume large bandwidths, so called OTT, Over the top content providers streaming content / media over internet to their consumers. It further stretches for few basic services such as DNS, NTP etc as much as the CDN providers, web enterprises, communication service providers, cloud and SaaS providers participating to enrich the content brokering services.

The basic premise of peering makes sense when local content is consumed by local users and does not required to leave the specific network or geography. In broadcasting, over-the-top content (OTT) is the audio, video, and other media content delivered over the Internet without the involvement of a multiple system operator (MSO) in the control or distribution of the content, this too helps simplify the transit routes to the shortest possible peers. Broadly there exists three categories of interconnect arrangements ..

• Transit (or pay) – In this arrangement the network operator pays money (or settlement) to another network for Internet access (or transit).
• Peer (or swap) – When two networks exchange traffic between their users freely, and for mutual benefit and do not charge each other, its called peering and this is the widely accepted arrangement by IXPs.
• Customer (or sell) – A network pays another network money to be provided with Internet access, typically an upstream ISP trying to recover the International Charging Arrangements for Internet Services (ICAIS) charged by respective countries towards access points hosted within that country.

The PeeringDB database mentions more than 600 IXPs around the world (as of April 2018). IXPs are essentially large Layer 2 LANs built with one or many Ethernet switches interconnected together across one or more physical buildings. Since the Internet connects tens of thousands of autonomous systems (ASes). The ASes engage in two types of business relationships to exchange traffic: transit relationship and peering relationship. An Autonomous System (AS) is a set of Internet routable IP prefixes belonging to a

network or a collection of networks that are all managed and supervised by a single entity or organization. An AS utilizes a common routing policy controlled by the entity and is the essential fact for participating in the peering arrangements. Border Gateway Protocol (BGP) is the protocol that manages the routed peerings, prefix advertisement and routing of packets between different autonomous systems across the Internet. BGP uses the ASN to uniquely identify each system. In effect, BGP is the routing protocol for AS paths across the Internet.

It is envisaged that as a result of peering, the Internet is rapidly evolving from a hierarchical structure consisting of mostly transit links to a flat structure with mostly peering links / entities going beyond the ISPs who traditionally lead and participated on peering initiatives.. another important aspect is that peering is the voluntary and free exchange of traffic between two networks, for mutual benefit. If one or both networks believes that there is no longer a mutual benefit, they may decide to cease peering: this is known as de-peering, there are known cases and situations when de-peering might not be win-win condition for both and few networks might Push other to peer again else diverting their traffic on congested paths impacting use experience. There are examples of OTT players playing their cards on other networks to influence their terms of engagement since they bring the content that is heavily in demand, their huge investments in the sub sea cables might put more ‘muscle’ to their negotiations, must say!

Peering Technologies

Technically, Peering interconnection is a relatively complicated if same networks interconnect at multiple points across the globe, relatively simpler approach is when individual providers being to define their import and export policies with some precisions for regional peering. This allows each participating network to connect to a number of other participant’s network and then express a preference as with which network will be used to source incoming traffic and which network will be used to pass outgoing data traffic. As stated above, the protocol used to support this function is the Border Gateway Protocol (BGP), and it is the sum of these individual BGP-based pair wise interconnections defines the global flow of data across their networks and of course to the Internet traffic at large.

A sophisticated technical approach is thus required to aid the routing enforcement and fine-tuning of peering and transit agreements between the participating networks. Three techniques are commonly used to determine routing are local preference, multi exit discriminators (MEDs) and AS-Path. Local-preference is used internally within a network to differentiate classes of networks. When networks interconnect in several locations, MEDs can be used to reference that network's interior gateway protocol cost resulting in both networks sharing the burden of transporting each other's traffic on their own network.

Peering problems

When there are technologies, the glitches follow is a reality, the peering domain challenges have two aspects. First the business aspect that drives market domination, share of capacities and most important cost structures that drive hard negotiations and the underlying respective business interest. The routing costs are lower in the transit hierarchy in which a relatively small number of core ISPs interconnect with each other to provide full routing service to themselves and to non-core ISPs. Smaller core ISPs could have an incentive to merge to reduce their disadvantages in case their peering arrangements are threatened and so to increase their bargaining power. The Brokered Private Peering Group (BPPG) is born out of such need and benefits regional players to consolidate and negotiate their peering arrangements together.

Transaction costs analysis suggests that the market organization will mirror the routing hierarchy, as it does in current practice. In this hierarchy, refusals to enter into or renew peering arrangements can lead to lower routing costs and contribute to economic efficiency.

The second one is more technical where device failures / downtime, route flapping, port channel failures, black-holing can create major impact on the Internet, the latencies will increase, alternate paths might clog due to traffic failovers etc. as they say Global problems can arise from local mistakes.

Intelligent peering policy

Not all network routes are created equal: some are more expensive than others, some are purchased or leased while others are part of peering arrangements, and some provide better or worse customer experiences. The first step in cost control is to enforce peak bandwidth levels on a per-peering link basis. However, there can be significant downsides if blunt policing is applied, as all traffic may need to compete for a shared resource, regardless of traffic priority.

By carefully managing traffic on wholesale and peering links, network operators can precisely control costs and even improve the QoE ( Read, Quality of Experience) for all network users. Multiple strategies, tools and solutions are getting developed and deployed to effectively enforce policies on the traffic, such as wholesale and peering Link management solution lets the network operator define different traffic priorities so that overall traffic levels are limited to an operator-defined peak. When particular regional or global traffic levels approach that peak, the lowest priority traffic is deprioritized; traffic is limited in this manner until a defined minimum threshold for the lowest priority traffic is reached. When the lowest priority traffic has reached its minimum threshold, the next-lowest priority traffic is subsequently limited in the same manner. This way, the highest priority of traffic is only limited when all lower priorities of traffic have reached their minimum thresholds.

Network operators are able to control costs by enforcing strict peaks, maximize the value (i.e., priority) of the traffic carried over each link, and can evaluate the quality delivered by each link and peer, optimizing peering and routing arrangements to protect, and even to raise the quality of Experience via optimizing and prioritizing..

Net Neutrality and Peering..

As the Internet usage increasingly becomes relevant to everyday life, global economies, and financial transactions. Governments and regulators are becoming increasingly intolerant of any outages, disputes, or service degradations that negatively impacts the consumer, Since Internet is a global resource, regulatory interventions might have only localized impact, yet can complicate network management for global networks. Peering Networks must work together to ensure the overall quality and stability of the Internet and must have agreed set of principles for the greater good ( Aspiration!!)

A big discussion came around the “Net Neutrality” recently talking about the partial prioritization of the internet traffic, since there are two sides to this like Access networks, ISPs, Telecom providers are in one side. They demand No neutrality regulation means these networks can take discriminatory actions over what kinds of content are allowed over the “last mile”, and how that content is treated. Whereas Content creators, OTTs and the networks that perform a significant amount of the workload in distributing that content globally, fall in the second category and feel otherwise.

In reality few large incumbents ( Read, few erstwhile telecom operators of the west ) surprisingly still dominate the network traffic shaping and if the OTT provider with deep pockets agrees to align with the operator their last mile traffic is not rate limited and at times gets “Exclusive attention” in transit as well, and thus threatens the “free and open” premise of the Internet. The success of the Internet, especially “Web 2.0”, is founded on its “free and open” nature. With both sides of the coin have their objectives and expectations the key stakeholder, the consumer seems taken for granted and debate still continues..

In Summary, As a network of networks, the Internet depends on coordination on for its success. The value of access to a given subscriber depends on the size of the whole network. Thus, the compatibility of, and interconnection among networks increases the value of access to all stakeholders. The Internet exchange architecture also provides incentives for ISPs, OTTs to free-ride on their competitors’ backbones and Increased communication, cooperation, and collaboration is needed among network peering coordinators, as well as at executive levels, to avoid unwanted government intervention and regulations.