28 May 2021 Effective IXP strategies for the Asia-Pacific

28 May 2021

Effective IXP strategies for the Asia-PacificA Comparative Case Study Report

CC BY-NC-SA 4.0

Table of ContentsAcknowledgment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Study objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

IGF BCP Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Asia-Pacific IXP Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2017: Myanmar Internet Exchange (MMIX), Myanmar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2015: Bangkok Neutral Internet Exchange (BKNIX), Thailand . . . . . . . . . . . . . . . . . . . . . . 10

2010: Singapore Internet Exchange (SGIX), Singapore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2004: Bangladesh Internet Exchange (BDIX), Bangladesh . . . . . . . . . . . . . . . . . . . . . . . . . 15

2003: National Internet Exchange of India (NIXI), India . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2001: Japan Network Access Point (JPNAP), Japan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2000: Korea Internet Neutral Exchange (KINX), Republic of Korea. . . . . . . . . . . . . . . . . 24

1997: Internet Exchange Australia (IX-Australia), Australia . . . . . . . . . . . . . . . . . . . . . . . . . 26

1995: Hong Kong Internet Exchange (HKIX), Hong Kong . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Other Region IXP Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2002: Internet Exchange Brazil (IX.br), Brazil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2000: Kenya Internet Exchange Point (KIXP), Kenya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1996: Equinix, United States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

1995: Deutscher Commercial Internet Exchange (DE-CIX), Germany . . . . . . . . . . . . . . . 37

Regional Ecosystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Key Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Reflections on best practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

The growing importance of the edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Starting or Scaling an IXP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Fostering a supportive ecosystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

IXP Case Study Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Effective IXP strategies for the Asia-Pacific 2

CC BY-NC-SA 4.0internetsociety.org

@internetsociety

Acknowledgment The Internet Society extends its deepest gratitude to the Asia-Pacific Internet Exchange Association (APIX) for its invaluable help and assistance.

AbstractInternet exchange points (IXPs1) are a key component of any efficient Internet ecosystem. Anywhere you find an open, competitive telecommunications market, you will find room for an Internet exchange – at least one.

Internet exchange points are multiplying, each adopting one of a range of business models, each serving different market segments and needs.

At a time when the oldest exchanges are well into their third decade, the number of operators, locations, and sites continues to grow, as does the number of participating peers and the volume of traffic exchanged.

The growth is uneven, however. Some Internet exchange point operators grow quickly, while others grow at a steadier pace. There are exchanges that, having reached the limit of their environment, are content with their role in the local ecosystem or fulfill their niche appropriately.

Some IXP operators are eagerly reaching out. As successful exchanges grow, they expand to new cities, adding multiple sites at each new location. They progressively expand from their initial site, usually a national capital, to create new nodes in second-tier metro areas.

Others spread themselves more widely, with locations and peering points across the globe, franchising their brand and operational expertise in emerging and mature markets.

Study ObjectivesThis report is inspired by a desire to better understand how IXPs are evolving across the Asia-Pacific. How do these vastly different exchanges fit into their local environment? Where are they headed? How do they compare with IXPs in other regions?

The result is a series of case studies, conducted to inform a better understanding of the state of peering in the Asia-Pacific region.

To provide additional context, one IXP from each of the four other regions is profiled in a similar way to provide points of comparison.

1 An Internet exchange point (IXP) is a physical and usually neutral location where different IP networks meet to exchange local traffic via a switch.



@internetsociety

Internet Governance Forum Best Current Practice Framework“Generally speaking, the success of an IXP should be measured by its ability to sustainably contribute to the development of the Internet ecosystem within its community. Too often, the performance of an IXP is only assessed by looking at the volume of traffic passing through the exchange, which is not an accurate indicator of success.” 2

In 2015 and 2016, the multistakeholder Internet Governance Forum (IGF) developed a pair of documents that set out to describe a Best Current Practice (BCP) for Internet exchange points.

This report uses the viewpoints from that multistakeholder document as a reference framework to interpret and understand the thirteen IXPs profiled in this report.

Reflecting on the themes and perspectives presented in the IGF provides us opportunities to learn more effective IXP strategies.

Following the 2015 IGF, the Best Practice Forum on Internet Exchange Points (IXPs) released an output document, “Enabling Environments to Establish Successful IXPs,” which described the purpose of Internet exchange points and how to create an enabling environment for them to succeed.3

At the 2016 Internet Governance Forum (IGF) the Forum developed a final output document, “Contributing to the success and continued development of Internet exchange points (2016).”4

The 2016 output document continued the work of 2015 in order “to serve as a resource for policymakers, regulators, governments, and decision-makers in the private sector.”

It recommends learning from the success of other IXPs and provides numerous case-study examples (some of which are also covered in this report). These describe successful strategies and positive environments that have been shown to support Internet exchange growth.

2 IGF 2016, Best Practice Forum on IXPs, “Contributing to the success and continued development of Internet exchange points,” http://www.intgovforum.org/multilingual/index.php?q=filedepot_download/3408/442

3 Internet Governance Forum (IGF) 2015, Best Practices Forum on Internet exchange points (IXPs), Enabling Environments to Establish Successful IXPs, https://www.intgovforum.org/cms/documents/best-practice-forums/creating-an-enabling-environment-for-the-development-of-local-content/582-igf-2015-bpf-ixps/file


IXP Name Launch Date

MMIX 2017

BKNIX 2015

SGIX 2010

BDIX 2004

NIXI 2003

IX.br 2002

JPNAP 2001

KINX 2000

KIXP 2000

IX Australia 1997

Equinix 1996

HKIX 1995

DE-CIX 1995



@internetsociety

The 2016 IGF best-practice document describes how the following characteristics are often found in well-functioning IXPs:

• Local IP transport capacity must be available for a reasonable price to allow organizations to connect to the exchange.

• Local content: Either locally created content or international content cached or hosted locally represents a large percentage of traffic and will make the exchange attractive to participants. Having access, via the IXP, to the content of important global and local content providers and content delivery networks can increase the value of the IXP membership for individual networks.

• Peering policies and technical facilities should allow bilateral and multilateral peering, but not enforce multilateral peering, as it “can hamper the growth of an exchange.”

• The institutional and governance model of an IXP can greatly influence its development trajectory. Rather than recommend any particular ownership or business model, the BCP makes clear that a model must work well given the local circumstances.5

The IGF BCP document also recognizes that successful exchanges often support the community with events or training and assist with the spread of technical knowledge to the general benefit of the community.

Collaborative studyThis report includes case studies for nine Asia-Pacific Internet exchange operators, all of which are members of the Asia-Pacific Internet Exchange Association (APIX)6. They were chosen to provide a cross section of business models seen in the region.

With assistance from APIX, each operator was invited to participate in a face-to-face interview. Seven accepted, and two others provided written answers to the interview questions. One operator was unable to participate due to logistical and scheduling constraints.

An additional four out-of-region exchanges were selected for comparison purposes. Those profiles rely entirely on publicly available information.

The Asia-Pacific exchanges are presented separately from the other four. The IXPs in each group are then ordered from the most recently launched to the oldest. The launch date is important here, as it seems to take some time for an exchange to mature.


6 Asia-Pacific Internet Exchange Association (APIX) serves as a forum for IXPs to exchange experiences, https://apix.asia/



@internetsociety

Additional researchIn addition to information gathered from the case study interviews, external sources are used to provide a broader view of the exchange operator’s environment. The telecommunications research company ,Telegeography7, produces country profiles for all of the economies featured in this report.

The detailed history and analysis provided in these profiles were invaluable and are the source for the majority of penetration and market-share statistics. Telegeography was also the source for the majority of information about international traffic flows and submarine cables, some of which is publicly available.

IXP metricsThe metrics regarding traffic volumes and participant numbers used in this report should be considered indicative only.

Not only do they change frequently, but the lack of authoritative information about exchange points also makes any statements about their relative size a dangerous occupation. Sometimes they can be sourced directly from the operator. Other times they cannot.

Many of the figures used here are taken from or inferred by information on the IXP website, or from the invaluable IXP databases maintained by Packet Clearing House8, PeeringDB9, and IXPDB (Euro-IX)10.

Other sourcesOther sources are cited at the appropriate time in footnotes.

Asia-Pacific IXP Case StudiesEvery successful Internet exchange contributes to its ecosystem by keeping local traffic local and reducing latency. Internet Service Providers (ISPs) benefit from reduced cost, better control over their traffic, and increased stability, resilience, and robustness of the local Internet. End users get a better-performing Internet service.

An exchange that cannot provide these benefits is unlikely to attract participants.

Exchange points often bring benefits beyond ports and speeds. IXPs are great attractors that build, or at least enable, an effective ecosystem, bringing different elements of the industry together. The role of an effective IXP is to do more than reduce transit costs.

7 Telegeography, https://www2.telegeography.com/8 The Packet Clearing House (PCH), https://www.pch.net/ 9 PeeringDB – The Interconnection Database, https://www.peeringdb.com/10 The IXP Database (IXPDB), Euro-IX, https://ixpdb.euro-ix.net/en/



@internetsociety

By creating a central point where cloud and content companies can interconnect with service providers, IXPs seem to have a positive impact on other elements of the ecosystem.11

2017: Myanmar Internet Exchange (MMIX), Myanmar

Facing the Bay of Bengal between Bangladesh and Thailand, Myanmar also has land borders with China, India, and Laos. The population of nearly 55 million live on 676,578 km2 of heavily forested land area. This is a combination of mountainous terrain and low river flats.The population density is quite low, at 80 people per km2. 1213

When it was first opened to competition and foreign investment in 2014, Myanmar’s telecommunication market was poorly developed. With little local loop infrastructure in place, there has been a heavy focus on mobile services. By 2020 mobile penetration rate stood 133 percent of the population. Whereas household broadband penetration was only 5 percent. In 2017 it was less than 1 percent.14

The majority of investment continues to go into mobile, which is enabling the rapid growth in access and demand. There are four operators in the market, and the high penetration rates and competition are starting to push prices down.

11 The overwhelming bulk of modern Internet traffic is generated and distributed by specialist content or cloud application companies or is based on cloud infrastructure. Many of these are household names. For simplicity, where the text refers to cloud and content providers or content companies, it is a generic reference to the mix of streaming providers, CDNs, web hosts, social media, online gaming networks, SaaS providers, cloud providers, etc., that are relevant to that community.

12 All population statistics taken from United Nations Statistics Division, Population by sex and urban/rural residence, http://data.un.org/Data.aspx?d=POP&f=tableCode%3a1

13 All land area statistics taken from United Nations Statistics Division, Environmental Indicators, https://unstats.un.org/unsd/ENVIRONMENT/totalarea.htm

14 All market share and penetration statistics are sourced from country profiles in the Telegeography GlobalComms Database, https://www2.telegeography.com/globalcomms-database-service

Economy Myanmar

Population 54,817,917 12

Land area 676,578km2 13

Exhange operator MMIX

URL mm-ix.net

Location Yangon

Established 2017

Participants at this location

13

Average traffic 7.3 Gbps

Prefixes N/A

Myanmar

Yangon



@internetsociety

While it is expected to grow, the fixed broadband market seems unlikely to catch up with mobile soon, if ever. Some fiber is being rolled out to enable fixed broadband services in metro areas. The incumbent operator has almost 40 percent of this market, but the other 60 percent is serviced by more than 2515 service providers, none with a market share of more than 10 percent.

A fiber backbone runs along its main inland routes to service internal centers. It is connected to its neighbors with terrestrial cables and the global Internet via three submarine cable landing stations. Three cables land in Myanmar: SEA-ME-WE-3 (South-East-Asia/Middle East/Western), SEA-ME-WE-5, and AAE-1 (Asia-Africa-Europe 1).

In the next two years, two new high-capacity cables will arrive. MIST (Myanmar/Malaysia India Singapore Transit) and a direct Singapore-to-Myanmar link will come online in 2021.16

Myanmar Internet Exchange launchThe Myanmar Internet Exchange (MMIX) began operating in late 2017. Myanmar’s first, and so far, only public exchange point is run by a limited private company. However, that was never the intention. The exchange operator was to be an industry association. Things have not worked out that way, yet. All the same, MMIX is operational and provides carrier-neutral, settlement-free peering facilities for its members.

Before launch, a series of industry discussions took place in Myanmar supported by Internet development organizations. During these workshops, the industry formed a consensus that an Internet exchange could deliver mutual benefits for industry participants. While the government gave its informal support in private meetings, there was no legislative or regulatory framework in place to accommodate an Internet exchange. Until the government can make a determination regarding IXP licensing and policy requirements, MMIX is operating in a gray area.

Ultimately, ten of the original eleven ISPs included in the early discussions entered into an memorandum of understanding to start MMIX. Once agreed, the next priority was to get some hardware and a place to host it.

Like many not-for-profit exchanges, MMIX started with a generous donation, in this case by APNIC (Asia-Pacific Network Information Centre), of switches and a server. A data center operator offered the group a special discount plus a free month’s rent. Unfortunately, once the exchange was up and running it became apparent that not all ISPs would be able to connect, as the local transport cost to the IX was too high. Those that could join did so and began paying fees, the first step toward sustainability.

At the time of writing, there are thirteen peers, and the exchange has traffic peaks of 15 Gbps.

15 Myanmar’s Internet Service Providers, https://en.myanmartechpress.com/myanmars-internet-service-providers/ 16 Submarine Cable Map, Telegeography, https://www.submarinecablemap.com



@internetsociety

The benefit of membersThe founding members agreed to form an industry association that would be the legal entity responsible for MMIX. A not-for-profit community-led exchange, it would be carrier and data center neutral. All decisions would be made by a board representing the participants. In that way, the exchange would respond to their needs and adopt cost-plus pricing to deliver the greatest benefit possible.

This model is known to engender trust and coopetition among potential participants at a new exchange. Competitive tensions can arise between potential participants, which are, after all, mostly competitive, for-profit ISPs.

The plans came unstuck as the government approval required to form an association became entangled with the operator licensing question. In the interim, the group had to establish the IXP operator as a private company, thus creating a legal entity that can enter into contracts and agreements with providers and participants.

Decisions are still made by a committee formed from membership. Technical operations are provided by volunteers from the participating peers. If the licensing issues can be resolved favorably, the plan is still to transfer ownership of the exchange to an industry association.

Until then, the exchange operations are funded by annual membership and monthly per-port fees. The cost of the data center is paid from this revenue.

Competitive instinctWhile the government continues to be supportive, the policy delay impacts MMIX by making active participant recruitment a delicate issue.

Myanmar has around 100 ISP license holders, although only forty or so actively provide services. Around 20 of these would be considered large-to-medium service providers. Then there are four large mobile operators. The largest of these is already connected to the exchange. The other three are waiting for regulatory clarification.

Some Myanmar network providers and data center operators have already shown interest in launching competing exchanges. Until the government comes to its decision on licensing and interconnection policy, there is no way of knowing if this is allowed. The policy stalemate applies here as well.

Despite not being able to sign up new members, the participation of content providers is helping to grow data volumes. MMIX is eager to secure more content providers and eventually even cloud operators to make the exchange more attractive to potential peers. A high proportion of traffic at the exchange comes from service provider customers accessing content caches.

Economies of scale are important to Internet exchanges. As more members join, the cost of exchange operations is spread across a larger user base. As a not-for-profit, this is directly reflected in the cost-plus pricing model.



@internetsociety

The exchange is hoping to open a second peering location in Yangon and, in the future, to start an exchange in Mandalay, Myanmar’s second-largest city. There is also the potential for establishing remote peering (via a layer 2 carrier network link) to the seat of government in Naypyidaw that could provide better quality access to e-government services.

Careful balanceNot all parts of the Myanmar Internet ecosystem are equally happy with the exchange. Large transit networks and international gateway networks can be negatively impacted by an exchange point, as they would normally profit from the traffic that now takes place settlement-free at the exchange. The risk to them is even greater if the pending licensing and policy decision by the Government has the effect of forcing major transit providers to openly peer at the exchange. Such mandatory multilateral peering (MMLP) is an outcome the operator is arguing against.

Rather, MMIX is hoping the large operators are encouraged to join the exchange but permitted to selectively peer, as this is most likely to result in agreements that are mutually beneficial.

It also promises a better outcome for potential cloud and content companies. If the large ISPs participate at the exchange, content companies can also participate there. If they can reach both large and small ISPs from a single location, they avoid infrastructure duplication and get better value from their investments.

Note: In November 2020, it was announced that Myanmar Internet Exchange Association is officially formed.17

2015: Bangkok Neutral Internet Exchange (BKNIX), ThailandBordering Cambodia, Laos, Malaysia, and Myanmar, Thailand faces the Gulf of Thailand but also has a coastline facing the Indian Ocean. The population is approaching 67 million people, at a density averaging about 130 people per km2 across a land area of 513,115 km2. More than 10 million, or 15 percent of the population, live in Bangkok, which sits at the tip of the gulf.18

An emerging economy, Thailand is one of the largest in Southeast Asia, although its GDP per capita is smaller than that of neighboring Malaysia and well short of Singapore.19

There are four subsea cable landing stations owned by the two major Telcos and at least eight cable systems landing in Thailand. These are either in the Gulf of Thailand or in the west at Satun, just north of the Malaysian border where the Malacca Strait meets the Andaman Sea.

17 https://www.mm-ix.net/news.php?idx=118 All city populations statistics taken from the United Nations Population Division of the Department of Economic and Social Affairs, Urban

Agglomerations with 300,000 Inhabitants or More in 2018, by country, 1950-2035 (thousands), https://population.un.org/wup/Download/19 All GDP Per capita statistics taken from https://data.worldbank.org/indicator/NY.GDP.PCAP.CD



@internetsociety

Terrestrial cables carry some data across the small stretch of the peninsula and meet other cable systems heading east. Two of these, FLAG (Fibre-Optic Link Around the Globe) and AAE-1, use this crossing to link the eastern to the western parts of the cable system. Another two new cables are currently under construction.

Thailand’s links to Singapore (4.9 Tbps), Malaysia (3.4 Tbps), and China (1.1 Tbps) account for the bulk of its 12 Tbps international bandwidth capacity. The remaining links are to the U.S. or Europe and some Asia-Pacific economies. The telecommunications market in Thailand is open and competitive, although it has some strict Internet content and cybersecurity laws.

International traffic typically travels through Singapore or Hong Kong, although there are also terrestrial cables that link Thailand with its immediate neighbors. Thailand has 11 licensed international gateway providers and more than 200 Internet service providers, most of which are very small. Household broadband penetration is 45 percent. Mobile penetration is high, at 186 percent.20

The mobile market has five operators. Three networks make up 99 percent of the market share.

In exchangeMore than a dozen organizations describe themselves as Internet exchange points, although some of these are layer 3 exchanges owned and operated by the largest network providers and were not considered data center or carrier neutral.

The first was the government-mandated National Internet Exchange (NIX) in 1995. This operated until 2003, when the Communication Authority of Thailand was corporatized into the 100 percent government-owned CAT Telecom. CAT competes with another government-owned Telco, ToT. The NIX was transferred to CAT and continues to this day as TH-IX.

20 All prefix counts taken from Packet Clearing House, PCH IXP Directory, https://www.pch.net/ixp/dir

Economy Thailand

Population 66,534,684

Land area 513,115km2

Exhange operator BKNIX

URL bknix.co.th/en/

Location Bangkok

Established 2015


38

Average traffic 26 Gbps

Prefixes 14,594 20

Thailand

Bangkok



@internetsociety

The economy’s first neutral exchange point didn’t emerge until 2015, after almost two years of discussions led by Thailand’s country code top-level domain, or ccTLD (.th) operator, the Thai Network Information Center (THNIC).

An initial meeting of industry participants reached a consensus to work toward an exchange in early 2013. A working group was formed to develop recommendations, which led to a series of meetings and consultations with stakeholders, including the regulator and potential peers.

A great deal of consideration was given to selecting the right location. The objective was to find a reliable, secure data center location that provided fair and reasonable network access. Coming to agreements that satisfied all participants took time and required a level of trust between them.

By early 2015 the exchange was ready for members to connect and was operated by a newly formed not-for-profit company under the .th foundation. The foundation’s board sets the policy, and a small staff runs the exchange using donated hardware located in a commercial, carrier-neutral data center.

Starting with 5 participants, the peers grew to 10 by the first anniversary. Now there are more than 30 peers. BKNIX continues to grow and hopes to reach 40 participants soon.

The exchange established a second site in 2016 at a data center closer to the Bangkok CBD than the first site. Then in 2020 a new exchange location was added in the northern city of Chiang Mai.

There were already nine exchange points in Thailand before BKNIX launched, but they were all carrier transit exchanges where no public peering is done between the connected networks.

Launching BKNIX so long after a healthy competitive ecosystem was established in Thailand has meant that many participants already have private bilateral peering agreements in place. This has meant that BKNIX participants manage their traffic over the exchange in a way that allows them to get full value from their prior commitments. In this way they are almost using the exchange as a redundant link, peering at only a portion of the exchange’s potential.

The operator hopes it can demonstrate that the exchange offers an efficient and reliable alternative at a price per port that will make it more economical than private peering.

Content challengeBKNIX hosts two member meetings a year to gather feedback from participants. They are frequently asked for more content to be added to the exchange. At launch, the exchange focused on attracting local service providers first. For example, port fees were waived for new participants during the first year. When the exchange eventually reached enough provider networks, it was able to start trying to attract cloud and content providers.



@internetsociety

However, some (but not all) CDNs are already directly connected to the large ISPs. International content often still enters from outside of Thailand as international bandwidth. Those who have established private peering facilities, or signed contracts for international bandwidth, have little incentive to share that access with their competitors. Content companies are unlikely to build duplicate infrastructure without a good business case. While large content networks are architected to place content as close as possible to the user, this must be balanced against cost.

2010: Singapore Internet Exchange (SGIX), Singapore

Singapore, with its 5 million people, is more densely populated than Hong Kong due to its smaller 683 km2 land area. The island state has made good use of its position, nestled between Indonesia, the world’s third most populous economy, and the Malaysian peninsula at the most southern tip of the continental mainland.

The Straits of Malacca is the traditional shipping route joining the Pacific and Indian Oceans, and many submarine cable systems take the same route. They might pass west through the Straits of Malacca and across the Indian Ocean to South Asia and Europe or spread east to join up with cables serving Southeast and East Asia and on to North America, or even head south to reach Australia.

Singapore is host to seven cable landing stations and more than thirty subsea cables, making it one of the best-connected locations on the planet.21

21 Submarine Cable Networks, https://www.submarinenetworks.com/en/stations/asia/singapore

Economy Singapore


Land area 683km2

Exhange operator SGIX

URL sgix.sg

Location Singapore

Established 2010


167


Prefixes 295,147

Singapore



@internetsociety

Singapore has an impressive 55 Tbps of international bandwidth. Indonesian transit capacity is nearly 15 Tbps. India and China together account for more than 14 Tbps. There are high-capacity links to Thailand (4.9 Tbps), Vietnam (3.9 Tbps), Japan (2.9 Tbps), and Malaysia (2.9 Tbps), along with contracted capacity to more than 30 nations in total. This demonstrates Singapore’s role as an international traffic hub.

Singapore is a high-growth, high GDP per capita, free-market economy with significant market share held by the state-owned national carrier.

The nation’s tiny size enabled it to adopt broadband technologies very easily, moving to nationwide xDSL with unbundled local loop in 2000 and then on to a full deployment of FTTx today.

With a wired broadband penetration of 107 percent and mobile penetration of 150 percent, the three main providers compete with fixed broadband and mobile services as well as supporting a range of MVNOs (Mobile Virtual Network Operators).

There is a strong second tier, where smaller service providers offer transit to consumers and corporations and then finally to the traditional global and regional carriers who are also operating there.

Government exchangeSGIX (Singapore Internet Exchange) is one of the largest not-for-profit Internet exchanges (IXs) in the region. Launched in 2010 as an initiative under the Singapore government’s Intelligent Nation 2015 (iN2015) master plan, the project was initially funded by the IMDA (Infocomm Media Development Authority).

The exchange started in two sites and then averaged a new site per annum, reaching 10 in 2020. While the number of ASNs has grown steadily, traffic growth has started to accelerate over recent years. The exchange’s 150 or so participants are generating over 800Gbps peak inbound traffic speeds.

Singapore’s regulatory environment encourages all network operators, including those with significant market power (SMP), to interconnect on agreed terms. SGIX enjoys participation from a full range of brand-name peers including operators from global and domestic network providers, social media, and video streaming companies, as well as cloud infrastructure providers, CDNs, online gaming companies, educational institutions, and research organizations.

Other exchangesAnother not-for-profit IXP, the Singapore Open Exchange (SOX), is a community-run exchange operated by the National University of Singapore. According to its website, there is an application process, but membership is free and there are no port charges for its 27 members.



@internetsociety

The Japanese exchange operator BBIX has had an exchange in Singapore since 2014. American data center operator Equinix has one of its 139 peering locations there as well. The Equinix exchange is arguably larger than SGIX, but as it is a commercial exchange that doesn’t publish its traffic statistics, it is difficult to be sure. DE-CIX (Commercial Internet eXchange), the German-based not-for-profit is launching a presence in Singapore as well.

SGIX does not see these other operators as competitors. Its mandate is to promote Internet peering in Singapore and to complement other exchanges in the ecosystem. Practicing full transparency both operationally and financially is designed to assist potential members to evaluate the exchange fairly against the alternatives.

The operator’s approach is to see peering managers as professional experts, able to make their own informed decisions about the prospects of peering at a given location. More important than a “hard sell,” trust must be earned from the community. SGIX understands there is no quick way for an IXP to succeed. It must progressively build its infrastructure, creating tangible value to attract peers.

2004: Bangladesh Internet Exchange (BDIX), Bangladesh

Bordered by India and Myanmar, Bangladesh is a heavily populated region of the Indian subcontinent. The landscape is dominated by the largest river delta in the world, and its large, flat landscape is traditionally prone to flooding. Dhaka, the capital, is home to more than 21 million of the total population of 166 million.

The second-largest city, Chittagong faces the Bay of Bengal and takes advantage of the coastline with two government-owned cable landing stations hosting the SEA-ME-WE-4 and SEA-ME-WE-5 submarine cables. Additional links are available to India via six international terrestrial cable service providers.

Economy Bangladesh



Exhange operator BDIX

URL dix.net

Location Dhaka

Established 2004


120


Prefixes 4,640

Bangladesh

Dhaka



@internetsociety

In 2020 Bangladesh had a total of nearly 2 Tbps international bandwidth capacity. More than half eof this was direct to Singapore. A quarter links to India, with the remainder linking to France and other European cities.

A developing, low-GDP per capita economy, Bangladesh, like its neighboring Myanmar, is classified by the United Nations as an LDC (Least Developed Economy). Given this geographic challenge, mobile Internet far outweighs fixed lines as a connectivity option. Today, wireless penetration is approaching 100 percent, while wired broadband connectivity reaches just 1.2 percent of households. Fixed broadband is only available in urban areas such as Dhaka, which has already completed a metropolitan FTTH roll-out.

Humble beginningsEven before the first submarine cable landed in Bangladesh, it was an environment perfect for an Internet exchange. The Bangladesh Internet Exchange (BDIX) emerged not long after the first satellite services spawned a small ISP in 1996. Within two or three years, there were a handful of dial-up providers each relying on a VSAT link for connectivity.

This fertile ground was a classic case where an email from your neighbor might transit Singapore, Hong Kong, or even the U.S. The round-trip latency was up to 1,000 milliseconds over the satellite, and it was expensive: A 64k uplink could cost a local ISP around USD8,000 per month more than twenty years ago. By 1999, the incumbent carrier had a subscriber base of just 25,000.

With little oversight of infrastructure build, ISPs were already running their own cables to meet their customers. In 2000, two providers decided they could bypass the satellite for their local traffic by running copper cable the three-kilometer distance between their existing Points-of-Presence (PoPs). Once the first two ISPs were connected, it was obvious to try to connect more.

That’s when things became complicated. Finding a business model to build an exchange proved difficult. One operator tried to launch a commercial exchange using radio links, but it wasn’t well supported.

AssistanceEventually, the impasse resolved when an Internet development organization offered to sponsor the formation of an Internet exchange and provided foundation equipment and an operating budget. Office space was rented, and capacity-building organizations assisted with technical support.

The office location didn’t have access to fiber connections, so many ISPs connected with radio. A small number of ISPs, which were closer to the Internet exchange, were able to connect with DSL. Things did not go entirely smoothly at first. Three ISPs broke away to establish a competing IXP, only to return to the fold later.



@internetsociety

As the funding support ran out, the operator put in place measures to make the exchange sustainable financially. It moved to smaller premises and introduced a membership fee. Members pay a one-time contribution per port and an ongoing monthly fee. There are three port sizes, 100M/1G/10G.

BDIX reduced latency for local traffic from 1000 ms over the satellite in 1997 to 5 ms via the exchange in 2004. Then everything changed in 2005, with the launch of the first submarine cable (SEA-ME-WE-4).

Latency to Singapore fell to 60 ms, and VSAT was relegated to a role as a redundant backup. By 2006, ISPs were rolling out fiber connections to the exchange and traffic began to grow. BDIX improved traffic so much for subscribers that it became a symbol of quality service for an ISP if it could advertise that it participated at the exchange.

During efforts to curb the illegal call termination market, the regulator introduced a new licensing regimen including a license class for Internet exchanges.

Two were awarded in 2012, and another seven in the years to 2020. These other exchanges are generally operated by international gateway licensees. The government “strongly encourages” all Internet exchange operators to peer with each other. On agreeable terms, of course.

That is proving problematic. The IXP operators are so far unable to agree on a technical solution or business terms to achieve this. Meanwhile, the government is also considering plans to regulate the tariff IXPs are required to charge their members.

Peering firstBecause it is a membership organization, all decisions are made by a board of trustees. A members meeting is held twice a year where the operator gathers feedback and inputs from the participants. A small staff maintains and operates the exchange.

BDIX has a community around it, and it is growing organically. The average traffic in 2009 was 50 Mbps. By 2015 average traffic had increased to 5.2 Gbps transacted by the 75 participants. It reached 8.2 Gbps in 2016, with more than 120 peers. In 2020, average traffic approached 50 Gbps.

BDIX is a basic peering platform, plus a handful of services including colocation at the BDIX PoP. However, the exchange is at a crossroads on content. The cloud and content companies have not yet established much of a presence in Bangladesh. Any content caches are typically found only on the large ISP networks. Each small ISP has to pay fees to access the content. This can become a major cost component for ISPs, but the ecosystem is too small for these small providers to meet the cache and CDN criteria individually.

Getting the right content connected directly to the exchange would be a boon to the participants. The exchange has already tested content caches on the infrastructure, and it is ready to launch.



@internetsociety

The last sticking point is the problem of cache fill costs. The exchange could buy the transit and pass the cost on to only the members who want to use the service.

However, it’s not clear that its license allows that. The exchange operator doesn’t want to proceed without a determination from the regulator.

2003: National Internet Exchange of India (NIXI), India

Strategically located between the markets of Southeast Asia and Europe, India is home to more than 15 percent of the world’s population. More than 1.3 billion people live at a relatively high density of 400 per km2 across India’s 3.28 million km2 land mass.

A developing economy with low GDP per capita, India’s population is relatively evenly spread. While India has two cities exceeding 10 million inhabitants, it has around 40 cities with a population of over 1 million and another 400 are home to more than 100,000 individuals.

India’s economy is one of the world’s fastest growing. Overall GDP per capita is low, but the urbanized and heavily populated areas to the north and along the coasts are home to a burgeoning middle class.

Due to the low levels of Internet penetration in rural India, 70 percent of Internet users are in 10 or so major cities, such as Delhi, Mumbai, and Chennai. Rural broadband is growing steadily. Although national household broadband penetration is only at six percent, that’s still a large number of subscriptions.

In contrast, there are more than one billion mobile subscriptions, and penetration is approaching 80 percent. 4G accounts for the bulk of mobile services.

India

Delhi

Economy India

Population 1,312,240,000

Land area 3,287,263km2

Exhange operator NIXI

URL nixi.in

Location Delhi

Established 2003


32


Prefixes N/A



@internetsociety

When you consider that there are well over 2,000 towns with a population over 10,000 and 600,000 villages with populations of only a few thousand, the challenge to connect them all is clear.

Since the launch of the Digital India project in 2015, the initiative has reached phase two of the BharatNet, which plans to connect all 600,000 villages. So far around 350,000 of these are already connected with fiber. The other 250,000 villages are expected to be connected by 2023/4.22

At the other end of the scale, cities like Mumbai, Bangalore, Chennai, Kolkata, and Hyderabad are names globally recognized for ICT and particularly for IT services exports.

The incumbent and surviving cable TV operators have become the big national Internet providers, while smaller, entrepreneurial operators take up the state- or district-level licenses. The number is approaching 2,000 and is likely to increase, supported by increased penetration as the fiber backbone reaches the remaining villages.

Two major submarine cable landing stations in Mumbai and Chennai are home to sixteen subsea cable systems, while another three stations in the south serve the Maldives and Sri Lanka. A dozen cables land in Mumbai, many on their way to Europe, while Chennai hosts cables that run down the Malacca Strait.

India’s contracted international bandwidth is substantial, at nearly 17 Tbps. There are multi-Tbps links to Singapore in the east and in the west to France and the U.K. India maintains contact with South and East Asian capitals as well as the Middle East and some other European cities.

National exchangeNational Internet Exchange of India (NIXI) is a public-private partnership between the government of India and industry ISPs. Four representatives from the Ministry of Electronics & Information Technology sit on the 15-member board. NIXI is not-for-profit, with all surpluses reinvested into the exchange.

Launched in 2003, NIXI created fertile ground for an exchange ecosystem in India. Today there are no licensing restrictions to operating an IXP in India. After the launch of NIXI, it took 14 years before any other IXP was established in India. Today there are 19, including two with higher daily traffic peaks than NIXI.

NIXI initially adopted mandatory multilateral peering, enforced a system of settlement fees, and barred content providers from joining the exchange. These restrictions have now been removed.

22 “Every village will get optical fiber internet in 1,000 days: PM Modi in Independence Day speech,” India Today, https://www.indiatoday.in/india/story/every-village-will-get-optical-fiber-internet-in-1-000-days-pm-modi-in-independence-day-speech-1711415-2020-08-15



@internetsociety

The “Requester Pays” settlement fee regime was disadvantageous to those participants with networks that consumed content: mostly retail ISPs. The larger players with international links were lucky to be on the right end of the equation. NIXI gradually reduced the fee over several years before it was finally eliminated.23

NIXI policies did not permit content providers on the exchange until October 201924. Content companies are not licensed in India, and the exchange was open to licensed operators only.

The delay in allowing cloud and content providers has shaped the ecosystem. The largest brand-name content and cloud companies have established their peering facilities at competing exchanges, or within ISP networks.

To attract the cloud and content providers onto the NIXI exchange, the operator is now offering them free connections and giving them “esteemed guest” status at the exchange.

National goalsThe nation’s Internet exchanges, coupled with India’s prodigious content industry, are helping achieve the government’s goal to keep local traffic local. Today, 90 percent of the nation’s traffic is kept local. This is helped by the fact that 65 percent of the domestic traffic is between ISPs and content or cloud providers.

While large networks are forced to announce their regional reach at NIXI, this is not generally seen as a necessary strategy among other IXP operators.

These days, a large proportion of peering traffic at IXPs is running directly from content to eyeballs. Colocating caches and content engines at a carrier-neutral data center, which also hosts an exchange point where they can peer with both large and small service providers, lowers operational costs for all. There are two other IXPs operating in India worth noting here. Extreme IX is an open carrier- and data center-neutral Internet exchange point, with more than 30 sites in 5 cities and a participant list that includes leading content sources. Daily traffic peaks are approaching 1 Tbps, with an average of 460Gbps over the past year.

Another peering location on the west coast at Mumbai was recently taken under German operator DE-CIX’s umbrella and now claims to be the second-largest peering site in the region.25

Expansive horizonsNIXI hopes to take its brand offshore and has ambitions to help neighboring countries establish similar government-led public- private not-for-profit exchanges.

23 NIXI Routing and Tariff Policy Ver.1.11 (2011), https://web.archive.org/web/20111016110217/https://nixi.in/en/routing-and-tarrif-policy24 NIXI is now open to peer with Content providers, https://nixi.in/en/events/Notice/2125 “DE-CIX Mumbai ranked second largest Internet Exchange in APAC, VAR India, https://varindia.com/news/decix-mumbai-ranked-second-largest-

internet-exchange-in-apac



@internetsociety

India has a large land area and many, many people. It has providers at a scale that might ultimately require highly distributed IXP locations and sites. It is likely to see an increasing number of exchange locations, or perhaps small exchange operators located close to the network edge.

If an exchange operator can establish in 100 or so cities, it might be a way to interconnect India’s enormous population efficiently and economically.

2001: Japan Network Access Point (JPNAP), Japan

The first stop for data heading west from California is the nearly 7,000 islands that make up Japan, home to the world’s 11th largest population. With five main islands and a total land area of less than 380,000 km2, Japan still has large areas of sparsely populated, mountainous terrain. This leaves the nation’s 126 million people living mostly in densely populated cities.

The population in the greater Tokyo area is more than 37 million, a third of the total population.

An advanced free-market economy with a high GDP per capita, Japan aggressively adopted the Internet through the 1990s after the incumbent POTS (Plain Old Telephone Service) carrier was privatized in 1985. A competitive market was able to flourish in the high-density living conditions. Today, broadband penetration stands at 200 percent of households, while mobile penetration is 145 percent of the population.

Well served with subsea cable transit to the United States, Japan is a long-established hub for traffic passing between the Eastern and Western Hemispheres. It is the first landing point for a large number of cables. These often continue to key destinations along the eastern seaboard of the Asian continent.

Japan

Tokyo

Economy Japan



Exhange operator JPNAP

URL jpnap.net

Location Tokyo

Established 2001


174

Average traffic 1.5 Tbps

Prefixes N/A



@internetsociety

Japan’s contracted international bandwidth in 2020 was estimated at 25 Tbps, with links to the U.S., China, and Singapore accounting for more than 20 Tbps. Other Asian capitals account for the bulk of the other capacity, with only small links out of the region to Europe.

Although the government maintains an ownership stake in NTT, the incumbent carrier was operationally separated and now provides unbundled local loop to the more remote parts of Japan. The three major carrier groups NTT, Softbank, and KDDI are joined by cable operators, WiMAX providers, FTTx, and DSL access providers. The speeds and subscription rates are amongst the best in the world.

The telecommunications landscape is an open, competitive, free market with some regulations to restrict operators with Significant Market Power.

In addition to extensive 3/4/5G GSM networks, the three main operators also run competing commercial Internet exchanges: Softbank (BBIX), KDDI (JPIX), and NTT (JPNAP). Global data center company Equinix operates exchanges in both Tokyo and Osaka, rounding out the big four exchange players.

Quality first By the late 1990s, the emerging Internet service providers in Japan were leasing circuits and buying transit from the U.S., causing a 16,000 km round trip if traffic was passing between neighboring ISPs. In response, the first IXP emerged out of the research community. However, the high QoS (Quality of Service) expectations of Japanese consumers meant that ISPs were soon looking for a carrier-grade solution. Naturally, the three large operators stepped in to create a market.

The first commercial exchange was launched in 1997 jointly by two major ISPs in Japan, including KDD (now KDDI). In 2001 NTT established an exchange, JPNAP, in partnership with IIJ (the Internet Initiative Japan) at two locations: NTT data center and IIJ data center. The latter is carrier neutral.

After openly peering in the early research phase, the large players moved to selective peering policies once the focus was on commercial operations. A third carrier-backed exchange, BBIX, was launched by Softbank a little later in 2003. Two years later, Equinix entered the market and started an Internet exchange service in Japan in 2005.

These all offer carrier-neutral access points, although carriers are involved with three Japanese exchanges. As an optional service, a VLAN is available for transit on top of the IX infrastructure if a participant would like to buy transit there. With high availability an accepted responsibility of these large network operators, no downtime is acceptable, and the exchanges are fully redundant with hot standby failover.

Domestic networks make up 60 percent of JPNAP participants. The international peers are split between foreign access providers (30 percent) and Tier 1 and 2 operators (20 percent). The remaining 50 percent are overseas content providers and CDNs.



@internetsociety

Competitive exchangeJPNAP is an open, carrier-owned, for-profit, neutral Internet exchange. While it is able to respond to market needs quickly, it can be restricted from competing in categories already contested by its parent. There is also fierce competition from the other two carrier-owned exchanges.

Competition has forced prices down on basic interconnect services. Per-port fees have reached a low price point, and the IXPs are now competing on the basis of how many peers are at their exchange and, accordingly, how much traffic can be exchanged to reduce the cost of transit.

The content providers already play a big role in the Japanese Internet. They mainly collocate and connect at the exchanges, but with the competing carrier environment, there’s some pushback about the number of exchanges. A content provider needs to be at all four exchanges to reach all Japanese subscribers. The cable TV companies and smaller ISPs connect first to their transit provider’s exchange and then to maybe one or two more. Japan’s IXPs run very high-density locations in Tokyo and Osaka.

Just recently, BBIX and JPNAP announced that their exchange operations would start to reach out to the network edges in Fukuoka, Japan.

Peering portalsOpen multilateral peering on the route server is the norm for 70 percent of JPNAP participants. They generally also have bilateral connections with selected peers. The remaining 30 percent of the peers maintain a selective peering policy only.

Content players are tending to move away from open multilateral peering as it results in a high number of participants at very low traffic volumes. The larger players run customer peering portals where prospective peers can apply and be automatically added if they meet the right peering criteria.

In the future, JPNAP plans to continue expanding its reach to establish peering points overseas. It so far provides services to KINX, JKT-IX, and MYNAP, as well as to other peering points with partners like IPTP networks and Megaport.

Another area of growth is the direct cloud connect market. For small and medium enterprises that do not want to or cannot connect directly to the exchange, ISPs and IXPs will collaborate on a solution to provide enterprise customer cloud connect services using a VLAN or virtual path across the top of the public Internet infrastructure.



@internetsociety

2000: Korea Internet Neutral Exchange (KINX), Republic of Korea

The Republic of (South) Korea’s closest cultural, economic, and communications ties are with Japan. The two countries are separated by only 210 km between Busan and Fukuoka.

Like Japan, the 100,000 km2 land mass of South Korea is densely populated. The tendency to accumulate in urban centers to the north means the population density can be far higher there. The metropolitan Seoul area (incorporating Incheon) colocates more than 25 million people in an area less than 12,000 km2. That is a population density of more than 2,000 people per km2 compared with the national average of approximately 500 p/km2.

An active, developed free-market economy, South Korea has a GDP per capita 25 percent lower than Japan’s.

Internationally, subsea cable landing stations in and around Busan in the south link South Korea to the global Internet. Connections are predominantly via the east coast of Japan. There are a growing number of other links to mainland China, Hong Kong, and Singapore, which provide paths to the West Coast of the U.S. and population centers in East and Southeast Asia. South Korea has nearly 7 Tbps of international bandwidth transit. There are direct links to the U.S., China, and Japan. Each has a capacity of around 2 Tbps. There are only comparatively small links to other countries in the Asia-Pacific region and Europe.

South Korea’s competitive telecom environment has delivered fixed broadband penetration rates of 105 percent and wireless penetration of 134 percent. The local loop was unbundled in 2002 and price managed by the regulator with cost-plus price ceilings.

South Korea

Seoul

Economy South Korea


Land area 99,538km2

Exhange operator KINX

URL kinx.net

Location Seoul

Established 2000


64


Prefixes 191,654



@internetsociety

Today, a nationwide fiber network completed in 2005, along with the “Giga Internet” network built by a consortium of the three largest providers in 2012, carries residential traffic. FTTx subscribers make up more than 80 percent of the wired broadband market. A further 14.5 percent are on cable infrastructure. South Korea regularly rates at the top of world rankings for the fastest residential Internet access speeds. However, all this high-speed Internet increases content consumption, a significant proportion of which is generated outside Korea.

Private companyThe Korea Internet Neutral Exchange (KINX) was launched in a commercial data center by 13 members of the Korean Internet Association. A year later, in 2000, it was transferred out of the association into a privatized, not-for-profit company. It had around 60 participants at the end of 2020.

There are now about 30 ISPs in Korea, although there has been a recent round of upward consolidation, with Tier 2 providers acquired by the larger operators. The dominant service providers, who don’t peer at the exchange, now account for more than 93 percent of the total broadband subscriber base.

This makes it difficult to attract major content players on to the exchange, as they tend to peer bilaterally with the large operators as opposed to peering at the IXP.

In 2016 the regulator introduced a “Sending Party Network Pays” regime of interconnect settlements. The purpose of the change was to encourage the Tier 2 network providers to develop and become Tier 1 providers.

This had a chilling effect on any ISP that hosted content on their network, as it would be charged for any user who accessed that content. The move effectively killed settlement-free peering in Korea, although KINX still guarantees this on its exchange. Costs reportedly doubled and tripled as the regulatory change flowed out through the industry.

The government sets a price ceiling. It is reviewed annually and the price is falling, but the outcome is still such that a network hosting content could be drained by settlement fees. The money tended to flow from larger to smaller operators. In response, the major players tried to pass this cost back to the content providers in the form of paid peering contracts, to the point where the cost to peer was higher than the price of transit.

Japan is only around thirty milliseconds away, so cloud and content providers could easily locate their main point of presence there, instead of directly on a South Korean domestic network.

With no terrestrial alternatives, the landing stations are the only source of international bandwidth, and these are operated by the dominant Korean providers.



@internetsociety

Once landed, traffic must travel north to Daejeon and on to Seoul on domestic transit links, which are also provided by the major players. There are reported delays in securing 10 Gbps domestic circuits, regardless of price.

In May 2020, the government instituted the “Content Providers’ Traffic Stabilization Law,” which requires content players to ensure the quality of their service when delivered over a network they do not control. The content providers had serious concerns, and at least one walked away from the market.

Further afieldKINX had the Korean Internet ecosystem to itself until 2018, when alternate IXs began to appear. There are a couple of carrier exchanges, and data center operator Equinix has entered the market.

The exchange operator sees little prospect for peering growth from local networks, as nearly all the would-be participants are already connected. Except for the three major carriers. Without their participation, attracting content providers is difficult. So, the Internet exchange is looking for other opportunities.

The exchange now has product offerings in transit, cloud, data center colocation, content delivery, and security. It is in the process of establishing connectivity from “a global network provider” so it can offer remote peering with overseas IXPs. There are already partnerships with JPNAP and HKIX as the exchange seeks to increase its reach, rather than chasing a large increase in traffic.

1997: Internet Exchange Australia (IX-Australia), AustraliaAustralia is a vast, sparsely populated continent that is geographically quite isolated from the rest of the world. It faces the Americas on its eastern seaboard, where the majority of the population lives in urban centers close to the coast.

On Australia’s west coast, the city of Perth faces the huge expanse of the southern Indian Ocean. Perth is one of the most remote cities in the world. Despite its location at the bottom of the Australian continent, it is closer to Jakarta than it is to Sydney.

It’s perhaps not surprising then that Perth emerged as a hotbed of entrepreneurial Internet activity in the early years of retail commercialization. One of Australia’s leading consumer ISP brands, iiNet, had its origins in a Perth suburban garage. It is not surprising either that iiNet was a founding member of WAIX (Western Australian Internet Exchange) – an initiative of the Western Australian Internet Association (WAIA).

Although the names and mandates have changed a little, IX-Australia is the descendant of this early project, and the Internet Association of Australia (IAA) is what was once WAIA.



@internetsociety

Australia is a developed market economy, rich in resources that provide well for the 25 million inhabitants who share the 7.6 million square kilometer land area. At a density of just 3 people per km2 Australia is a land of contrasts with concentrated urban populations between 2 and 5 million, plus a strip of regional and rural townships along the east coast, and not much else.

Fiber rolloutThe tyranny of distance has always plagued the Australian telecommunications infrastructure. A national broadband initiative with plans to deliver wholesale broadband services to 93 percent of the population recently reported it reaches nearly 12 million premises.

Household broadband penetration in Australia sits at around 80 percent, below the 88 percent average for similar GDP per capita countries.

In addition to the government-owned national access network, Australia has a competitive mobile ecosystem with three major network providers and a range of MVNOs. Mobile penetration is 121 percent, with 5G deployments rolling out across major urban centers.

Telecommunications regulation in Australia is pro competition, with structural separation supported by provisions that allow the regulator to “declare” service to enforce sharing of infrastructure, such as the local loop. There is a vibrant Tier 2 market that occasionally suffers from consolidation, but leased circuits and dark fiber are readily accessible and cost effective.

International bandwidth primarily arrives in Sydney via the Pacific from the U.S. using subsea cables such as the 20-year-old Southern Cross Cable Network (SCCN), which makes a 30,500-km round trip via New Zealand to Hawaii and on to the west coast of the United States.

Australia

Economy Australia



Exhange operator Internet Association of Australia

URL ix.asn.au

Location Multiple across Australia

Established 1997


228


Prefixes 137,458



@internetsociety

Australia’s 7 Tbps of international bandwidth capacity is shared among a dozen destinations. Predominantly this traffic is to and from the U.S., New Zealand, Singapore, Japan, China, and Indonesia. The 3.3 Tbps U.S. link is the largest and nearly half the total.

A cable landing station in Perth has links to the east coast at Sydney as well as a series of cables that provide access through Singapore. Perth is also part of the massive 39,000 km SEA-ME-WE-3 cable that hugs the southern coast of Asia and Europe from the Republic of Korea to Germany. A direct Perth-to-Oman link will launch in 2021.

Association exchangeFormed by the association as a not-for-profit carrier- and data center-neutral exchange, WA-IX began to expand between 2011 and 2014 to reach five major Australian cities.

It now has 35 sites and more than 300 member ASNs. The exchanges are hosted in commercial, carrier-neutral data centers and use cost recovery pricing. IX-Australia is also a licensed telecommunication carrier as it owns and operates its own fiber infrastructure. Other links are leased from the vibrant-but-consolidating Tier 2 ecosystem.

There are three other significant exchanges in the market. Megaport is a Network as a Service provider, while a newcomer, for-profit EdgeIX, launched in 2019. Pipe Networks, a recently consolidated Tier 2 provider, operates a carrier-exchange in six cities and has more than 300 active connections. Equinix is also present in the major capitals.

Content a priorityA recent survey of IX-Australia members reinforced the importance of content. Access to content and the cheapest possible access to peers are the high priority, as one might expect. Low-cost peering is more important than a lot of interesting vertical offerings. The operator is attracting content and cloud providers as well as an increasing number of large enterprise users who rely heavily on the Internet.

IX-Australia has an extended reach product that allows participants to peer remotely across the continent. Members that reach one peering point can be present at any exchange nationwide using the exchange’s own circuits.

IX-Australia also has an agreement to manage the five sites that make up the New Zealand Internet Exchange (NZIX) on behalf of another industry association (NZIX Inc). IX-Australia takes responsibility for the peering fabric and the administrative and operational procedures, right down to providing local hands for maintenance. It’s a model the Australian exchange is considering as a way to extend its peering reach.

A partnership with a New Zealand-wide wholesale provider allows participants there to peer across the entire national infrastructure.26 The outcome could be a fundamental change to the concept of reach and the economics of peering in New Zealand.

26 NZIX Offers Peering New Zealand Wide Via Chorus UFB, https://ix.nz/2020/11/30/nzix-offers-peering-new-zealand-wide-via-chorus-ufb/



@internetsociety

1995: Hong Kong Internet Exchange (HKIX), Hong Kong

The traditional gateway between mainland China and the west, Hong Kong is a key financial and shipping hub in Asia. Barely bigger than Singapore at 1,100 km2, the administrative region is slightly less densely populated. The 7.5 million population fills the landmass at the rate of nearly 7,000 people per km2.

Highly developed with a high GDP per capita economy, Hong Kong is a major hub for Asia’s Internet. There are 11 submarine cables landing in eight locations, giving Hong Kong a reach from the United States to Europe. Hong Kong is also the major traffic hub for China markets. Connection times to reach the rest of East and Southeast Asia are low, at around 30 ms to Singapore, 46 ms to Beijing, and 50 ms to Shanghai.

The territory’s emphasis on new technologies has seen it provide wide-scale fiber-based broadband infrastructure, while 5G services have been available since April 2020. Intense competition within a small market has led to consolidation, with the number of mobile operators reduced to four, though there is room in the market for several MVNOs.

The small distances in Hong Kong make xDSL an effective technology, but the market has moved primarily to fiber and mobile. Fixed broadband penetration stands at 91 percent, but mobile penetration is 131 percent.

The competitive network ecosystem consists of 27 local fixed operators and four mobile operators. Home broadband is dominated by four major players, and enterprise has three major players to choose from. In all, there are more than 250 Internet service providers.

Hong Kong

Economy Hong Kong SAR (China)


Land area 1099km2

Exhange operator HKIX

URL hkix.net

Location Hong Kong

Established 1995


333


Prefixes 156,777



@internetsociety

Major HubEstablished in 1995, Hong Kong Internet Exchange (HKIX) is a wholly owned subsidiary of the Chinese University of Hong Kong Foundation Limited, though it is an open exchange serving more than just the research and education networks.

HKIX is a leading exchange by world standards and has peak traffic approaching 2 Tbps and more than 330 participants.

The university’s internal Information Technology Services Center (ITSC) is responsible for the exchange, with equipment originally provided with the university’s budget. Growth is funded from participants through per-port charges.

Traffic is growing at around 30 percent annually. Every month, new participants are being added – even in 2020. Domestic access providers are joined at the exchange by a wide range of gaming and retail networks, cloud, and content providers. The exchange estimates that 90 percent of the potential market is already peering. New growth comes from new networks entering the market.

Serving as the best gateway to China, Chinese ISPs tend to set up PoPs in Hong Kong and connect with the rest of the world via HKIX. The exchange’s peering policy permits participants to route their international traffic over the peering fabric, so traffic statistics are increased as a result.

There is so much traffic passing through Hong Kong; HKIX is just one of a small number of exchanges present. Equinix is there, as is AMS-IX (Amsterdam Internet Exchange), BBIX, and the Japanese exchange has a presence, as does Megaport. In addition to the two core sites at the university, HKIX has also established five satellite sites with data center operators including CITIC Telecom, KDDI, NTT and SUNeVision.

The HKIX operator continues to seek opportunities to grow the number of ASNs and more locations across the region, but it is also looking to develop better habits among its peers and to improve routing security with the implementation of RPKI (Resource Public Key Infrastructure) and RTBH (Remotely Triggered Black Hole) filtering.

The exchange is also looking at Cloud, IoT, and 5G usage as possible growth accelerants. Direct cloud exchange for corporations is becoming increasingly important in the Hong Kong market, for example.



@internetsociety

Other Region IXP Case Studies

2002: Internet Exchange Brazil (IX.br), Brazil

With a population of 210 million people living in a land area only slightly larger than Australia, the average population density of Brazil is only ~25 persons per km2. With a land area that is the fifth largest in the world, Brazil is urbanized into highly concentrated centers along the east coast.

There are more than a dozen cities larger than 1 million people, with population centers predominantly toward the south. Beyond that are 5,570 smaller municipalities.

São Paulo at more than 13 million people and Rio de Janeiro at almost 7 million people are the two largest centers. Facing the South Atlantic, the greater São Paulo region includes five other cities of over 1 million, taking the greater metropolitan area to more than 35 million.

There are landing stations all along the coast. The Brazilian Festoon is a domestic submarine cable system that runs from Rio de Janeiro up more than a dozen stations to Fortaleza. Along with São Paulo and Rio de Janeiro, Fortaleza is a key site for many cable systems as it is the hop between the two Brazilian cities in the south and Miami, Florida, in the U.S.

There are four major mobile networks. MVNs are a new addition to the competitive mix. Two companies also offer fixed connections on 4G LTE. Three of the mobile leaders have fixed broadband offerings. Fiber is available in the larger cities, but the government is looking to satellite to provide access in remote areas. The government is also investing and encouraging investment in 4G and fiber. The fixed broadband market is predominantly xDSL and cable, although fiber is growing. Plans for a 5G spectrum auction are currently on hold.

Brazil

São Paulo

Economy Brazil



Exhange operator IX.br (PTTMetro)

URL ix.br

Location São Paulo

Established 2002


2116


Prefixes N/A



@internetsociety

Despite a contingent of more than 7,000 Internet service providers, Brazil’s broadband penetration is only around 55 percent of households, up from 27 percent in 2010. Mobile penetration is 102 percent. Overall Internet penetration has reached 70 percent of the population.

Moving fastBrazil is an open, competitive, free telecoms market, where the Brazil Internet is coordinated by the CGI.br (Brazilian Internet Steering Committee). This unique multistakeholder committee is composed of 21 members, including representatives from government, private enterprise, civil society, and the technical community. Among its other responsibilities, the CGI.br directs the organization that manages the .br domain space (NIC.br). It is also the National Internet Registry (NIR) and operates an Internet exchange that is arguably now the largest exchange in the world.

Brazilian access providers have taken to Internet exchanges with enthusiasm since the first of the IX.br exchanges opened in 2002 as PTT Metro.

São Paulo is the largest, with peak traffic loads over 10 Tbps (average of 4.78 Tbps) and more than 2,100 ASNs. The IX is available across more than 30 sites. Small providers account for around 15 percent of the Internet market, and nearly a third of the providers are now connected to the IX.br.

One of the services offered is an exchange switch port that can be shared by multiple ASNs. In this way, several small providers, in a remote city, can share one leased circuit back to the exchange. Sometimes operators bypass smaller exchanges to reach the larger São Paulo site.

Domestic transit companies also sell transit from a PoP in a regional area back to the larger exchange points.

Participants are not permitted to use the exchange as their only connection to the global Internet but peering at IX.br can provide them access to 60-70 percent of the traffic required to run a residential ISP in Brazil.27 The permissive policy approach extends to allowing transit providers to sell transit over VLANs running on the exchange infrastructure.

No cost peeringThe exchange is not-for-profit and almost fully funded by NIC.br. There are no per-port costs for all but the heaviest of users. The two main exchanges were free until quite recently when São Paulo and Rio de Janeiro began charging in 2017. Peering is free at all the other locations, unless your traffic goes above 1 Tbps.

The exchange lowers its cost by entering agreements with data center operators where they provide the rack space, power, and dark fiber to the central points. In return, they gain the right to sell cross connects to the participants.

27 Peering opportunity in Brazil - IX.br, https://2020.apricot.net/program/presentations/#peering-opportunity-in-brazil



@internetsociety

Open peering is the norm with 97.72 percent of ASNs, which leaves only the transit providers opting for selective peering.28 Cloud and content providers are extremely well presented at the two largest exchanges, with every major player in the Western Hemisphere represented in at least two exchanges.

2000: Kenya Internet Exchange Point (KIXP), Kenya

The Republic of Kenya is an East African nation that is home to more than 54 million people. Similar in size and population to Myanmar, Kenya has a land area of 676,577 km2 and a population density of 82 people per km2.

Kenya is fortunate to have two cable landing stations at Mombasa, facing the Indian Ocean and connecting Kenya to the outside world. Sharing terrestrial borders with Ethiopia, Somalia, South Sudan, Tanzania, and Uganda, Kenya has a national fiber backbone that reaches all parts of the country and interconnects the landlocked Uganda at its border near Lake Victoria.

There will soon be six cables landing at the two stations. Kenya’s geographic location means it benefits from an ever-increasing amount of bandwidth availability running south along the coast; it also has easy access to the main networks running through the Red Sea at Djibouti.

Overall Internet usage is estimated at 40 to 50 percent of the population. Kenya is a mobile-first ecosystem, with only a small amount of cable or fiber in the ground. This restricts the opportunity for fixed broadband subscribers, to the point where the incumbent simply abandoned its ADSL services in 2015 to focus on fiber. Fixed broadband penetration is around five percent of households, and although some fiber is starting to reach the home, a third of the existing subscribers are on cable.

28 Dissecting the Largest National Ecosystem of Public Internet eXchange Points in Brazil, https://www.researchgate.net/publication/295861650_Dissecting_the_Largest_National_Ecosystem_of_Public_Internet_eXchange_Points_in_Brazil

Kenya

Nairobi

Economy Kenya



Exhange operator KIXP

URL tespok.co.ke

Location Nairobi

Established 2000


53


Prefixes 184,030



@internetsociety

In contrast, mobile penetration is 106 percent. There are four mobile operators in the open competitive market. The lead operator has more than 66 percent of the subscribers.

A new exchangeNearly a year of preparations are said to have gone into the launch of Kenya’s first Internet exchange point, the Kenya Internet Exchange Point (KIXP). The product of an initiative by the local ISP association, TESPOK (Telecommunications Service Providers of Kenya), the Nairobi-based exchange began operating in 2000.29

It closed almost immediately.

Telkom Kenya filed a complaint with the regulator, arguing that KIXP violated its exclusive access rights and was an illegal network operating without a license. The regulator agreed that an Internet exchange would require a license and, to much protest, shut down the newly launched IXP.

TESPOK argued the IXP was a closed user group and therefore legal, but it took almost a year before a consortium of small ISPs was able to form a private company called Kenya Internet Exchange Point Ltd. It applied for and was granted a license in 2001. KIXP went live for the second time in 2002, with five ISPs and eight other networks peering.

Kenya became the first economy in the world to require an IXP to be licensed.30

Prior to launch, all traffic between service providers went offshore to interconnect. The immediate payoff for exchange participants was an estimated 30 percent of upstream traffic they were able to keep local. During the first two weeks of KIXP’s operation, latency dropped from averages of 1,200 to 2,000 milliseconds via satellite to 60 to 80 milliseconds across the exchange. Monthly bandwidth costs fell to a tenth of what they were previously.31

MMLPAThe exchange had a mandatory multilateral peering agreement (MMLPA) policy in place for more than 10 years, which some commentators blame for slow progress in the early years. The requirement was dropped in 2012, and further regulatory reforms were undertaken in 2015.

The exchange now has locations in Mombasa and Nairobi, with a total of 64 peers. The participants include government and education networks, content providers, and even the mobile operators. Efforts by development organizations have supported the Kenya exchange, and it follows a not-for-profit model that fits with its funding and support profile.

Like similar exchanges, the data center operators provide the IXP with free space in exchange for the rights to provide adjacent services. The IXP benefits from the free

29 KIXP Background, https://www.tespok.co.ke/?page_id=1165130 “IXP helped foster government industry cooperation in Kenya,” Afrinews Blog, https://afrinic.net/blog/105-ixp-helped-foster-government-

industry-cooperation-in-kenya31 KIXP Background, https://www.tespok.co.ke/?page_id=11651



@internetsociety

colocation and the proximity to the networks, and the data center benefits by selling the exchange cross connects and a growing ecosystem.

There is only one other exchange of note. The Dutch-based Asteroid exchange has a handful of peers with a good representation of content leaders at the submarine cable landing station in Mombasa. With Mombasa likely to become an important hub for the East African Internet ecosystem, the market for peering could grow further.

1996: Equinix, United States

The home of the Internet needs little introduction. A land area over 9 million km2 is home to around 330 million United States residents at a density average of 34 people per km2. Concentrated metropolitan areas on the East Coast extend south to the Gulf of Mexico and connect, via terrestrial links, to Southern California and the technology centers all the way up the West Coast.

A great deal of global traffic traditionally passes through the United States. The Atlantic submarine cable systems are home to the most advanced and densely served subsea links on the planet. Traffic from Europe joins traffic from the U.S. on the West Coast, where a series of landing stations feed traffic to Asia. These either transit via Guam and Hawaii if they are heading south or run directly to Japan or the Philippines. The bulk of traffic in and out of South America traverses through Florida.

As you would expect, this is the home of the earliest Internet exchanges, which were required during the commercialization of the Internet in the early 1990s. PAIX, the Palo Alto Internet eXchange, was the first commercial, carrier-neutral exchange point in the United States. Launched in 1996, it was owned and operated by Digital Equipment Corporation. Today it is owned and operated by data center operator Equinix.

United States

Palo Alto

Economy United States



Exhange operator Equinix

URL ix.equinix.com

Location Palo Alto

Established 1996


92


Prefixes 1,342,039



@internetsociety

The North American market is dominated by commercial exchanges, although the not-for-profit Seattle IX (SIX) handles more traffic than any other public exchange in the U.S., with peak speeds approaching 2 Tbps. A good deal of traffic in the continental U.S. is carried as transit, compared with Europe where peering is more prevalent and a 2 Tbps peak wouldn’t make it into the top five exchanges.

As a commercial operation, Equinix has certain advantages over associations and other not-for-profit exchanges. While community-led exchanges must treat all participants equally, put the interests of members first, and avoid competing with its members, a commercial IX can create conditions that make itself an attractive colocation site over and above the interconnection offering.

A commercial exchange can more easily target and entice desirable participants with favorable pricing, as NIXI is doing with content companies. Equinix can bundle a range of data center and interconnection services. Aside from NIXI’s special treatment for content companies, many other not-for-profit, carrier- and data center-neutral exchanges might find it politically difficult to offer free colocation or free peering to selected participants, no matter how important they might be to the exchange ecosystem.

Such commercial enterprises do grow organically, but differential pricing practices allow for better margins than the cost recovery charging at not-for-profit exchanges. The prospect of profit also means commercial exchanges have the potential to raise capital or take on debt to grow through acquisition.

Equinix is a good demonstration of this point. It is a data center operator first and foremost, but the company has quickly shot up the Internet exchange leader boards thanks to a series of acquisitions, particularly over the past 10 years.

Accelerated growthIn 2010 Equinix completed the acquisition of Switch and Data, a commercial carrier-neutral data center and exchange operator that had already acquired PAIX in 2002, not from Digital, but from a fiber-optic network operator called AboveNet. Through these iterations of ownership, PAIX remained carrier neutral. It grew to locations in Atlanta, Dallas, New York, Seattle, and Vienna, Virginia (outside of Washington, DC).

In 2017 Equinix made another acquisition of 29 data center buildings from Verizon, including NAP of the Americas (NOTA), further bolstering its exchange credentials. NOTA in Miami was and still is a hub for data traffic to the Caribbean and South and Central America.

A series of smaller acquisitions have placed Equinix as the leading retail data center operator worldwide and coincidentally a leading IXP, with 139 exchanges globally located in 36 metropolitan areas across 20 countries with more than 1,500 participants for a total number of ASNs approaching 2,000. The number of non-unique ASNs at all Equinix exchanges worldwide is over 3,000. This compares favorably to IX.br and is not far short of DE-CIX non-unique ASNs of around 2,500 across the world.



@internetsociety

As a commercial exchange, Equinix doesn’t publish traffic statistics. So it’s not clear how much traffic is actually being exchanged under either its bilateral or multilateral peering services. However, considering its business and cultural links to the large U.S.-centric content and cloud providers, it’s easy to imagine that Equinix’s global traffic is substantial.

Entries in PeeringDB show several participants with port capacities of 200 Gbps and even 400 Gbps at multiple nodes. Equinix in Singapore has multiple large content providers with 400 Gbps capacity.

1995: Deutscher Commercial Internet Exchange (DE-CIX), Germany

With 83 million people, Germany has a population that sits roughly halfway between that of Thailand and Vietnam. The land area and density are also quite analogous, although Thailand has more land area for its similar-sized population. Overall, Germany’s population density sits at a low-to-average 232 persons per km2. A significant 30 percent of the land area is forested, and more than half is given over to agriculture.

There are plenty of large cities in Germany by European standards. Berlin has a population of more than three and a half million. There are another three cities over 1 million. Then Frankfurt, with three-quarters of a million people, rounds out the top five most populous cities.

It was in Frankfurt where the Deutscher Commercial Internet Exchange (DE-CIX), one of the world’s leading exchanges, was first located. It was in an old post office building back in 1995.

Germany

Frankfort

Economy Germany



Exhange operator DE-CIX

URL de-cix.net

Location Frankfurt

Established 1995


1,027


Prefixes 1,492,263



@internetsociety

Three ISPs built the exchange to avoid “tromboning” traffic to the United States. The exchange operator was formed as an industry association. As other peers were added, the exchange grew, moving into a commercial data center in 1998.

DE-CIX says it became something of a gateway to Central and Eastern Europe, although this alone can’t entirely account for the exchange’s growth. Neighboring Amsterdam has grown AMS-IX, an equally successful exchange, on basically the same model, and in London, LINX (London Internet Exchange) makes it three of the world’s largest IXPs that have grown out of Europe.

These early exchanges established the model of open, neutral, settlement-free peering facilities working for the interests of its members. This not-for-profit model is widely adopted in the developing world as it builds trust and infrastructure and fits well with capacity building programs. This “European model” contrasts with the for-profit exchange model we see emerging out of North America.

Of course, not all European IXPs are not-for-profit. No matter which metrics you choose, DE-CIX, AMS-IX, and LINX are the leading hubs in Europe and among the biggest exchanges worldwide. DE-CIX in particular is on a path to become a global exchange, with locations in 25 cities and growing.

Indian connectionOne of those locations is Mumbai, India. The exchange was once the Mumbai-IX and previously the Mumbai Convergence Hub. It is now a DE-CIX site following a joint venture with the original operator. The already successful Indian interconnection point was launched in 2014 as an open, neutral not-for-profit along the lines of a European exchange. Today, the Mumbai DE-CIX is approaching 300 ASNs, with traffic peaking over 1 Tbps across six sites in Mumbai.

An important policy that made it attractive to participants was that it allowed content and cloud providers to peer at a time when NIXI would not accept them. DE-CIX now runs four Indian exchanges in Mumbai, Chennai, Kolkata (Calcutta), and Delhi.

Regional EcosystemsFinally, an overview of each region demonstrates how the development of Internet exchange points varies considerably between them.

Europe has significant intercontinental bandwidth west to North America and southeast to the Middle East and on to the Asia-Pacific.32 However, internal international bandwidth between European neighbors is higher than anywhere else on the planet.33

32 “466 Tbps: The Global Internet Continues to Expand,” Telegeography Blog, https://blog.telegeography.com/466-tbps-the-global-internet-continues-to-expand

33 International Internet bandwidth Per capita: Countries Compared, https://www.nationmaster.com/country-info/stats/Media/Internet/International-Internet-bandwidth/Mbps/Per-capita



@internetsociety

North America is comparable. These two markets have more exchanges, with more participants and greater data volumes than the other three regions.

They both have highly developed, digital economies and competitive communications environments that offer the lowest domestic and international transit costs.

Room for more?Europe has the most IXPs per capita and continues to grow, launching at the rate of 34 in the past year (20 percent).34 Three European exchanges dominate the IXP ecosystem: DE-CIX (Frankfurt), AMS-IX (Amsterdam), and LINX (London).

In terms of IXP metrics, a number of Eastern European exchanges are beginning to catch up with these long-term leaders (notably Poland and Russia).35

Both DE-CIX and AMS-IX operate IXPs in multiple countries and have an increasing presence in Asia-Pacific. These IXPs are reaching out of their region, making their contacts and expertise available while also providing operational reliability for their ISP participants and for cloud and content providers in other markets.

While Europe has double the number of exchanges than any other region, the United States has more exchanges than any single economy.

There are as many Internet exchanges in the U.S. as in the whole Asia-Pacific.

There are more than 100 exchanges, including global data center operator Equinix and a good number of metropolitan or multi-metro commercial exchanges.

There are community-led open exchanges, of which the Seattle Internet Exchange (SIX) is the largest. SIX has peak traffic exchange rates of around 1.8 Tbps and more than 270 peers. Most often, though, North American exchange services are offered as an adjunct to either transit or data center services, rather than carrier- and data center-neutral IXPs.

Latin America has the stand-out IX.br, which in sheer numbers is the world’s largest exchange, with more than 2,000 ASNs and aggregate traffic peaking at over 13 Tbps at the São Paulo location.

Brazil is almost totally reliant on capacity running south from New York and Miami, but coastal Brazil is well served with landing stations. The permissive policies and subsidized fees at the exchange likely contribute to its success. The leading global cloud and content companies are well represented and have large bandwidth capacities in the 100s of Gbps.

Panama, Argentina, and Chile each have an exchange with 100-200 participants, and the Santiago PIT Chile location exchanges peak data volumes over 3.5 Tbps.

34 Internet Exchange Point Growth by Region, PCH, https://www.pch.net/ixp/summary_growth_by_region35 See PCH IXP Directory, https://www.pch.net/ixp/dir#!mt-filters=%7B%22reg%22%3A%5B%22dropdown%22%2C%22%22%2C%22Europe%22%

5D%7D!mt-sort=prts%2Cdesc!mt-pivot=prts



@internetsociety

Meanwhile, Africa continues to develop. West and South African nations reach the globe through Spain and Portugal, while Kenya in East Africa taps into the major routes running through the Middle East to Europe or Asia. Africa is home to more than 50. More than half are not-for-profit.36 Outside of South Africa, the exchanges are small in comparison with more developed markets but sufficiently reflective of the developing economies they serve.

Asia-PacificThe world’s first and second largest populations are found in nations located at either end of the Asia-Pacific. In between are an array of countries as diverse as it gets, from the world’s least developed countries to the world’s smallest populations. Population densities range from 3 to 8,000 people per km2.

The patchwork of economies and the distances involved seem to have obstructed the emergence of an international exchange operator equivalent to DE-CIX or AMS-IX. This is despite ongoing discussions about creating a sub-regional IXP.37

Consider that around the time HKIX was forming, Europe was giving birth to those exchanges in Frankfurt, Amsterdam, and London.

Subscriber market sizeAsia-Pacific accounts for around two-thirds of the world’s population and half of the world’s Internet users. Excluding Chinese broadband subscribers, the “rest-of-Asia” market is roughly the size of Europe’s 226 million broadband subscribers.

Consider that Japan has 96 million subscribers and Korea 21 million. The rest of Asia (more than fifty countries) accounts for only 80 million subscribers. This is less than Germany, France, and the United Kingdom combined.

Yet peak international traffic in the E.U. is 109,287 Gbps compared with either Asia at 43,574 Gbps or even the U.S. and Canada combined at 42,340 Gbps.38

Tokyo, Hong Kong, SingaporeTwo of Asia’s hubs for international traffic, Hong Kong and Singapore, have not-for-profit exchanges (HKIX and SGIX). The third major hub is Japan, where three carrier-led exchanges compete in the same market.

A new Asia-Pacific hub appears to be forming in Mumbai, India. While Singapore is a conduit to Indonesia, Malaysia, and others, Mumbai is a gateway to India’s 1.3 billion people. It is also strategically positioned along the cable routes west to Europe.

36 Benchmark Survey Report – 2018, African IXP Association (AFIX), https://www.af-ix.net/sites/default/files/AFIX%20-%202018%20Benchmark%20Survey%20Report.pdf

37 “A Study on the Costing, Operational Principles and Modalities of a Proposed South Pacific Internet Exchange,” Economic and Social Commission for Asia and the Pacific (ESCAP), https://www.unescap.org/resources/study-costing-operational-principles-and-modalities-proposed-south-pacific-internet

38 Global Internet Map 2018, Telegeography, https://global-internet-map-2018.telegeography.com/



@internetsociety

Could Mumbai become a fourth hub for International bandwidth39 and home to an exchange the size we see from the Netherlands, Germany, or Brazil?

The region is slow to change. While Europe’s IXPs grew by 20 percent in 2020/21, Asia, with its 104 IXPs, grew by only 5 percent in the same period.40

Key MessagesRather than focusing only on the usual metrics used to compare exchanges (number of ASNs, total port capacity, peak/average traffic), case studies provide an opportunity for deeper insight into how those metrics reflect the history, business model, current operations, and future directions of an Internet exchange.

Many, many factors influence the Internet ecosystem in a given region, economy, district, or city. As predicted by the IGF’s BPF, there is no single business model or formula an operator can adopt to ensure its success. What matters is whether the goals of the exchange operator can match with the goals of participants.

These case studies indicate that environmental factors play a significant role in the formation and effectiveness of a successful exchange.

This is the BPF’s “enabling environment.”

As technology and infrastructure improves, the barrier to entry is limited, making global reach and fast growth possible for any exchange operator.

A neutral exchangeThe majority of Asia-Pacific economies have at least one not-for-profit carrier- and data center-neutral public Internet exchange. These exchanges hold no allegiance to any transit provider or data center operator. Their decisions are made only to benefit the participants in the exchange.

This status doesn’t automatically confer leadership according to the usual metrics. Not-for-profit exchange operators must still compete with for-profit exchanges (many of which are also completely neutral).

Neutral for-profit exchanges have their specific benefits, as do data center operators, submarine cable landing stations, and transit providers.

Network operators and content providers peer at multiple places. They generally do this for reasons other than the institutional ownership of the exchange.

39 “Subsea cables and interconnection hubs - The interplay of diversifying routes and peering markets,” https://www.de-cix.net/en/about-de-cix/academy/white-papers/subsea-cables-and-interconnection-hubs-the-interplay-of-diversifying-routes-and-peering-markets

40 Internet Exchange Point Growth by Region, PCH, https://www.pch.net/ixp/summary_growth_by_region



@internetsociety

No guaranteesCommunity-driven membership style exchanges have been shown to work effectively in economies new to peering. They build trust among participants and are in environments where a commercial operator might not be able to achieve a profit.

If the domestic infrastructure is available and cheap enough to make peering cost effective, an Internet exchange can enjoy success. Nonetheless, exchange operators new and old must respond to their ecosystem’s individual needs.

Even not-for-profit exchanges are a business-to-business service. An operator must be able to demonstrate its benefits to participants. Peering managers/coordinators look for value, the cost to reach and participate at the exchange, and which parts of the Internet the exchange can reach for them.

To survive, an exchange must at least provide a reliable and efficient service, or operators will not use it, even if they are required to connect.

If the quality is as good, and the cost to participate is less than the cost to pay for equivalent transit, the operator will invest to connect and make use of the exchange.

The fundamentals still applyThe success of peering in any national ecosystem depends on many variables as described in the IGF BPF.

The number of models is almost as numerous as the individual ecosystems in which we find IXPs. While there is no one single model that suits all situations, neutrality does remain an important factor. A sign of a healthy ecosystem seems to be the number of exchanges complementing or competing with each other.

However, the case studies show that the reasons for operating or participating in an exchange are relatively universal.

Even as international IP transit costs and domestic transport costs trend downwards, an IXP offers an efficient, scalable, and low-latency alternative. IXPs provide traffic management options for service providers and can satisfy the increasing end-user demand for low latency applications.

As indicated in the IGF BPF, not-for-profit exchanges ultimately need to charge member fees (even the cleverly funded IX.br has introduced per-port fees for its largest participants). We saw this reflected in the experience of MMIX, which was quickly able to establish a fee model but is still reliant on donated time for operations.



@internetsociety

The right ecosystemIrrespective of business model, an exchange requires technical skill, participant trust, community engagement, and operational excellence to succeed. Achieving this relies on the availability of supportive aspects in the ecosystem, access to diverse infrastructure, a competitive service-provider market, and a capable workforce. All this must be supported by a regulatory environment that is conducive to open, settlement-free peering. Finally, you need an Internet-aware subscriber base to attract local or international content companies.

A government in an economy with no IXP might be well served to directly support the establishment of one. However, unless there is a willingness to commit long-term budgeted funding, plans need to be put in place to make the exchange self-sustaining and preferably self-governing. Uncertain or restrictive IXP policies or external telecoms regulations can hamper an exchange if government priorities are not conducive to flexible peering options.

An Internet exchange in an emerging competitive telecommunications market can sometimes be controversial as the value it creates for the user or the service provider is in part derived from the replacement of paid transit services.

Great attractorsIdeally, a successful exchange not only serves the ecosystem but also helps create it. Not-for-profit exchanges in particular act as opinion leaders and capacity builders. They often are closely affiliated with Network Operator Groups (NOGs), run their own training events and peering forums, and encourage best practice among the participants.

IXPs also attract critical infrastructure such as root servers and nameservers, time servers, research tools, and other facilities useful for Internet service providers.

The IGF BPF documents highlight the importance of locally created or locally hosted content, whether that is social media or e-government services. These case studies tend to confirm the importance of cloud and content companies. They show how content influences the ecosystem and the trajectory of an exchange.

The migration to cloud-based applications and the unrelenting growth in content consumption are driving the need for servers and storage closer to the customer. Network-savvy cloud and content companies carefully plan where to place infrastructure, so that it maximizes the quality of service ultimately delivered to consumers.

Where participants at an IXP reach only a small percentage of local users, it will be a less attractive investment for content providers to participate. Where a large proportion of subscribers can be reached from the IXP, it is more likely to attract content companies.

Finally, exchanges hopefully attract peers. Once an Internet exchange reaches a critical mass, the data center that hosts it becomes a natural co-location point.



@internetsociety

Reflections on best practice

The growing importance of the edgeThe goal of the first IXPs might have been to avoid domestic email messages from transiting international gateways. But we now live in an asymmetric era where the bulk of user traffic is generated by people watching TV or cat videos, online gaming, or accessing social-media sites.

Consider that 49 percent of Asia-Pacific websites are hosted in the U.S. and Canada. Compare that with 75 percent of websites hosted on home soil in North America, or 56 percent of European websites hosted in Europe. Asia’s preference for American hosting drives up international bandwidth demand while creating a market for CDNs.

Further, when you consider that only a handful of companies account for the vast majority of all Internet traffic, attracting a couple of video streaming services, a large CDN or two, and retail cloud service providers to an exchange will give its participants access to a large part of what its customers are doing online.

Hosting cloud and content services as close to the end user as possible improves the users’ quality of experience. The growth in high-bandwidth and low-latency applications is accelerating. This is driving caches and cloud further from the network hubs to the edges of the network.

Follow the data centers The emergence of edge application services has the effect of reducing the volumes of data that must be moved back to central hubs by placing the sources in progressively smaller cities or districts.

Like these content sources, Internet exchanges require data center conditions. Internet exchange operators must partner with a commercial data center operator or build their own.

Established best practice is for IXPs to partner with a data center operator rather than build and operate their own facilities, which can quickly create a burden on the exchange operations.

Even global cloud-based application providers have discovered it’s best to leave data centers to data center experts.

Commercial Internet exchanges might find it unprofitable to establish locations in these second-tier markets, in the same way that access providers find it unprofitable to service smaller communities.



@internetsociety

Not-for-profit exchanges might need to step up if future generations of low-latency applications are to spread beyond the major metropolitan areas (think smart cities and driverless cars).

They will need to find suitable data centers in smaller locations to do that. NIXI sees potential for hundreds of exchanges across India.

Once in a data center, the exchange typically attracts a host of useful and interesting applications and services around it. Exchanges can become not only a central hub of interconnection, but a central hub of content, processing power, applications, transit, and non-platform features such as security, capacity building, best practice, education, and support.

Starting or Scaling an IXPNot every Asia-Pacific nation has an existing IXP. Some might not see the need for one. Where there are only a small number of provider options, traffic exchange can be done at private peering locations. The introduction of cloud and content changes the equation as they need to be accessible to all end users. Any location where a competitive market has developed can host a viable or useful exchange. It then acts as a development tool.

A case study on BDIX is included in the 2016 IGF BPF document for good reason. It is an example of how Internet development organizations can assist in the formation of a national IXP. There are similar examples in the case studies here.

Many begin with stereotypical stories of association-led exchanges growing organically from donated equipment. However, Japan’s carrier-led, carrier-neutral exchanges show that other approaches work equally well in different environments. The government-led SGIX and NIXI have both contributed to their national Internet in different ways, as does Equinix.

We see from highly developed ecosystems that there is little restriction on the number of IXPs they can support. Many metropolitan areas are served by multiple sites of an individual exchange, but also by a growing number of alternative exchanges.

In some cases, competing not-for-profit exchange operators will enjoy significant success. In other cases, purely commercial exchanges built around transit provision or data center operations are predominant. There are even a number of highly specialized exchanges that serve ultra-small communities, or those that share an interest, activity, or some commonality other than geography. While these are public, they have a very singular appeal.

The cost to entry for a small exchange is already low. Remote infrastructure providers and SDN-based exchanges are changing operational approaches and are being effectively used by new market entrants.



@internetsociety

Initial stepsThe IGF BPF doesn’t provide definitive instructions for establishing a new IXP. Many of the examples point to a community-led approach. However, it recognizes that each exchange operator must respond to the context its ecosystem provides.

The need to build trust among competing service providers favors a community approach. This might be helpful at first, but once it becomes familiar and participants experience the benefits, they will begin to seek other opportunities to peer.

The question moves from “Do I peer?” to “Where do I peer next?” If it makes business sense to peer at a mid-sized commercial exchange operated by a Tier 2 transit provider, then the answer is clear.

The community-led exchanges in Asia-Pacific have well-established places in their ecosystem and bring supportive, capacity building services to the local community. While there is always a place for a not-for-profit Internet exchange with a cost-recovery model, it might not necessarily remain as the only exchange, or even the biggest.

SustainabilityOur case studies included examples of free, subsidized, and independent exchange operators, as predicted in the BPF. The free business model (IX.br is our closest example) is difficult to replicate sustainably. However, IX.br’s ability to offer free peering is certainly having an impact on how data is transported around Brazil.

To achieve a sustainable future, most not-for-profit IXPs will charge for their service based on a transparent cost-recovery model. Some also have access to external financial or in-kind support such as from a nongovernmental organization or a university.

A high proportion of European IXPs are owned and managed neutrally. They are carrier-neutral, data center-neutral, and open (participant-neutral). Other regions have shown this is not an absolute, although the further you depart from this, the less like an IXP you become.

Internet exchanges have the benefit of being an attractive customer to carrier-neutral data center operators. Exchanges can assist their sustainability by partnering with them to eliminate, or significantly reduce, their costs.

Grow and evolveOnce the exchange is financially stable and technically reliable, it can focus on adding features and services to attract more peers. Technical and operational challenges grow, but in a general sense, bigger is better when it comes to Internet exchanges.

The exchange needs service-provider participants, which in turn attracts cloud and content providers, but the cost and service quality need to match those from transit providers.



@internetsociety

Network operators would often rather pay transit than risk their customer’s dissatisfaction. Large cloud and content providers buy transit in bulk or build their own networks, so their cost base is typically attractive.

Other sources of new participants are often overseas, or at least remote. Dedicated layer 2 circuits managed by the exchange in between its own nodes, or to other exchanges, offer direct connection to distant networks.

As with the content cache fill operations, not-for-profit exchanges must fund such connections from per-port fees or charge additional fees for the advanced services.

KINX shows how an exchange operator can successfully evolve into a full-service colocation and transit provider while still maintaining its mandate to offer settlement-free peering in a regulatory constrained environment.

New prospectsTraditional bricks-and-mortar businesses are increasingly online and network aware. Moving their own operations to the cloud and their storefront to the screen, some of these organizations are already seeking better control of their connections and are appearing at Internet exchanges with their own circuits.

As more large enterprise networks secure their own address blocks and ASNs, they are potential new peers. This might lead to increased relevance for exchanges where other ecosystem elements permit. Enterprise networks will either join exchange points directly or, if they do not have their own Internet resources, turn to ISPs and IXPs to provide seamless cloud connectivity.

Fostering a supportive ecosystemThe basic goals and outcomes of an Internet exchange are easily met when conditions are right. Organizations agreeing to peer for their mutual benefit is an unremarkable circumstance once trust is established.

Despite this, threats to a successful exchange are numerous. While the policies and business model that an IXP adopts can influence its development, the overall environment must be conducive to peering. At the end of the day, Internet exchanges need a beneficial regulatory and infrastructure environment.

Poor infrastructure and/or expensive IP transport that makes it more difficult or costly for service providers to physically connect can hamper an exchange.

In some cases, the higher cost can sometimes be explained by geography. However, a lack of competition for metropolitan and long-haul transport options or limited and expensive international transit can make life difficult for operators to make use of an exchange.



@internetsociety

The BPF says IXPs are an incentive for local ISPs to build their own infrastructure to reach the exchange. However, not all Asia-Pacific economies have a licensing environment or are cost effective enough to make this a viable option. This forces them to rely on paid transport to reach an exchange.

While an IXP can contribute to a growing ecosystem, it is not enough on its own. Infrastructure sharing encouraged or enforced if necessary, can help make more options available. This can serve as an alternative to second-tier infrastructure development until those operators gather enough business strength to invest on their own or in partnership.

Reducing or eliminating licensing regimes, particularly in the area of interconnection, will remove barriers and allow operators to find the most effective and efficient solutions to their needs. If there is no room or regulatory support for small-to-medium providers, there will be little growth for the exchange, particularly in a consolidating market.

Content competitionA constrained exchange often finds it more difficult to attract international content partners. If dominant service providers do not participate at the exchange, content companies might choose to peer with them privately rather than at an open IXP where smaller ISPs could also peer with them.

Most national telecommunications markets in our case studies are dominated by three or four major providers for mobile, and often a smaller number of fixed Internet providers. These national operators might peer privately, and it will be difficult to attract content to a newly launched exchange.

Ideally, large service providers should be encouraged to join, or at least colocate with the exchange and privately peer there (with suitable participants), to improve the prospects for content sharing.

Adverse regulatory environments, such as we see in South Korea, might cause market disturbance that freezes further development and causes uncertainty amongst content players. Or if there is a lack of clarity, as we see in Kenya and Myanmar, IXP growth is difficult.

Economical internetworkingThese case studies demonstrate the importance of a favorable regulatory environment. Myanmar’s licensing delays increase uncertainty and impede the exchange’s ability to grow. They also prevent other organizations from launching competitive alternatives. If this problem is not resolved in a timely manner, the outcome could be that the exchange fails to gain enough momentum to reach sustainability.

However, once the benefits become clear to the Internet industry, there seems to be inevitable pressure for more exchanges. In the absence of adverse regulatory conditions,



@internetsociety

exchange point growth seems faster than basic metrics of population and penetration. This reflects the increasingly sophisticated network management techniques designed to contribute to an efficient and economical peering environment.

ConclusionThe case studies here reinforce many of the learnings from the IGF BPF. There are ideas here that can help any economy to develop its peering and interconnection ecosystem.

Rather than simply an interchange between local ISPs, the exchange is now increasingly important as an interconnect point between “eyeball” networks and the content and cloud monoliths that rely on them.

In that sense, focusing on any particular business model as a basis for understanding exchange points underestimates the variety and number of interconnection services offered in a growing Internet. Not-for-profit exchanges predominate our case studies but selecting one model over another is a decision influenced by the exchange operator’s goals and the realities of the local ecosystem.

Exchange relationshipsAs traffic exchange is required closer and closer to the edge, more exchanges might be needed in smaller cities and locations. Large for-profit exchanges might find this unprofitable.

If existing operators do not expand to fill the gap, new players, such as domestic transit providers, might evolve to meet the specific market needs. Or partnerships such as those in NZ might begin to emerge.

We might also see larger exchange operators partnering with smaller counterparts closer to the edge. This is reflected on a global level with the increased footprint being developed by DE-CIX.

A best practice for these inter-exchange relationships is still evolving and will likely take a few more years to become clear.

In practice, the policies an exchange adopts and the trusted relationships it can develop are a better indicator of success than institutional ownership.

ISPs in Japan are willing to pay for carrier-grade exchange points from a commercial IXP. In the same way, data center customers see the sense in leveraging Equinix’s global reach, reliability, and existing participant roster.

As important as the internal functioning of the exchange is the environment external to the exchange. An IXP might grow only as large as its geographical user base, the service providers that address that user base, and the infrastructure and regulatory environment in which it finds itself.



@internetsociety

Growth vectorsOther growth vectors are being pursued, including white-label operator services, as we see in the relationship between IX-Australia and NZ-IX, or NIXI’s vision of branded exchanges across South Asia. These initiatives mirror activities by AMS-IX and DE-CIX.

On another level, the increased prevalence of exchange eSeller services and remote peering offerings from exchanges can develop new solutions for service providers that allow them to aggregate transit and reach distant peers with little effort.

Until exchanges are built out in smaller regional areas, regional hubs for content and traffic exchange will continue to dominate, and established IXPs will continue to see demand for remote peering solutions.

The colocation of content and application servers, of name servers, and of time servers is just the beginning. Aggregation of traffic over shared transit links, remote peering, and other traffic-management opportunities will likely grow faster than transit.

Finally, the increasing importance of cloud infrastructure and application services across the corporate world is driving the enterprise toward the exchanges, looking for lower latency.

Either access providers will be able to meet the stringent demands on these services, or we could see more corporations acquiring ASNs and bypassing their transit providers to connect directly to the exchange.



@internetsociety

Economy IXP Name Launch Population Land Area Location Peers Avg. Traffic # Prefixes

Myanmar MMIX 2017 54,817,917 676,578 km2 Yangon 13 7.3 Gbps N/A

Thailand BKNIX 2015 66,534,684 513,115 km2 Bangkok 38 23 Gbps 14,594

Singapore SGIX 2010 5,703,569 683 km2 Singapore 167 493 Gbps 295,147

Bangladesh BDIX 2004 166,500,000 143,998 km2 Dhaka 120 44 Gbps 4,640

India NIXI 2003 1,312,240,000 3,287,263 km2 Delhi 32 39.3 Gbps N/A

Brazil IX.br 2002 210,147,125 8,514,877 km2 Sao Paulo 2116 6.2 Tbps N/A

Japan JPNAP 2001 126,265,000 377,873 km2 Tokyo 174 1.5 Tbps N/A

Korea (Rep. of) KINX 2000 51,337,424 99,538 km2 Seoul 64 274 Gbps 191,654

Kenya KIXP 2000 47,562,772 580,367 km2 Nairobi 53 14.9 Gbps 184,030

Australia IX Australia 1997 25,365,571 7,741,220 km2 Sydney 228 283 Gbps 137,458

United States Equinix 1996 327,167,434 9,629,091 km2 Palo Alto 92 41 Gbps 1,342,039

Hong Kong (SAR) HKIX 1995 7,507,400 1,099 km2 Hong Kong 333 1.1 Tbps 156,777

Germany DE-CIX 1995 83,039,099 357,022 km2 Frankfurt 1,027 7.1 Tbps 1,492,263

IXP Case Study Matrix

Note: Data in this table might reflect multiple sites (PoPs) of a single metropolitan exchange location (city). Many IXP operators have multiple other locations with multiple sites per location.



@internetsociety

Documents

28 May 2021 Effective IXP strategies for the Asia-Pacific