IX SIG
Minutes
Thursday 2 March 2006, Perth Convention and Exhibition Centre (PCEC), Perth, Australia
Meeting commenced: 2:00 pm
Chair: Philip Smith
The Chair thanked the sponsors and introduced the SIG and explained the agenda.
Contents
- Open action items
- The routing aggregation policy: a failed social experiment at the LINX
- Introduction to Euro6IX project
- Euro-IX update
- IXP panel discussion - world IXP forum: testing the waters
- Exchange point operational experiences
- JPNAP update
- NPIX update
- NIXI update
- Equinix Sydney update
- JPIX update
- NetNod update
- AMS-IX update
- BBIX update
Open action items
- None.
The routing aggregation policy: a failed social experiment at the LINX
- It was noted that some of the badly aggregated networks were in that state because of poor configuration. It was suggested that subtle reminders could be very useful in improving routing. However, it was also noted that the CIDR report has already been doing that for many years.
- It was clarified that LINX was only aiming at their members' routes, not those of the members' customers.
- There was a discussion about traffic engineering and the effect this might have had on those who resisted the policy. It was explained that LINX never fully understood the reluctance of the opposing members.
- None.
Introduction to Euro6IX project
- It was argued that the model discussed puts the IX in the role of an ISP, which is not ideal and not likely to last.
- It was suggested that LIRs wanting to use IPv6 should get allocations from the RIRs and not seek these services from an IX. However, the presenter reported that another experiment in Canada based on this model has been working well. It was also suggested that some big European operators are preparing business models along these lines.
- It was argued that this model is an IPv6 multi-provider deployment test bed, but not actually an Internet Exchange. It was suggested that the lessons from this model could be very useful to report at a future IPv6 SIG rather than an IX SIG.
- The presenter noted that an important lesson from the model is that it is not possible to predict from where traffic will be returned.
- It was suggested that it is bad to be pushing the concept of a layer 3 exchange point.
- None.
Euro-IX update
- There was a question about the benchmarking club. The presenter explained the various factors recorded there.
- The presenter explained the current membership fees.
- None.
IXP panel discussion - world IXP forum: testing the waters
- BW explained that in North America, there has been a stark division between commercial and non-commercial exchanges, which has led to odd forms of collaboration. So in many circumstances, commercial operators pitch their services to peering coordinators. This has consolidated into week-long cruise ship meetings.
- It was noted that the job of peering coordinators is to meet with each other and interact. It was also noted that the cruise actually works out cheaper than holding the same meeting in a hotel.
- BW explained that there is no serious discussion of IXs in the ARIN and NANOG meetings in the way that there is at the other RIRs. He suggested that issues of corporate secrecy and competitive nature of business in the US means that there is not the level of collaboration that happens elsewhere. He also suggested that the US is not at the cutting edge of peering technologies and practices.
- GV explained that in Latin America, there is an informal group called NAPLA, which has been meeting since 2001 to share IX experiences and review procedures and services. There are now moves to establish a permanent forum. NAPLA and LACNIC will be meeting in May to discuss this. There will also be discussion of a regional interconnection project at that meeting. He explained that NAPLA is very keen to learn from other experiences to avoid repeating mistakes that have been made elsewhere.
- AA explained that the situation in Africa is similar to that in Latin America, but the legacy of early Internet in Africa makes the situation more difficult. There were problems of monopolistic telcos which tried to eliminate small ISPs, but this is being reversed. Africa has problems with very high international transit costs. There are now 10 IXPs in Africa and there are several more being formed. There is an African ISP association which is also encouraging IXP formation. AA noted that the WSIS process also has helped many countries to start looking at the Internet in new ways. AfriNIC recently took the decision to allow free address assignments to IXPs. AfriNIC is also discussing IXP issues with AfrISPA. There are intentions to set up local exchange points to help with international transit costs.
- KL explained that Europe was the first region to really develop IXP collaborations. He noted the role of Euro-IX in encouraging collaboration and interaction. He noted that there is also a lot of informal exchange of information between different IXPs and operator forums. There has also been a lot of staff exchanges and sharing of common resources.
- GU explained that the IXP situation varies across the Asia Pacific region. In general, there is not a lot of active collaboration across the region. However, the IXP and peering sessions at APRICOT and APNIC meetings are increasing the level of collaboration. There seems to be good collaboration between Japan and Korea related to switching fabric. He also discussed specific examples about HK and SG. Outside of these areas, there are several non-commercial IXPs. GU noted that many countries in Asia have monopoly ISPs that provide IXP services. He noted that APJII has been successful in creating a good IXP for Indonesia.
- The presenter noted that there now appear to be about 260 IXPs worldwide.
- There was an expression of thanks for arranging this session and encouraging more cooperation between IXPs. It was suggested that it is advantageous for IXP operators to collaborate because they generally don't compete with each other and are often geographically removed. Also, non-profit IXPs don't have to worry about competition.
- It was noted that there is a problem with creating too many IXPs in that discussions can then begin to localise and practical barriers to collaboration may follow. It was suggested that encouraging people to publish experiences in an accessible way would be very useful.
- It was noted that in this region, language diversity can be a real barrier to collaboration. It was noted that this can be a problem in Europe as well.
- None.
Exchange point operational experiences
- None.
- None.
JPNAP update
- There was a question about separating traffic counts. JPNAP is currently working on this.
- None.
NPIX update
- None.
- None.
NIXI update
- There were clarifications of the member numbers. There are now 51 participants. The presenter explained that there are about 19 in Mumbai, about the same in Delhi, and around 11 in Chennai. More details of this are available on the NIXI web site.
- None.
Equinix Sydney update
- The interconnection between states is shaped to 100 Mbps.
- The tariff for extended peering is still being determined.
- There was a question about preventing abuse of wide area networks between states. There is no system in place at this point to prevent this, although it will be considered.
- None.
JPIX update
- There was a question about any commonalities between JPIX and JPNAP. It was suggested that it is likely, though they are not double counting. It was noted that there are 28 ASNs that appear at both JPIX and JPNAP.
- JPNAP is using SQL backend. It was noted that disk I/O can cause problems with this. However, the presenter noted that it is performing well for JPNAP. The sampling rate is important for this.
- None.
NetNod update
- NetNod connection fees include the fibre and both ports.
- In Stockholm, the fibres are redundantly routed.
- None.
AMS-IX update
- None.
- None.
BBIX update
- None.
- None.
Nigel Titley, Flag Telecom
This presentation discussed the experience of the LINX mandatory routing policy. LINX is the London Internet Exchange, with 205 members and 90Gb peak traffic, founded in 1994. The presenter explained the organisation of LINX, which is a limited company, in which each member holds a one pound share. The company is bound by the articles of association and a memorandum of understanding (MoU). All members must comply with the MoU.
There are two social responsibility clauses in the MoU: all routing policy must be publicly registered and routes should be aggregated as far as possible. The aggregation clause was not regularly checked. There was a feeling that it should either be made enforceable and regularly checked, or it should be removed.
The presenter noted that the routing table is growing rapidly as people do not aggregate properly. At the LINX48 meeting there was a decision to introduce a mandatory, enforceable route aggregation policy. This was duly formulated. Checking then began and LINX began alerting members that they may be in breach and publishing a list of members in breach. At first this process did result in substantial improvements. However, some large ISPs noted that they had very large blocks and didn't want to announce the entire prefix, so adjustments were made to the requirements. Euro-IX provided a platform for spreading awareness of this policy.
The final result was that many members improved their aggregation, as many did not realise how fragmented their routing had become. However, there was a significant member backlash from large members (mostly US-based) which argued that an IX had no right to tell them how badly aggregated they were. At the LINX50 meeting, the members voted to remove the mandatory aggregation policy.
Although the policy is generally considered to be a failure from one view, it did have some positive outcomes. An online check will soon be available for networks to check whether they meet the best current practice on aggregation.
Questions and discussion
Action items
Jordi Palet, Consulintel
Presentation [pdf]
This presentation described a project that has been underway for several years, with the intention of pushing the deployment of IPv6 in Europe. The project involved network design and deployment, research of advanced services, development of applications, and other related work. The project has been active in disseminating experience all around the world. The participants were generally the research arms of many companies, but some commercial operations were involved as well.
The project received European Commission funding but also got considerable support from industry and educational sectors. The project also resulted in the publication of a book dealing with necessary legal and regulatory considerations.
The presenter displayed and discussed the network map for the project. The main idea of the project was to have infrastructure to provide interconnection services for layer 2 and layer 3. There was an experiment to allow IXs to make direct assignments from their own prefix to the customers.
The presenter discussed RFC 2374, which puts IXs in an intermediation role between ISPs and customers. The presenter demonstrated how the address assignment mechanism can work. He also showed how the routing can be achieved. Under this model, the IX becomes a place where new services can be offered to users. The presenter suggested this could be seen as a way to create a virtual ISP.
The presenter also mentioned the UK6X exchange, which is an example of this model and was the first in the UK to offer IPv6 services. He described the architecture of this exchange.
The presenter reviewed some other advantages of this model, such as cost savings. However, there are also disadvantages stemming from competition issues.
Questions and discussion
Action items
Serge Radovcic, Euro-IX
The presenter explained the background and operations of Euro-IX, which was formed in 2001 and now accepts members world-wide. It is not an exchange point itself, but is an association of exchange points. There are now 37 affiliated IXPs in 23 countries.
The presenter noted that the aggregated peak traffic of European member IXPs is just over 500Gbps and is continually rising. The 37 members combined have 2,520 connectees. Euro-IX has a variety of web-based tools that can be used to query IX information. Euro-IX has a benchmarking club and other information repositories, such as maintenance diaries and a switch database.
The presenter gave a detailed breakdown of the members, which is included in the presentation. Euro-IX holds forums twice per year. The next forum will be in Dublin in May 2006. Euro-IX is considering taking a more active role in public affairs.
Questions and discussion
Action items
Serge Radovcic, Euro-IX
Presentation [pdf]
The presenter noted that he had brought together a panel to spark discussions and share IXP experiences from around the world and increase collaboration.
The panel members are: Kurtis Lindquist, Bill Woodcock, Adiel Akplogan, German Valdez, Gaurab Raj Upadhaya.
Questions and discussion
North America
Latin America
Africa
Europe
Asia
Action items
[Break 3:35 pm - 4:00 pm]
The Co-chair chaired this session.
Stephen Baxter, Pipe Networks
Presentation [pdf]
This presentation gave an overview of the operational experiences of Pipe Networks which runs peering points across Australia. Pipe is a an MPLA exchange with VLAN access and ATM. Pipe uses route servers for routing updates and has written its own operational software. Pipe is a commercial operator and has an SLA for its service offerings.
The presenter noted that most people use Pipe's services for transit avoidance. Many customers also use Pipe to source primary or secondary Internet access. He noted that there is now a lot of layer 2 forwarding resale, which is an interesting use of the peering fabric. He noted that there has been a lot of business consolidation in Australia since Pipe started operations. He noted that usage of Usenet services has been going down.
The presenter described the basic architecture of the network. He noted that its routing table is about four percent of the global routing table. Pipe have not deployed route flapping as the table has been quite stable. He described the routing and switching equipment used. He also describe how they manage routing. He noted that they have separate ASNs at their different peering points.
The presenter described some of the strange behaviour that they are observing. In particular, he warned that ISPs should generally turn off "ip verify unicast reverse-path". He suggested that symmetrical routing solves many issues. He also noted that the BGP routing algorithm usually makes poor commercial choices. He noted several other common problems.
Questions and discussion
Action items
Toshinori Ishiii, JPNAP
Presentation [pdf]
This presentation discussed the operation of the JPNAP Layer 2 IX. Multifeed was one of the first Internet data centres in Japan and started JPNAP in 2001. It provides several special services, including aggregation services, traffic engineering services, and a BGP peer traffic graph. It provides services in Tokyo and Osaka.
The presenter gave details of the JPNAP service history, which are available in the presentation.
The peak traffic reaches 105.6Gbps, making JPNAP one of the biggest IXPs in the world. Forty percent of users are using 10 Gb interfaces now. JPNAP now offers an "Optical Switch" back-up system. JPNAP recently started IPv4/IPv6 dual stack services.
JPNAP holds closed user meetings every four months.
Questions and discussion
Action items
Gaurab Raj Upadhaya, NPIX
This presentation discussed the operation of the NPIX layer 2 IXP in Kathmandu. He described the recent member connections and the problems that they experienced with Fibre. NPIX now has a new building space to ease this problem. NPIX has conducted training on BGP and multihoming, which has finally eliminated static routing.
The TU foundation in Sweden has funded the installation of an I-root server.
The presenter noted that NPIX sees about 15 Mbps of traffic. There was a Nepali version of Linux released last year, which created very high traffic flows.
The presenter gave an overview of the routing statistics, noting that there are still two providers doing longer prefixes than /24. All members are now using BGP and are no longer using static routes. There are 11 ASNs visible at NPIX.
NPIX has its own transit LAN now. This was necessary for MRTG and netflow projects. It also makes it much easier to add services. The presenter noted some of the plans for the next six months. There have been various requests for a back-up site and this will be considered at the next meeting.
All international traffic from Nepal goes via satellite and this is very expensive.
Questions and discussion
Action items
Puneet Tiwari, ISPAI/NIXI
This presentation discussed the strategy and operation of NIXI. The presenter noted the demographic environment of the Indian Internet. There is a very large and growing potential audience that will be joining the internet in the coming years. The government has recognised the need for convergence. The presenter explained the vision and mission of NIXI, to develop world class infrastructure for all of the Indian Internet community.
NIXI has two main projects, the IXP and the administration the .IN domain. NIXI is operating in Mumbai, Delhi, Chennai, and Calcutta. The presenter explained the road map for NIXI's future activities, which is detailed in the presentation.
NIXI is a non-profit body; however, it has been proposed to commercialise it.
Questions and discussion
Action items
Akito Kurokawa, Equinix
Presentation [pdf]
This presentation discussed the operation of the Equinix's Sydney exchange, which started in 2003. It now has 18 participants, including domestic and international carriers and ISPs. It is a neutral IX, which also provides proxy peering for networks without their own AS.
Equinix now has about 120 Mbps of traffic, with some recent pronounced peaks.
The presenter explained the extension of the exchange to Melbourne and Brisbane. This will enable interstate peering by MLPA. It is planned to be launched in March 2006. There are leaf nodes going to Melbourne and Brisbane, which are L2 switches terminating in L3 switches in Sydney.
Questions and discussion
Action items
Takejiro Takabayashi, JPIX
Presentation [pdf]
This presentation discussed the operation of the JPIX exchange point. In 1997, a need was identified for a commercial IX. JPIX was formed to offer a collocation service. The presenter described the structure of the organisation. JPIX offers remote access by various methods to the data centre. The collocation service is rack based.
JPIX has started a free, experimental IPv6-IX service, which is currently being used by 16 networks. JPIX has other optional services, including a route confirmation service, a route server for exchanging routes, an NTP server, a net news server, traffic statistics graphs, and sFlow traffic analysis services.
JPIX services are now distributed across various metropolitan and regional sites. The presenter displayed some network configuration details.
Traffic is growing very rapidly. The current traffic volume is 64Gbps. JPIX has 70 10GbE customers.
Questions and discussion
Action items
Kurt E Lindqvist, NetNod
Presentation [pdf]
This presentation discussed the operation of the NetNod exchange point operator in Sweden. It was created in 1996 to create a resilient and independent exchange infrastructure and a single national exchange of traffic. The design was influenced by concerns of national security.
The location choice was relatively complex. NetNod uses various telecommunications bunkers which were built by the military as part of national critical infrastructure. NetNod now operates in five cities. Sweden is a geographically long country, so local traffic exchange was considered to be a desirable goal.
Traffic exchange patterns have been changing and much more local exchange takes place. Peer-to-peer networking may be influencing this. The locations of the actual bunkers are kept secret. The facilities are built for resilience and are kept simple with simple switching needs.
Besides running IXPs, NetNod provides other critical national infrastructure services, including Swedish time servers, I-root services (now in 30 anycast locations around the world), .SETLD services, and a number of TLDs. NetNod also provides various bandwidth measurement services.
NetNod has 36 customers and 40Gbps peak traffic, aggregated from all cities.
Questions and discussion
Action items
Cara Mascini, AMS-IX
This presentation discussed the operation of the Amsterdam Internet Exchange, AMS-IX, which is a not-for-profit association founded in the early 1990s. In 1997, a company was formed to operate the exchange. All members hold an equal share.
The presentation contains details of port and member statistics. AMS-IX generally has linear growth of members and ports, but with exponential traffic growth, doubling every 10 months. In 2004, AMS-IX launched 10GbE services, which have been under heavy demand.
The presenter displayed the network topology, drawing attention to the photonic switches to connect the 10G switches and automatic fail-over mechanisms.
AMS-IX has peak traffic of 135Gbps. Traffic volume has grown rapidly lately.
AMS-IX recently surveyed members. They found that existing members stay connected because of the level of cost savings. The performance of the NOC team rated well, as did other platform performance parameters (as measured by the RIPE TTM project). The survey did find, though, that AMS-IX needs to improve reporting. The presenter noted that most customers find it fairly smooth to set up peering. Most parties have an open or semi-open peering policy. About 64 percent of participants peer with more than 100 ASNs. Most networks are connected to an average of four exchanges.
The presenter briefly described AMS-IX's current developments. It has recently set up a route server, although this has not been used much yet. There are many developments in areas such as VoIP and ENUM services and other services such as streaming.
Questions and discussion
Action items
Masaru Akai, BBIX, Inc.
Presentation [pdf]
This presentation discussed the operation of BBIX, a relatively new IX in Japan. BBIX is a regionally decentralised IX, to encourage regional traffic to be exchanged without the respective region. BBIX guarantees unconditional peering and can provide traffic exchange to about 5 million users in Japan. The presenter displayed the traffic measurements for the various nodes.
Questions and discussion
Action items
Meeting closed: 5:35 pm
Minuted by: Gerard Ross
Open action items
- None.
