Using RIPE Atlas for network measurement and monitoring · Using RIPE Atlas for network measurement and monitoring ... network health (e.g. Nagios and Icinga) • These tools can

Using RIPE Atlas for network measurement and monitoring Sebastian Castro, NZRS (Material kindly provided by RIPE NCC) APRICOT, Auckland, February 2016

Source: RIPE NCC

•  Learn how to: Benefit from using RIPE Atlas measurements for network monitoring and troubleshooting Use API calls to create measurements Integrate RIPE Atlas with existing monitoring systems

•  Opportunity for hands-on practice

Goals

Auckland, Feb 2016 APRICOT: RIPE Atlas Tutorial 2

Source: RIPE NCC

•  I assume you have already used RIPE Atlas

•  Do you have a RIPE NCC Access account? •  If not - quickly create one: ripe.net/register

•  Do you have credits to spend?

•  If not - tell me to transfer some credits

Prerequisites


Source: RIPE NCC

•  What is your background? network operator? software engineer / programmer? data scientist? sysadmin? other? please specify :-)

Warm-up question


Source: RIPE NCC

• Introduction to RIPE Atlas • Creating measurements • Integration with network monitoring systems

• Real-time performance monitoring • Take part in the RIPE Atlas community • News:

• CLI Toolset • Additional slides

Overview


Source: RIPE NCC

Introduction to RIPE Atlas


Source: RIPE NCC

•  RIPE Atlas is a global active measurements platform

•  Goal: view Internet reachability

•  Probes hosted by volunteers Small probes Anchors

•  Data publicly available

Introduction


Source: RIPE NCC

RIPE Atlas coverage


Source: RIPE NCC

•  Ongoing global measurements towards root nameservers

Visualised as Internet traffic maps

•  Ongoing regional measurements towards “anchors”

•  Users can run customised measurements ping, traceroute, DNS, SSL/TLS and NTP

RIPE Atlas results


Source: RIPE NCC

• 9,200+ probes connected

• 5,000+ active users in the last quarter • 2,500+ results collected per second • 35,000+ customised measurements weekly

Numbers for January 2016


Source: RIPE NCC

Creating a Measurement


Source: RIPE NCC

•  A customer reports a problem: they cannot reach one of your servers You can schedule pings or traceroutes from up to 500 RIPE Atlas probes from a particular region to check where the problem might be

•  Measuring packet loss on a suspected “bad” link •  Testing anycast deployment •  Identify DNS cache poisoning/lies

Benefits of your own measurements


Source: RIPE NCC

• Running your own measurements cost credits ping = 10 credits, traceroute = 20, etc.

• Why? Fairness and to avoid overload • Daily spending limit & max measurements

user can create • Hosting a RIPE Atlas probe earns credits • Earn extra credits by:

Being a RIPE NCC member Hosting an anchor Sponsoring probes

Credits system


Source: RIPE NCC

Credits overview

My Atlas > Credits Give credits to someone


Source: RIPE NCC

•  Log in to atlas.ripe.net •  “My Atlas” > “Measurements” •  Three methods:

1.  Quick & Easy •  Type •  Target •  Done!

2.  Advanced GUI usage 3.  CLI scripting using API

Scheduling a measurement


Source: RIPE NCC

•  Mostly used for a periodic, long time measurement If just once, ASAP, choose “One-off”

•  Choose type, target, frequency, # of probes, region… Interactive interface helps you at each step

•  Each measurement will have unique ID •  “API Compatible Specification” is generated too

2: Using GUI to schedule a measurement


Source: RIPE NCC

•  Using command-line & scripting: Application Programming Interface (API) https://atlas.ripe.net/docs/measurement-creation-api/ https://atlas.ripe.net/keys/

•  You will need API keys To create measurements without logging in To securely share your measurement data

3: Using API to schedule a measurement


API documentation:

•  https://atlas.ripe.net/docs/measurement-creation-api/

https://atlas.ripe.net/doc/credits https://atlas.ripe.net/doc/udm •  https://atlas.ripe.net/keys/ •  https://atlas.ripe.net/docs/keys2/

18

Source: RIPE NCC

•  Go to “My Atlas” > “Measurements”

Looking up measurements results


Source: RIPE NCC

•  List of probes: sortable by RTT

•  Map: colour-coded by RTT

•  Seismograph: stacked multiple pings with packet loss

Available visualisations: ping


Source: RIPE NCC

Exercise Create a Measurement


Source: RIPE NCC

• Create a ping measurement: Involving ten probes To a target of your choice Source is your country Duration of two days

1.  Warm-up: Create a measurement using the GUI

2.  Create API Key 3.  Schedule a measurement using the API

Tasks


Source: RIPE NCC

•  Useful hint: once you generate a measurement, copy “API Compatible Specification” to text file

•  Note Measurement-ID

Sub-task 1: Use web interface


Source: RIPE NCC

•  Click on “Create an API Key” •  Choose type: “create a new user-defined measurement” •  “Object” is not applicable (N/A) for this type •  Give it a label •  Give it a duration of validity (leave empty for defaults) •  “Key” value to be passed on to the API call (next step)

Sub-task 2: Create API key


Source: RIPE NCC

•  Schedule a measurement using API Use the “key” you just generated Hint: copy and past API call syntax from the measurement generated by the GUI

•  Example: $ curl -H "Content-Type: application/json" -H "Accept:

application/json" -X POST -d '{ "definitions": [ { "target":

"ping.xs4all.nl", "description": "My First API Measurement",

"type": "ping", "af": 4 } ], "probes": [ { "requested": 10,

"type": "country", "value": "RS" } ] }'

https://atlas.ripe.net/api/v1/measurement/?key=YOUR_API_KEY

Sub-task 3: Use API


Source: RIPE NCC

Example with command line


Source: RIPE NCC

Integration with Network Monitoring Systems


Source: RIPE NCC

•  Network operators use tools for monitoring network health (e.g. Nagios and Icinga)

•  These tools can receive input from RIPE Atlas via the API

•  Benefits: pings from 500 out of 8,000+ probes around the world See your network from the outside Plug into your existing practices

Network monitoring


Source: RIPE NCC

1.  Create a RIPE Atlas ping measurement

2.  Go to “Status Checks” URL

3.  Add your alerts in Nagios or Icinga

Steps for integration


Source: RIPE NCC

•  Status checks work via RIPE Atlas' RESTful API https://atlas.ripe.net/api/v1/status-checks/MEASUREMENT_ID/

•  You define the alert parameters, for example: Threshold for percentage of probes that successfully received a reply How many of the most recent measurements to base it on The maximum packet loss acceptable

•  Documentation: https://atlas.ripe.net/docs/status-checks/

Creating status checks


Source: RIPE NCC

•  Community of operators contributed configuration code! Making use of the built-in “check_http” plugin

•  GitHub examples: https://github.com/RIPE-Atlas-Community/ripe-atlas-

community-contrib/blob/master/scripts_for_nagios_icinga_alerts

•  Post on Icinga blog: https://www.icinga.org/2014/03/05/monitoring-ripe-atlas-

status-with-icinga-2/

Icinga examples


Source: RIPE NCC

Exercise Setting up “Status Checks”


Source: RIPE NCC

•  Set up and configure a “status check” For an existing ping measurement https://atlas.ripe.net/measurements/3369314/ Hint: https://atlas.ripe.net/api/v1/status-checks/3369314/

•  Configure the status check in such a way that you will trigger an alert for this measurement

•  Optional: set-up status check for your own ping measurement!

Tasks


Source: RIPE NCC

•  One possible solution: Set the median RTT to a lower level: https://atlas.ripe.net/api/v1/status-checks/3369314/?median_rtt_threshold=10

•  Example of the alerts

Solution


Source: RIPE NCC

Real-time performance monitoring


Source: RIPE NCC

•  RIPE Atlas streaming is an architecture that allows users to receive the measurement results as soon as they are sent by the probes - in real time Publish/subscribe through web sockets

•  There are two types of data: Measurement results Probe connection status events

RIPE Atlas streaming


Source: RIPE NCC

•  Visualising network outages •  Server and performance monitoring •  In March 2015: used by almost all hackathon

teams: https://labs.ripe.net/Members/becha/ripe-atlas-hackathon-results

•  Documentationhttps://atlas.ripe.net/docs/result-streaming/ https://labs.ripe.net/Members/suzanne_taylor_muzzin/data-streaming-in-ripe-atlas

Real time streaming uses


Source: RIPE NCC

Probe (dis)connection events

https://labs.ripe.net/Members/andreas_strikos/amsterdam-power-outage-as-seen-by-ripe-atlas


Source: RIPE NCC

Exercise Using streaming API


Source: RIPE NCC

•  Scenario: customers are complaining that it occasionally takes a long time to reach your service or server

•  Action: ping your server from 500 probes Decide what is acceptable latency threshold to apply Notice and react when you start receiving samples

•  Task: Use the ping measurement ID 2340408 Choose which threshold (e.g. greater than 30ms) Imposes the threshold on “min” (the minimum result of the three ping attempts)

Monitoring server reachability


Source: RIPE NCC

1. Go to http://atlas.ripe.net/webinar/streaming01.html

2. Open the development console 3. Wait for results to arrive 4. Optional: Save the HTML file locally

and edit the code to your liking

Steps


Source: RIPE NCC

Page Source


Source: RIPE NCC

Example of results


Task 2: View (dis)connect events

See in the console of your browser the connection and disconnection events of all the RIPE Atlas probes Steps:

•  Create your empty HTML page •  Connect to the streaming •  Subscribe to stream_type: “probestatus”

44

Solution <script src="http://atlas-stream.ripe.net/socket.io.js"></script> <script> var socket = io("http://atlas-stream.ripe.net:80", { path : "/stream/socket.io" }); socket.on("atlas_probestatus", function(status){ console.log("I received ", status); }); socket.emit("atlas_subscribe", { stream_type: “probestatus" }); </script>

45

Source: RIPE NCC

The Newest Features LatencyMON


Source: RIPE NCC

LatencyMON


Source: RIPE NCC

•  If multiple targets are involved, the auto-grouping will be by target.

Adding multiple measurements


Source: RIPE NCC

•  You can search for any probe attribute •  You can specify a group name

Adding a group of probes


Source: RIPE NCC

Command Line Interface (CLI) Toolset


Source: RIPE NCC

•  Access RIPE Atlas from the terminal / shell console

•  Quick & dirty shortcuts for network troubleshooting

•  FLOSS (open source) tools written & maintained by RIPE NCC open for contributions by the community

•  Before you can use the toolset download the tools install configure

Why CLI RIPE Atlas tools


Source: RIPE NCC

1.  create a measurement 2.  generate a simple report abut ongoing measurement

3.  look at the results 4.  collect results from the ongoing measurement (streaming)

Use cases for CLI tools


Source: RIPE NCC

•  Simplest: one-off, using default values: (50 probes to “target”)

Creating ping measurement


ripe-‐atlas measure ping -‐-‐auth API_KEY -‐-‐target wikipedia.org

Source: RIPE NCC

• Geo-specific from 20 probes from Canada: ripe-‐atlas measure ping -‐-‐target example.com -‐-‐probes 20 -‐-‐from-‐country ca

• 20 Canadian probes that definitely support IPv6: ripe-‐atlas measure ping -‐-‐target example.com -‐-‐probes 20 -‐-‐from-‐country ca -‐-‐include-‐tag system-‐ipv6-‐works

•  Create a recurring measurement: ripe-‐atlas measure ping -‐-‐target example.com -‐-‐interval 3600

Other examples of ping


Source: RIPE NCC

•  Report for the traceroute • ripe-‐atlas measure traceroute -‐-‐probes 2 -‐-‐target google.ca

Traceroute


Source: RIPE NCC

Searching for existing measurements


ripe-‐atlas measurements -‐-‐af 6 -‐-‐status ongoing -‐-‐limit 15 -‐-‐search google

Source: RIPE NCC

Searching for very specific probes


ripe-‐atlas probes -‐-‐asn 3333 -‐-‐field id -‐-‐field asn_v6 -‐-‐field country -‐-‐field is_public -‐-‐field description -‐-‐field status

Source: RIPE NCC

•  Source: https://github.com/RIPE-NCC/ripe-atlas-tools/

•  Documentation: https://ripe-atlas-tools.readthedocs.org/

•  How to contribute: https://github.com/RIPE-NCC/ripe-atlas-tools/blob/master/CONTRIBUTING.rst

Links


Source: RIPE NCC

Take part in the RIPE Atlas community


Source: RIPE NCC

•  Individual volunteers host probes in homes or offices

•  Organisations host RIPE Atlas anchors

•  Sponsor organisations give financial support or host multiple probes in their own networks

RIPE Atlas community (part 1)


Source: RIPE NCC

•  Ambassadors help distribute probes at conferences, give presentations, etc.

•  Developers contribute free and open software •  Network operators create measurements to

monitor and troubleshoot •  Researchers and students write papers

RIPE Atlas community (part 2)


Source: RIPE NCC

•  https://atlas.ripe.net & http://roadmap.ripe.net/ripe-atlas/

•  Users’ mailing list: [email protected] •  Articles and updates: https://labs.ripe.net/atlas •  Questions and bugs: [email protected] •  Twitter: @RIPE_Atlas and #RIPEAtlas

Contact us


Source: RIPE NCC

https://atlas.ripe.net/docs/rest/ https://github.com/RIPE-NCC/ripe.atlas.sagan

https://atlas.ripe.net/docs/measurement-creation-api/ • https://atlas.ripe.net/doc/credits • https://atlas.ripe.net/doc/udm

https://atlas.ripe.net/keys/ https://atlas.ripe.net/docs/keys2/

Documentation


Source: RIPE NCC

Additional slides


Source: RIPE NCC

Finding results of public measurements


Source: RIPE NCC

•  There are many measurements already running!

•  Search for existing public measurements first

•  Schedule your own measurement if you don’t find what you’re looking for

Use existing measurements


Source: RIPE NCC

•  Log in to atlas.ripe.net •  Use your RIPE NCC

Access account •  Same account for LIR

Portal, RIPE Atlas, RIPEstat, RIPE Labs...

•  Create an account if you don’t have one already

Logging in


Source: RIPE NCC

•  Go to “My Atlas” > “Measurements”

Looking up measurements results


Source: RIPE NCC

•  List of probes: sortable by RTT

•  Map: colour-coded by RTT

•  Seismograph: stacked multiple pings with packet loss

Available visualisations: ping


Source: RIPE NCC

Seismograph tips Vertical pattern: possible problem in your network

Out of each 3 pings max, min & average are calculated

Filtering & grouping by: country, IPv4 ASN, IPv6 ASN

Chose absolute or relative RTT values

Red = packet loss

Horizontal pattern: possible problem in that ASN network


Source: RIPE NCC

•  List of probes, colour-

coded number of hops •  Map •  Traceroute paths

map, geolocation using OpenIPMap: https://github.com/RIPE-Atlas-Community/openipmap

Available visualisations: traceroute


Source: RIPE NCC

•  Map, colour-coded response time or diversity

•  List of probes, sortable by response time

Available visualisations: DNS


Source: RIPE NCC

•  Documentation for analysing measurements results:

https://atlas.ripe.net/docs/rest/ https://github.com/RIPE-NCC/ripe.atlas.sagan

•  More tools: https://github.com/RIPE-Atlas-Community https://github.com/RIPE-Atlas-Community/ripe-atlas-community-contrib/blob/master/README.md

Documentation


Source: RIPE NCC

Exercise Analyse Measurement Results


Source: RIPE NCC

•  Download results of a specific public measurement

•  Read the text of the result, to understand structure

Tasks


Source: RIPE NCC

•  Find the measurement ping, IPv6 to google.com msm-ID 1004005

•  Click on measurement, then “Download” Specify the time period (for example, YESTERDAY)

•  Results in JSON

Task 1: Download measurement results


Source: RIPE NCC

•  Solution URL: https://atlas.ripe.net/api/v1/measurement/1004005/result/?start=1435104000&stop=1435276799&format=json

•  Save the measurement(s) locally $ curl https://atlas.ripe.net/api/v1/measurement/1004005/result/?start=1435104000&stop=1435276799&format=json > measurement-test.json

Tips for downloading results


Source: RIPE NCC

Packet loss: difference between sent & received!

•  [{“af":6,"avg" 61.32,

•  ”dst_addr":"2a00:1450:4004:802::1014",“dst_name":"www.google.com",

•  “dup":0,

•  “from":"2001:8a0:7f00:b201:220:4aff:fec5:5b5b",

•  “fw":4660,“lts":411,

•  “max”:62.148,"min":60.372,

•  “msm_id":1004005,"msm_name":"Ping",

•  "prb_id":722,"proto":"ICMP","rcvd":10,

•  “result":[{"rtt":62.148},{"rtt":61.437},{"rtt":61.444},{"rtt":61.448},{"rtt":61.794},{"rtt":61.533},{"rtt":60.372},{"rtt":60.373},{"rtt":61.384},{"rtt":61.267}],

•  “sent":10,"size"64,

•  “src_addr":"2001:8a0:7f00:b201:220:4aff:fec5:5b5b",

•  "step":240,"timestamp":1410220847,"ttl":54,"type":"ping"},

Task 2: Look at the result Destination (IP & name)

Reference (msm ID)

Source (probe public IP address)


Source: RIPE NCC

•  Find out how many times RTT was above 60ms Use Python o Javascript or something else

•  For the Javascript solution, you can use this as a starting point:

https://stat.ripe.net/widgets/demo/script_me.html

Task 3: Analyse results (optional)


Source: RIPE NCC

Python:

Parse json and find total avg:import json f = open(“measurement.json”,”r”) measurements = json.load(f) for m in measurements: for r in m[“result”]: rtt = r[“rtt”]

If rtt > 60: i += 1

i must be > than 14563.

Task 4: Examples of code Javascript:

var dataAPIUrl = "https://atlas.ripe.net/api/v1/measurement/1004005/result/?start=1410220800"; jQuery.ajax({

url: dataAPIUrl, error: function() {

alert("error"); },

success: function( response ) { var i = 0;

for ( var i = 0, n = response.length; i < n; i++) { var measurement = response[i];

for ( var j = 0, m = measurement.result.length; j < m; j++) { var rtt = measurement.result[j].rtt; console.log(rtt); if (rtt > 60)

i++; }

} jQuery("p").html("The RTT has been above 60ms for " + i + " times");

},

dataType: "jsonp" });


Source: RIPE NCC

RIPE Atlas anchors


Source: RIPE NCC

•  Well-known targets and powerful probes Regional baseline and “future history”

•  Anchoring measurements Measurements between anchors 200 probes targeting each anchor with measurements Each probe measures 4-5 anchors Vantage points for DNSMON service

•  130+ RIPE Atlas anchors

RIPE Atlas anchors


Source: RIPE NCC

https://atlas.ripe.net/results/maps/network-coverage/#anchors

Locations of anchors


Source: RIPE NCC

IXP country Jedi Measuring Impact of IXPs on Keeping Traffic Local


Source: RIPE NCC

•  Operators Routing and traffic optimisation

•  IXP operators Shows how IXPs help keep traffic local and regional

•  IPv6 advocates Comparing IPv4 and IPv6 paths

•  Country level: regulators, politicians, cyber-security… How much traffic stays within the country? Where do the paths go? Comparing countries with each other

Benefits (part 1)


Source: RIPE NCC

•  RIPE Atlas community More probes in more networks = higher quality of measurements data

•  Geolocation data community Use case for improving data quality

•  Examples: https://labs.ripe.net/Members/emileaben/measuring-ixps-with-ripe-atlas https://labs.ripe.net/Members/emileaben/measuring-countries-and-ixps-in-the-see-region http://sg-pub.ripe.net/emile/ixp-country-jedi/CL+AR-2015-04/geopath/

Benefits (part 2)


Source: RIPE NCC

http://sg-pub.ripe.net/emile/ixp-country-jedi/SI-2015-04/ixpcountry/

Paths going via an IXP?


Source: RIPE NCC

•  Green: “good”, as far as we can see it •  Not a judgment, only

one way of visualising data

•  Red or blue: path is going out of country •  If this is a surprise: talk

to your upstream(s) •  Yellow: path is not going

via a local IXP •  If this is undesired:

make a new peering agreement

Interactive diagnostic tool

http://sg-pub.ripe.net/emile/ixp-country-jedi/SI-2015-04/ixpcountry/


Source: RIPE NCC

•  traceroute measurements using RIPE Atlas probes

•  Steps: Identify ASNs in the country using RIPEstat Identify IXPs and IXP LANs using PeeringDB Construct mesh: from all (*) country’s probes to each other

*Maximum of two probes per ASN and only “public” probes with “good” geolocation •  Hops geolocated using “OpenIPMap” database

Method


Source: RIPE NCC

•  Use this tool to find possible suboptimal routing and fix it Find your ASN in the mesh Find the person from another ASN Take them out for tea :)

•  To improve accuracy of this diagnostic tool If your ASN is not on the graph, apply for a RIPE Atlas probe Add more probes to your country to increase “resolution” If you move, remember to update your probe’s geolocation

Actions (part 1)


Source: RIPE NCC

•  Re-use and rewrite the code: it is free and open source software https://github.com/emileaben/ixp-country-jedi

•  Improve infrastructure geolocation: contribute data to OpenIPMap! https://marmot.ripe.net/openipmap/ https://github.com/RIPE-Atlas-Community/openipmap

Actions (part 2)


Contact: www.nzrs.net.nz

[email protected]

Want to host a probe?

Documents

Using RIPE Atlas for network measurement and monitoring · Using RIPE Atlas for network measurement and monitoring ... network health (e.g. Nagios and Icinga) • These tools can