Lecture07 - UHjohnsson/COSC4377_2013/Lecture07.pdfprogramming with UDP and TCP Slide ... FTP client FTP server ... – Can greatly reduce network load – Client must also support

9/21/2013

1

Lennart Johnsson

2013‐09‐17COSC4377

COSC4377

Lecture 7

Lennart Johnsson

2013‐09‐17COSC4377

2

Chapter 2: outline

2.1 principles of network applications– app architectures

– app requirements

2.2 Web and HTTP

2.3 FTP

2.4 electronic mail– SMTP, POP3, IMAP

2.5 DNS

2.6 P2P applications

2.7 socket programming with UDP and TCP

Slide from Kurose & Ross, 6th Ed

9/21/2013

2

Lennart Johnsson

2013‐09‐17COSC4377

3

FTP: the file transfer protocol

file transferFTP

server

FTPuser

interface

FTPclient

local filesystem

remote filesystem

user at host

transfer file to/from remote host client/server model client: side that initiates transfer (either to/from remote) server: remote host

ftp: RFC 959, October 1985, http://tools.ietf.org/html/rfc959 RFC 2228, June 1997, security ext., http://tools.ietf.org/html/rfc2228 RFC 2428, September 1998, IPv6, http://tools.ietf.org/html/rfc2428

ftp server: control - port 21, data – port 20Slide derived from Kurose & Ross, 6th Ed

Lennart Johnsson

2013‐09‐17COSC4377

4

FTP: separate control, data connections

• FTP client contacts FTP server at port 21, using TCP

• client authorized over control connection

• client browses remote directory, sends commands over control connection

• when server receives file transfer command, serveropens 2nd TCP data connection (for file) to client

• after transferring one file, server closes data connection

FTPclient

FTPserver

TCP control connection,server port 21

TCP data connection,server port 20

server opens another TCP data connection to transfer another file

control connection: “out of band”

FTP server maintains “state”: current directory, earlier authentication


9/21/2013

3

Lennart Johnsson

2013‐09‐17COSC4377

5

FTP commands, responses

sample commands:

• sent as ASCII text over control channel

• USER username

• PASS password

• LIST return list of file in

current directory

• RETR filename retrieves

(gets) file

• STOR filename stores

(puts) file onto remote host

sample return codes

• status code and phrase (as in HTTP)

• 331 Username OK, password required

• 125 data connection already open; transfer starting

• 425 Can’t open data connection

• 452 Error writing file


Lennart Johnsson

2013‐09‐17COSC4377

6

FTP: Active and Passive modes Active FTP

Client initiates connection to server port 21 using a client port N>1023 for control and port N+1 for data

Server using port 21 responds to client port N

Client sends command PORT N+1 to server

FTP server using port 20 initiates data connection to client's data port N+1

FTP client sends ACKs to server's port 20

Passive FTP (to avoid firewall, NAT, and related issues, e.g. used by Web browsers)

Client initiates connection to server port 21 using a client port N>1023 for control and port N+1 for data

Server using port 21 responds to client port N

Client sends command PASV to the server

Server creates a port P>1023 and informs client to connect to P for data transfer

Client using port N+1 initiates connection to server port P

Server using port P sends ACK to client port N+1

Reference: http://slacksite.com/other/ftp.html6

9/21/2013

4

Lennart Johnsson

2013‐09‐17COSC4377

File Transfer Protocols using UDPExamples: SABUL (Simple Available Bandwidth Utilization Library)

http://pubs.rgrossman.com/dl/journal‐028.pdf

RBUDP (Reliable Blast UDP)http://www.evl.uic.edu/cavern/papers/cluster2002.pdf

FOBS (Fast Object‐Based File Transfer System)http://www.umcs.maine.edu/~dickens/pubs/HPDC.FINAL.pdf

………………..

SABUL, Chicago Amsterdam, RTT 110ms

http://pubs.rgrossman.com/dl/journal‐028.pdf

7

Three concurrent SABUL streams, each with 1 Gbps theoretical peak

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP: The Protocol• A high‐performance, secure, reliable data transfer protocol optimized for high‐bandwidth wide‐area networks– FTP with well‐defined extensions– Uses basic Grid security – Multiple data channels for parallel transfers– Partial file transfers– Third‐party transfers– Reusable data channels– Command pipelining

• GGF recommendation GFD.20http://www.ogf.org/documents/GFD.20.pdf, April 2003http://www.ogf.org/documents/GFD.21.pdf, July 2003

Current version, 5.5.4, http://www.globus.org/toolkit/docs/latest‐stable/gridftp/Tutorial, http://www.mcs.anl.gov/~kettimut/tutorials/SC07GridFTPTutorialSlides.pdf

8

9/21/2013

5

Lennart Johnsson

2013‐09‐17COSC4377

Control and Data Channels

Control

Data

Typical Installation

Control

Data

Separate Processes Striped Server

Control

Data

GridFTP (and FTP) use (at least) two separate socket connections: A control channel for carrying the commands and responses A data Channel for actually moving the data

Control Channel and Data Channel can be (optionally) completely separate processes.

9

Lennart Johnsson

2013‐09‐17COSC4377

Parallel Data Streams

• Multiple TCP streams between sender and receiver

• Sender pushes multiple blocks in parallel streams

• Blocks reassembled at receiving side and put into correct order

• Protection against dropped packets for each stream

Parallel TransferFully utilizes bandwidth of

network interface on single nodes10

9/21/2013

6

Lennart Johnsson

2013‐09‐17COSC4377

Striped GridFTP Service

• Multiple nodes work together as a single logical GridFTP server

• Every node of the cluster is used to transfer data into/out of the cluster– Each node reads/writes only pieces

they’re responsible for

– Head node coordinates transfers

• Multiple levels of parallelism– CPU, bus, NIC, disk etc.

– Maximizes use of Gbit+ WANs

Striped TransferFully utilizes bandwidth of

Gb+ WAN using multiple nodes.

Par

alle

l Fil

esys

tem

Par

alle

l Fil

esys

tem

11

Lennart Johnsson

2013‐09‐17COSC4377

Source: Bill Alcock, GlobusWorld 200612

9/21/2013

7

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP

• 100% Globus code– No licensing issues– Stable, extensible

• IPv6 Support• XIO for different transports• Striping multi‐Gb/sec wide area transport• Pluggable

– Front‐end: e.g., future WS control channel– Back‐end: e.g., HPSS, cluster file systems– Transfer: e.g., UDP, NetBLT transport

p g

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

0 10 20 30 40 50 60 70

Degree of Striping

Ban

dw

idth

(M

bp

s)

# Stream = 1 # Stream = 2 # Stream = 4

# Stream = 8 # Stream = 16 # Stream = 32

Disk-to-disk onTeraGrid

13

WS = Web Services, HPSS = High‐Performance Storage SystemNetBLT = Network Block Transfer, RFC 998, March 1987

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP Features

• TCP buffer size control

– Tune buffers to latency of network

– Regular FTP optimized for low latency networks, not tunable

• Dramatic improvements for high latency WAN transfers

– 90% of network utilization possible

– 27 GB/s achieved with commodity hardware

14

9/21/2013

8

Lennart Johnsson

2013‐09‐17COSC4377

• Integrated instrumentation: Developers can use client API and plug‐in mechanism to leverage different instrumentation

– Performance markers

– Restart markers

– Throughput performance

– Netlogger style performance tracking

GridFTP

15

Lennart Johnsson

2013‐09‐17COSC4377

Extensive Logging in the Server

• Multiple log levels: ERROR, WARN, INFO, DUMP, ALL

• Log to stdio, syslog, file, …

• Log all connections/transfers to single file or unique files

• Netlogger style logging

• Control permissions on log files

16

9/21/2013

9

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP Features

• Data Storage Interface (DSI)– Interfaces to various storage types– Implement simple functions such as send, receive, mkdir,…

– DSI modules available for HPSS and SRB

• Globus FTP client library (API):– Integration of data transport capabilities directly into applications

– Plug‐in architecture for installing fault recovery and performance tuning algorithms

– Asynchronous programming model

17

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP: Client API• Simple client flow comprises:

1. Setup transfer details including number of parallel data channels, TCP buffer size, local buffer number and size

2. Open connection to server URL and provide a “completion callback” function to be called when transfer complete

3. Setup local buffers to hold read/write

4. Register “data callback” function to be called for filling/flushing buffers

5. Set “not done flag”

6. Loop/wait until “completion callback” clears not done flag

• Work is done inside the “data callback” function– Local buffer filled with data (receiver) & ready to be flushed

– Receive the offset into the file and any error code

– fseek() to the correct place and fwrite() to file

– Register another empty buffer/callback combination

18

9/21/2013

10

Lennart Johnsson

2013‐09‐17COSC4377

GridFTP: Tool Mechanics

• Server mechanics– globus-gridftp-server

– Usually runs as root

– Usually run as a daemon; connections fork new process and setuid

– Can run inetd/xinetd if so desired

– Port 2811 is standard but is configurable

– Logging and security highly configurable

• Client mechanics– globus-url-copy

– Options for parallel channels, TCP buffer size, data buffer size, debugging, recursive directory transfers, etc.

19

Lennart Johnsson

2013‐09‐17COSC4377

• Server‐side computation– Extended retrieve (ERET), Extended store (ESTO)

– Simple pre‐processing (partial get, sub‐sampling )

– Can greatly reduce network load

– Client must also support ESTO/ERET functionality

GridFTP:Secure, High Performance Data

Transport

20

9/21/2013

11

Lennart Johnsson

2013‐09‐17COSC4377

21

Chapter 2: outline



2.2 Web and HTTP

2.3 FTP


2.5 DNS




Lennart Johnsson

2013‐09‐17COSC4377

e‐mail and Messaging Statistics

• e‐mail

• 2.2 billion users

• 247 billion emails are sent each day

http://www.radicati.com/wp/wp‐content/uploads/2012/04/Email‐Statistics‐Report‐2012‐2016‐Brochure.pdf

Over-the-Top (OTT) messaging v SMS messaging.

OTT messages SMS messages

MMS messages

2011 3,492 billion 7,844 billion 207 billion2012 5,846 billion 8,600 billion 228 billion2016 20,293 billion 9,554 billion 277 billionSources: Portio ResearchFebuary 2012

via: mobiThinking

http://mobithinking.com/mobile‐marketing‐tools/latest‐mobile‐stats/c#mobilemessaging

Key Global Telecom Indicators for the World (estimates)

Globalmillions

Per 100 people

Mobile cellular subscriptions 5,981 86.7

Fixed telephone lines 1,159 16.6

Active mobile broadband subscriptions 1,186 17.0

Fixed broadband subscriptions 591 8.5

Source: International Telecommunication Union(November 2011)

22

9/21/2013

12

Lennart Johnsson

2013‐09‐17COSC4377

23

Electronic mail

Three major components:• user agents • mail servers • simple mail transfer protocol:

SMTP

User Agent• a.k.a. “mail reader”• composing, editing, reading

mail messages• e.g., Outlook, Thunderbird,

iPhone mail client• outgoing, incoming messages

stored on server

user mailbox

outgoing message queue

mailserver

mailserver

mailserver

SMTP

SMTP

SMTP

useragent

useragent

useragent

useragent

useragent

useragent


Lennart Johnsson

2013‐09‐17COSC4377

24

Electronic mail: mail servers

mail servers:• mailbox contains incoming

messages for user

• message queue of outgoing (to be sent) mail messages

• SMTP protocol between mail servers to send email messages

– client: sending mail server

– “server”: receiving mail server

mailserver

mailserver

mailserver

SMTP

SMTP

SMTP

useragent

useragent

useragent

useragent

useragent

useragent


9/21/2013

13

Lennart Johnsson

2013‐09‐17COSC4377

25

Electronic Mail: SMTP [RFC 2821]

• uses TCP to reliably transfer email message from client to server, port 25

• direct transfer: sending server to receiving server

• three phases of transfer– handshaking (greeting)

– transfer of messages

– closure

• command/response interaction (like HTTP, FTP)– commands: ASCII text

– response: status code and phrase

• messages must be in 7‐bit ASCI


Lennart Johnsson

2013‐09‐17COSC4377

26

useragent

Scenario: Alice sends message to Bob1) Alice uses UA to compose

message “to”[email protected]

2) Alice’s UA sends message to her mail server; message placed in message queue

3) client side of SMTP opens TCP connection with Bob’s mail server

4) SMTP client sends Alice’s message over the TCP connection

5) Bob’s mail server places the message in Bob’s mailbox

6) Bob invokes his user agent to read message

mailserver

mailserver

1

2 3 4

5

6

Alice’s mail server Bob’s mail server

useragent


9/21/2013

14

Lennart Johnsson

2013‐09‐17COSC4377

27

Sample SMTP interaction

S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <[email protected]> S: 250 [email protected]... Sender ok C: RCPT TO: <[email protected]> S: 250 [email protected] ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection


Lennart Johnsson

2013‐09‐17COSC4377

28

SMTP: final words

• SMTP uses persistent connections

• SMTP requires message (header & body) to be in 7‐bit ASCII

• SMTP server uses CRLF.CRLF to determine end of message

comparison with HTTP:

• HTTP: pull

• SMTP: push

• both have ASCII command/response interaction, status codes

• HTTP: each object encapsulated in its own response msg

• SMTP: multiple objects sent in multipart msg


9/21/2013

15

Lennart Johnsson

2013‐09‐17COSC4377

29

Mail message format

SMTP: protocol for exchanging email msgs

RFC 822: standard for text message format:

• header lines, e.g.,– To:

– From:

– Subject:

different from SMTP MAIL FROM, RCPT TO:commands!

• Body: the “message”– ASCII characters only

header

body

blankline


Lennart Johnsson

2013‐09‐17COSC4377

30

Mail access protocols

• SMTP: delivery/storage to receiver’s server

• mail access protocol: retrieval from server– POP: Post Office Protocol [RFC 1939]: authorization, download

– IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including manipulation of stored msgs on server

– HTTP: gmail, Hotmail, Yahoo! Mail, etc.

sender’s mail server

SMTP SMTPmail access

protocol

receiver’s mail server

(e.g., POP, IMAP)

useragent

useragent


9/21/2013

16

Lennart Johnsson

2013‐09‐17COSC4377

31

POP3 protocol

authorization phase• client commands:

– user: declare username

– pass: password

• server responses

– +OK– -ERR

transaction phase, client:• list: list message numbers

• retr: retrieve message by number

• dele: delete

• quit

C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents>S: . C: dele 1 C: retr 2 S: <message 1 contents>S: . C: dele 2 C: quit S: +OK POP3 server signing off

S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on


Lennart Johnsson

2013‐09‐17COSC4377

32

POP3 (more) and IMAP

more about POP3• previous example uses

POP3 “download and delete”mode

– Bob cannot re‐read e‐mail if he changes client

• POP3 “download‐and‐keep”: copies of messages on different clients

• POP3 is stateless across sessions

IMAP

• keeps all messages in one place: at server

• allows user to organize messages in folders

• keeps user state across sessions:

– names of folders and mappings between message IDs and folder name


9/21/2013

17

Lennart Johnsson

2013‐09‐17COSC4377

Extra material on SMTP• ESMTP (Extended SMTP;

RFC 1869, 1995) allows 8‐bit data transmission; use “EHLO” (extended Hello) to replace “HELO”

• Authentication process in FTP, POP3, and IMAP is not encrypted. Use FTP/SSL, POP3S, or IMAPS if possible.

Multipart MIME (Multi‐part Internet Mail Extension)From: "Senders Name" <[email protected]>

To: "Recipient Name" <[email protected]>

Message‐ID: <[email protected]>

Date: Sat, 24 Sep 2005 15:06:49 ‐0400

Subject: Sample Multi‐Part

MIME‐Version: 1.0

Content‐Type: multipart/alternative;

boundary="‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3“

‐‐‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3

Content‐type: text/plain; charset=iso‐8859‐1

Content‐Transfer‐Encoding: quoted‐printable

Sample Text Content


Content‐type: text/html; charset=iso‐8859‐1

Content‐Transfer‐Encoding: quoted‐printable

<html><head></head>

<body>Sample HTML Content</body></html>



http://en.wikipedia.org/wiki/Extended_SMTP

33

Lennart Johnsson

2013‐09‐17COSC4377

Another Example – Binary encoded Attachment

From: <[email protected]> CRLFTo: <[email protected]> CRLFSubject: foobar CRLFMIME‐Version: 1.0 CRLFContent‐Type: multipart/mixed; CRLF boundary= "KkK170891tpbkKk__FV_KKKkkkjjwq" CRLFCRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq CRLFContent‐Type: text/plain; charset=US‐ASCII CRLFCRLFhere goes the text message CRLFCRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq CRLFContent‐Type: application/octet‐stream CRLFContent‐Transfer‐Encoding: base64 CRLFContent‐Disposition: attachment; CRLF filename= "suggested name of the attachment" CRLFCRLFhere goes the Base64 encoded attachment CRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq‐‐ CRLFCRLF.CRLF

Blank line

Blank line

Blank line

Blank line

Part 1

Part 2

34

9/21/2013

18

Lennart Johnsson

2013‐09‐17COSC4377

35

Chapter 2: outline



2.2 Web and HTTP

2.3 FTP


2.5 DNS




Lennart Johnsson

2013‐09‐17COSC4377

36

DNS: domain name system

people:many identifiers:

– SSN, name, passport #

Internet hosts, routers:

– IP address (32 bit) ‐used for addressing datagrams

– “name”, e.g., www.yahoo.com ‐used by humans

Q: how to map between IP address and name, and vice versa ?

Domain Name System:• distributed database

implemented in hierarchy of many name servers

• application‐layer protocol:hosts, name servers communicate to resolvenames (address/name translation)– note: core Internet function,

implemented as application‐layer protocol

– complexity at network’s “edge”


9/21/2013

19

Lennart Johnsson

2013‐09‐17COSC4377

37

DNS: services, structure

why not centralize DNS?• single point of failure

• traffic volume

• distant centralized database

• maintenance

DNS services

• hostname to IP address translation

• host aliasing– canonical, alias names

• mail server aliasing

• load distribution

– replicated Web servers: many IP addresses correspond to one name

A: doesn’t scale!


Lennart Johnsson

2013‐09‐17COSC4377

Domain Names Statistics

TLD Total DomainsCOM 102,764,716NET 14,160,747ORG 9,832,140INFO 5,713.205BIZ 2,327,124

TOTAL 134,797,932

http://www.webhosting.info/registries/

9/16/2013

As of October 2009 there were 21 generic TLDs and 250 country domains (based on ISO‐3166 country abbreviations)http://en.wikipedia.org/wiki/Domain_name

• 324 – Number of TLDs

• 86.9 million – Number of country code

top‐level domains (.CN, .UK, .DE, etc.).

• 220 million – Number of registered

domain names.

Q3 2011

http://royal.pingdom.com/2012/01/17/internet‐2011‐in‐numbers/

38

Domain names managed by IANA, the Internet Assigned Numbers Authority, www.iana.org

9/21/2013

20

Lennart Johnsson

2013‐09‐17COSC4377

39

Root DNS Servers

com DNS servers org DNS servers edu DNS servers

poly.eduDNS servers

umass.eduDNS servers

yahoo.comDNS servers

amazon.comDNS servers

pbs.orgDNS servers

DNS: a distributed, hierarchical database

client wants IP for www.amazon.com; 1st approx:

• client queries root server to find .com DNS server

• client queries .com DNS server to get amazon.com DNS server

• client queries amazon.com DNS server to get IP address for www.amazon.com

… …

Lennart Johnsson

2013‐09‐17COSC4377

40

DNS: root name servers

• contacted by local name server that can not resolve name

• root name server:

– contacts authoritative name server if name mapping not known

– gets mapping

– returns mapping to local name server

13 root name “servers”worldwide

a. Verisign, Los Angeles CA(5 other sites)

b. USC-ISI Marina del Rey, CAl. ICANN Los Angeles, CA

(41 other sites)

e. NASA Mt View, CAf. Internet Software C.Palo Alto, CA (and 48 other sites)

i. Netnod, Stockholm (37 other sites)

k. RIPE London (17 other sites)

m. WIDE Tokyo(5 other sites)

c. Cogent, Herndon, VA (5 other sites)d. U Maryland College Park, MDh. ARL Aberdeen, MDj. Verisign, Dulles VA (69 other sites )

g. US DoD Columbus, OH (5 other sites)


9/21/2013

21

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root Servers

http://www.root‐servers.org/

377 servers as of 2013‐09

41

Lennart Johnsson

2013‐09‐17COSC4377

Server Operator Locations

A VeriSign, Inc.Sites: 6 Global: 6 Local: 0Los Angeles, CA, US; New York, NY, US *; Frankfurt, DE *; Hong Kong, HK; Palo Alto, CA, US *; Ashburn, VA, US *

B Information Sciences InstituteSites: 1 Global: 0 Local: 1Earth

C Cogent CommunicationsSites: 6 Global: 6 Local: 0Herndon, VA, US; Los Angeles, CA, US; New York, NY, US; Chicago, IL, US; Frankfurt, DE; Madrid, ES

D University of MarylandSites: 1 Global: 1 Local: 0College Park, MD, US *

E NASA Ames Research CenterSites: 12 Global: 1 Local: 11Mountain View, CA, US; Sydney, AU; Cape Town, ZA; Tokyo, JP; Brussels, BE; London, UK; San Paulo, BR; Atlanta, GA, US; Chicago, IL, US; New York, NY, US; Seattle, WA, US; Los Angeles, CA, US

F Internet Systems Consortium, Inc.

Sites: 49 Global: 2 Local: 47Ottawa, Canada *; Palo Alto, CA, US *; San Jose, CA, US; New York, NY, US *; San Francisco, CA, US *; Madrid, ES; Hong Kong, HK; Los Angeles, CA, US *; Rome, Italy; Auckland, NZ *; Sao Paulo, BR; Beijing, CN; Seoul, KR *; Moscow, RU *; Taipei, TW; Dubai, AE; Paris, FR *; Singapore, SG; Brisbane, AU *; Toronto, CA *; Monterrey, MX; Lisbon, PT *; Johannesburg, ZA; Tel Aviv, IL; Jakarta, ID; Munich, DE *; Osaka, JP *; Prague, CZ *; Amsterdam, NL *; Barcelona, ES *; Nairobi, KE; Chennai, IN; London,UK *; Santiago de Chile, CL; Dhaka, BD; Karachi, PK; Torino, IT; Chicago, IL, US *; Buenos Aires, AR; Caracas, VE; Oslo, NO *; Panama, PA; Quito, EC; Kuala Lumpur, Malaysia *; Suva, Fiji; Cairo, Egypt; Atlanta, GA, US; Podgorica, ME; St. Maarten, AN *

G U.S. DOD Network Information CenterSites: 6 Global: 6 Local: 0Columbus, OH, US; San Antonio, TX, US; Honolulu, HI, US; Fussa, JP; Stuttgart‐Vaihingen, DE; Naples, IT

H U.S. Army Research LabSites: 2 Global: 2 Local: 0Aberdeen Proving Ground, MD, US *; San Diego, CA, US *

I Netnod (formerly Autonomica)

Sites: 43 Stockholm, SE *; Helsinki, FI; Milan, IT *; London, UK *; Geneva, CH *; Amsterdam, NL (2 sites) *; Oslo, NO *; Bangkok, TH *;Hong Kong, HK *; Brussels, BE *; Frankfurt, DE *; Ankara, TR *; Bucharest, RO *; Chicago, IL, US; Washington, DC, US; Tokyo, JP (2 sites) *; Kuala Lumpur, MY *; Palo Alto, CA, US *; Jakarta, ID; Wellington, NZ; Johannesburg, ZA *; Perth, AU *; Singapore, SG; Miami, FL, US; Mumbai, IN; Beijing, CN; Manila, PH; Doha, QA *; Colombo, LK *; Vienna, AT *; Paris, FR *; Taipei, TW *; Porto Alegre, BR; Yerevan, AM *; Thimphu, BT *; Manama, BH; Karachi, PK; Tallinn, EE *; Kiev, UA *; Ulaanbaatar, MN *; Lulea, SE

J VeriSign, Inc.

Sites: 70 Global: 63 Local: 5Dulles, VA, US (2 sites); Dulles, VA, US (1 sites); Ashburn, VA, US *; Miami, FL, US; Atlanta, GA, US; Seattle, WA, US; Chicago,IL, US; New York, NY, US *; Honolulu, HI, US; Mountain View, CA, US (1 sites); Mountain View, CA, US (1 sites); San Francisco, CA, US (2 sites) *; Dallas, TX, US; Amsterdam, NL; London, UK; Stockholm, SE (2 sites); Tokyo, JP; Seoul, KR; Beijing, CN; Singapore, SG; Dublin, IE; Kaunas, LT; Nairobi, KE; Montreal, CA; Perth, AU; Sydney, AU; Cairo, EG; Cairo, EG; Warsaw, PL (2 sites); Brasilia, BR; Sao Paulo, BR; Sofia, BG; Prague, CZ; Johannesburg, ZA; Toronto, CA; Buenos Aires, AR; Madrid, ES; Fribourg, CH; Hong Kong, HK (2 sites); Turin, IT; Mumbai, IN; Oslo, NO; Brussels, BE; Paris, FR (2 sites); Helsinki, FI; Frankfurt, DE *; Riga, LV; Milan, IT; Rome, IT; Lisbon, PT; San Juan, PR; Edinburgh, UK; Tallinn, EE; Taipei, TW; New York, NY, US *; Palo Alto, CA, US *; Anchorage, US; Moscow, RU; Manila, PH; Kuala Lumpur, MY; Luxembourg City, LU; Guam, GU, US; Vancouver, CA; Wellington, NZ

http://www.root‐servers.org/42

9/21/2013

22

Lennart Johnsson

2013‐09‐17COSC4377

Server Operator Locations

K RIPE NCC

Sites: 18 Global: 5 Local: 13London, UK *; Amsterdam, NL *; Frankfurt, DE *; Athens, GR *; Doha, QA; Milan, IT *; Reykjavik, IS *; Helsinki, FI *; Geneva, CH*; Poznan, PL *; Budapest, HU *; Abu Dhabi, AE; Tokyo, JP *; Brisbane, AU *; Miami, FL, US *; Delhi, IN; Novosibirsk, RU; Dar esSalaam, TZ

L ICANN

Sites: 120Ezeiza, Argentina *; Mascot, Australia (2 sites) *; Brisbane, Queensland, Australia *; Perth, Western Australia, Australia (2 sites) *; Vienna, Austria *; Al Muharraq, Bahrain *; Dhaka, Bangladesh *; Brussels, Belgium (2 sites) *; Rio de Janeiro, Brazil *; Salvador, Bahia, Brazil *; Fortaleza, Ceara, Brazil *; Sao Jose dos Campos, Brazil *; Sao Paulo, Brazil *; Brasilia, Distrito Federal, Brazil *; Belo Horizonte, Minas Gerais, Brazil *; Londrina, Parana, Brazil *; Curitiba, Parana, Brazil *; Porto Alegre, Rio Grande do Sul, Brazil *; Florianopolis, Santa Catarina, Brazil *; Campinas, Sao Paulo State, Brazil *; Sofia, Bulgaria *; Mississauga, Ontario, Canada (2 sites) *; Ottawa, Ontario, Canada (2 sites) *; Toronto, Ontario, Canada *; Santiago, Chile *; Concepcion, Biobio Region, Chile *; Bogota, Colombia *; Prague, Czech Republic *; Copenhagen, Denmark *; Quito, Ecuador *; Cairo, Egypt *; Lyon, France *; Marseille, France *; Paris, France *; Paris‐Orly, Paris, France *; St Denis, Reunion, France *; Berlin, Germany *; Hamburg, Germany *; Munich, Bavaria, Germany *; Dortmund, North Rhine‐Westphalia, Germany *; Dusseldorf, North Rhine‐Westphalia, Germany *; Heraklion, Crete, Greece *; Hagatna, Guam *; Jakarta, Java, Indonesia *; Dublin, Ireland *; Turin, Italy *; Narita, Japan *; Tokyo, Honshu, Japan *; Semey, Kazakhstan *; Nairobi, Kenya *; Luxembourg‐Findel, Luxembourg *; Monterrey, Nuevo Leon, Mexico *; Maputo, Mozambique *; Kathmandu, Nepal *; Haarlemmermeer, Netherlands *; Christchurch, New Zealand *; Mangere, New Zealand (2 sites) *; Wellington, New Zealand *; Ullensaker, Norway *; Beijing, Peoples Republic of China (2 sites) *; Callao, Peru *; Metro Manila, Philippines *; Warsaw, Poland *; Moscow, Russia *; Dammam, Saudi Arabia *; Jeddah, Saudi Arabia *; Riyadh, Saudi Arabia *; Dakar, Senegal *; Changi, Singapore *; Cape Town, South Africa (2 sites) *; Johannesburg, Gauteng, South Africa *; Incheon, South Korea *; El Prat de Llobregat, Spain *; Malmo, Sweden *; Stockholm, Sweden *; Geneva, Switzerland *; Zurich, Switzerland *; Istanbul, Turkey (3 sites) *; Kharkiv, Ukraine *; Kiev, Ukraine *; Odessa, Ukraine *; Saginaw Bay, Alaska, United States *; Burbank, California, United States *; Los Angeles, California, United States (2 sites) *; San Jose, California, United States *; San Jose, Costa Rica *; Santa Ana, California, United States *; Denver, Colorado, United States *; Jacksonville, Florida, United States *; Melbourne,Florida, United States *; Miami, Florida, United States (2 sites) *; Atlanta, Georgia, United States (2 sites) *; Honolulu, Hawaii, United States *; Chicago, Illinois, United States *; Boston, Massachusetts, United States (2 sites) *; New York City, New York, United States *; Portland, Oregon, United States *; Philadelphia, Pennsylvania, United States *; Culpeper, Virginia, United States *; SeaTac, Washington, United States *; Montevideo, Uruguay *; Sanaa, Yemen *; Vancouver, Canada *; Leeds, United Kingdom *; Anchorage, Alaska, United States *; Porto, Portugal *

M WIDE ProjectSites: 6 Global: 5 Local: 1Tokyo, JP (3 sites) *; Seoul, KR; Paris, FR *; San Francisco, CA, US *

http://www.root‐servers.org/ 43

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root Server K

44

9/21/2013

23

Lennart Johnsson

2013‐09‐17COSC4377

45

TLD, authoritative servers

top‐level domain (TLD) servers:– responsible for com, org, net, edu, aero, jobs, museums, and all top‐level country domains, e.g.: uk, fr, ca, jp

– Network Solutions maintains servers for .com TLD

– Educause for .edu TLD

authoritative DNS servers:– organization’s own DNS server(s), providing authoritative hostname to IP mappings for organization’s named hosts

– can be maintained by organization or service provider


Lennart Johnsson

2013‐09‐17COSC4377

46

Local DNS name server

• does not strictly belong to hierarchy

• each ISP (residential ISP, company, university) has one– also called “default name server”

• when host makes DNS query, query is sent to its local DNS server– has local cache of recent name‐to‐address translation pairs (but may be out of date!)

– acts as proxy, forwards query into hierarchy


9/21/2013

24

Lennart Johnsson

2013‐09‐17COSC4377

47

requesting hostcis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS serverdns.poly.edu

1

23

4

5

6

authoritative DNS serverdns.cs.umass.edu

78

TLD DNS server

DNS name resolution example

• host at cis.poly.edu wants IP address for gaia.cs.umass.edu

iterated query: contacted server

replies with name of server to contact

“I don’t know this name, but ask this server”


Lennart Johnsson

2013‐09‐17COSC4377

48

45

6

3recursive query: puts burden of name

resolution on contacted name server

heavy load at upper levels of hierarchy?

requesting hostcis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS serverdns.poly.edu

1

27

authoritative DNS serverdns.cs.umass.edu

8

DNS name resolution example

TLD DNS server


9/21/2013

25

Lennart Johnsson

2013‐09‐17COSC4377

49

DNS: caching, updating records

• once (any) name server learns mapping, it cachesmapping– cache entries timeout (disappear) after some time (TTL)

– TLD servers typically cached in local name servers• thus root name servers not often visited

• cached entries may be out‐of‐date (best effort name‐to‐address translation!)– if name host changes IP address, may not be known Internet‐wide until all TTLs expire

• update/notify mechanisms proposed IETF standard– RFC 2136


Lennart Johnsson

2013‐09‐17COSC4377

(18 nodes, 5 global, 13 local)

Global

http://k.root‐servers.org/

Some statistics for the K DNS Root server 9/25/2012

50

9/21/2013

26

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root server I

http://www.netnod.se/

43 sites

51

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root Server I load 9/25/2012

http://www.netnod.se/ix‐stats/sums/All.html

'Daily' graph (5 Minute Average)

52

9/21/2013

27

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root Server I load 9/18 – 9/25 2012'Weekly' graph (30 Minute Average)

http://www.netnod.se/ix‐stats/sums/All.html

53

Lennart Johnsson

2013‐09‐17COSC4377

DNS Root Server I load 2 years ending 9/25/12

'Two year' graph (1 Day Average)

http://www.netnod.se/ix‐stats/sums/All.html54

Documents

Lecture07 - UHjohnsson/COSC4377_2013/Lecture07.pdfprogramming with UDP and TCP Slide ... FTP client FTP server ... – Can greatly reduce network load – Client must also support