Internet Data PortalJay Daley, .nz
https://idp.nz
ICANN54Internet Data Portal 2
• Repository for Internet data relating to NZ• All open data• Place to explore• Forum to discuss• … and more
What is it
ICANN54Internet Data Portal 3
.nz Zone Scan – TTL August
ICANN54Internet Data Portal 4
Zoomed in
ICANN54Internet Data Portal 5
Underlying data
ICANN54Internet Data Portal 6
• Chose Socrata US basedSaaS running on AWS
• RequirementsDecent catalogueChart designerAPI access (OData in particular)
• Alternatives rejectedCKANDKAN
The product
ICANN54Internet Data Portal 7
• .nz Zone ScanRun monthlyLooks for 50+ indicatorsAggregate stats uploaded after each scan
• .nz Registration dataTransaction per day by type of transactionDomains in each stateAggregate stats uploaded each day
• OtherWeb Index from Web Foundation - republishedNZ ISP Survey from StatisticsNZ
Current datasets
ICANN54Internet Data Portal 8
• Wide has one column per measureYear, Fibre, ADSL, VDSL, Cable
• Long has fewer columns that encode moreYear, Access Type, Connections
• We use Long, which meansMany more rowsNeed tools that handle clusteringMore flexibility in analysis
Long vs Wide
ICANN54Internet Data Portal 9
• Fields (excluding admin fields)Classification, Metric, Count Type, Count
• Classification/Metric examplesTTL_MX [0, 5m[TTL_MX 5mDomain Errors Domains with broken delegationDomain Errors Domains with inconsistent glueDNSSEC Domains with DS recordsDNSSEC Key Algorithm RSA/SHA-1
• Count Type is one ofDomains, Servers, Hosts, Nameservers, DNSKEY
.nz Zone Scan data structure
ICANN54Internet Data Portal 10
• Fields Day, Parent Level Domain, Measure, Measure Type, Count
• Measure/Measure Type examplesDue for renewal StatePending release StateCreated TransactionsCancelled Transactions
• Parent Level Domain examplesnz, co.nz, org.nz, ac.nz, kiwi.nz, geek.nz
.nz Reg stats data structure
ICANN54Internet Data Portal 11