perfSONAR:Schema, Topology and

Discovery

Martin Swany

OGF IPR Policies Apply

“I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:

the OGF plenary session, any OGF working group or portion thereof, the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, the ADCOM, or any member thereof on behalf of the ADCOM, any OGF mailing list, including any group list, or any other list functioning under OGF auspices, the OGF Editor or the document authoring and review process

Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”

OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.

SchemaKey Goals: Extensibility, Normalization, ReadabilityBreak representation of performance measurements down into basic elementsData and MetadataMeasurement Data

A set of of measurement events that have some value or values at a particular time

Measurement MetadataThe details about the set of measurement data

Schema Normalization

Can simply the database representation for many types of measurement data

While optimizations are certainly possible, many measurement types can be viewed as one value over time

Assists Combination/Concatenation of metrics

Creating derived metrics

Normalization helps with inferring relationships between types of metrics

Schema Basic Elements - Metadata

SubjectThe measured/tested entity

Characteristic/Event Type (Verb)What type of measurement, value, or event occurred

Parameters (Adjectives and Adverbs)How, or under what conditions, did this event occur?

Schema Basic Elements - Data

Some sort of value - DatumExistence of an event might point to the case where there no additional value

As in “Link up/down” or threshold events

TimeMust be extensible since even agreement about the right structure is not easy

E.g. UNIX timestamp vs NTP time

A MessageMessageMessage

Metadata

Data

An Object StoreStore

Metadata

Data

A Data is Linked to a MetadataMetadata

<id>someId</id>

Data

<metadataIdRef> someId</metadataIdRef>

A Metadata may be linked to another

Metadata

<id>someId</id>

Metadata<id>someOtherId</id>

<metadataIdRef> someId</metadataIdRef>

Schema Namespaces

All measurements have some sort of Data and Time

All measurements can be described by the Metadata identifying who, what and how

The specific structures of the Data and Metadata elements depend on the measurement

Approach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements

Schema Namespaces - 2

Why encode the measurement/event type in the namespace?

The encoding is essentially redundant

Some components of the system can pass Data and Metadata elements through without understanding their specific structure

Allows and implementation to decide whether it supports a particular type of data or not

Allows validation based on extended (namespace-specific) schemata

Schema Namespaces and Extensibility

One key to extensibility is the use of hierarchy with delegation

Similar to OIDs in the IETF management world

The Global Grid Forum’s NM-WG has a hierarchy of characteristics

Good starting point

However, not all tools are cleanly mapped onto the Characteristic space

Often a matter of some debate

Schema Namespaces and Extensibility - 2

Organization-rooted tools namespace addresses this

Some top-level toolsping, traceroute

Easy to add new tools in organization-specific namespaces

Performance Event RepositoryAdd a schema and get a URI

Add Java classes

Versioning Example<nmwg:message id="snmpmsg5" type="MetadataKeyRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:snmp="http://ggf.org/ns/nmwg/tools/snmp/2.0/" xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/">

<nmwg:metadata id="meta2"> <snmp:subject id="subject2"> <nmwgt:interface> <nmwgt:hostName>scotch.pc.cis.udel.edu</nmwgt:hostName> <nmwgt:ipAddress type="ipv4">128.4.133.141</nmwgt:ipAddress> </nmwgt:interface> </snmp:subject> <nmwg:eventType>SNMP.Get.Request</nmwg:eventType> </nmwg:metadata>

Versioning

Separate revisions of base schema and eventType schema

Open Issues:Query for supported versions must be defined

This will enable a migration strategy

Topology Schema

Topology SchemaTopology schema

Reusable Subject elements for common casesAlso reduces redundancy

Relationships between Subjects

Same basic structure at all layersNetworks are graphs

Define:NodesInterfacesLinksNetworks

Topology

Topology - Recursive Links

Topology SchemaStructured by layers and the same elements recurring thereVaried again by namespaces

Reuse visualization logic, etc.

4 Layers: Base (both abstract and L1), L2, L3, L4

<?xml version="1.0" encoding="UTF-8"?>

<nmwg:store xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/3.0/" xmlns:nmwgtl3="http://ggf.org/ns/nmwg/topology/l3/3.0/" xmlns:nmwgtl2="http://ggf.org/ns/nmwg/topology/l2/3.0/">

Relationships between Subjects To completely capture the relationships, we need to do a few more thingsRecursive definition of links

Logical links consist of physical links

Ordered lists of linksLike above, but we need to introduce an Index attribute

NetworksPhysically consist of links but that is not always the most convenient logical viewSpecial element to which Interfaces or Links belong

Relationships between Subjects

Elements at the same layer have relationshipsA link “contains” two interfaces

At Layer2 or Layer3

Elements of the same sort have relationships between themselves at different layers

A Layer 1 Interface (physical NIC) can have one or more Layer 2 Interfaces, which can each have one or more Layer 3 Interfaces

Node is special Since a Node doesn’t really have any higher-layer characteristic independent of its Interfaces

Schema Status

Three documents in preparation for OGFBase Schema

Topology Schema

Extension guide With examples of utilization, traceroute, ping

Drafts to NMWG by January

OGF meeting in February

Schema Status

Three schema componentsBase

Defines Message, Store, Data, Metadata, Parameters

TopologyDefines Subjects that can appear in the Metadata and Version 3 represents their relationships as well

EventTypesEach data type extends the Base and can define what Parameters are acceptable and what subjects are required

Outstanding Issues

Namespaces for Message typesCurrently a simple text field, but namespaces might make Message syntax/semantics easier to track (and version)

Uniform EventTypes that match the EventType namespace

XML Factoring to further reduce redundant information

Some work at UD on “views”