42
volib A Python implementation of the VO Data Modeling Language Omar Laurino SAO

volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

  • Upload
    others

  • View
    14

  • Download
    0

Embed Size (px)

Citation preview

Page 1: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

!

volib A Python implementation of

the VO Data Modeling LanguageOmar Laurino

SAO

Page 2: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Scope

Page 3: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Scope• Provide VO developers with a framework for dealing

with complex data models (client and server side) and create reusable software

Page 4: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Scope• Provide VO developers with a framework for dealing

with complex data models (client and server side) and create reusable software

• Provide VO users with friendly, extensible I/O libraries while shielding them from the technical details of the data models specs

Page 5: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Scope• Provide VO developers with a framework for dealing

with complex data models (client and server side) and create reusable software

• Provide VO users with friendly, extensible I/O libraries while shielding them from the technical details of the data models specs

• Stress on a clean, simple API. “Focus on the science, not on the specifications”

Page 6: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Scope• Provide VO developers with a framework for dealing

with complex data models (client and server side) and create reusable software

• Provide VO users with friendly, extensible I/O libraries while shielding them from the technical details of the data models specs

• Stress on a clean, simple API. “Focus on the science, not on the specifications”

• Code generation, if needed

Page 7: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Specifications• VO-DML WD: Consistent language for describing data models

Machine-readability

Model reuse

Model extensibility

Portable identifiers

Backward compatibility with older serializations

• Mapping Data Models to VOTable WD: define a standard strategy for serializing any data model instance to VOTable

Page 8: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Features• VO-DML WD: Consistent language for describing data models

Model reuse —> Reusable libraries

Model extensibility —> Inheritance/Polymorphism

Machine-readability —> Model-agnostic. Code generation.

Portable identifiers —> Context and Resolvers

Versioning

• Mapping Data Models to VOTable WD: define a standard strategy for serializing any data model instance to VOTable

VOTable I/O

Page 9: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Representing Data Models

Page 10: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Representing Data ModelsVO-DML Types

Page 11: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Representing Data ModelsVO-DML Types

Types from Imported Models

Page 12: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Representing Data ModelsVO-DML Types

Types from Imported Models

Types from Same Model

Page 13: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Representing Data Models

Declarative approach allows users/developers/providers to easily create new classes and serialize them as

compliant VOTable and VO-DML/XML

VO-DML Types

Types from Imported Models

Types from Same Model

Page 14: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Page 15: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/PolymorphismParent Class

Page 16: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/PolymorphismParent Class

Extensions

Page 17: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Page 18: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Any children of SkyError can be used as coordinate errors

Page 19: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Any children of SkyError can be used as coordinate errors

Page 20: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Any children of SkyError can be used as coordinate errors

Page 21: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Inheritance/Polymorphism

Any children of SkyError can be used as coordinate errors

This custom class can be serialized just as a GenericEllipse

Page 22: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Some useful properties

Page 23: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Some useful properties

Page 24: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Some useful properties

Page 25: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO

Page 26: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO• Updates from latest draft, abstract types

Page 27: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO• Updates from latest draft, abstract types

• VOTable (de-)serialization

Page 28: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO• Updates from latest draft, abstract types

• VOTable (de-)serialization

• VO-DML/XML generation from Python classes

Page 29: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO• Updates from latest draft, abstract types

• VOTable (de-)serialization

• VO-DML/XML generation from Python classes

• Implement VO-DML Quantities as Astropy Quantities (units conversions, quantity comparison/combinations)

Page 30: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

TO DO• Updates from latest draft, abstract types

• VOTable (de-)serialization

• VO-DML/XML generation from Python classes

• Implement VO-DML Quantities as Astropy Quantities (units conversions, quantity comparison/combinations)

• Data binding/Callbacks - Glue integration? (see C. Beaumont talk @ADASS about hackable interfaces)

Page 31: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary

Page 32: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

Page 33: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

Page 34: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

Page 35: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

decoupled from implementation

Page 36: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

decoupled from implementation

concentrate on science, not VO specs.

Page 37: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

decoupled from implementation

concentrate on science, not VO specs.

no need to be a Python guru to create your own interoperable classes/models.

Page 38: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

decoupled from implementation

concentrate on science, not VO specs.

no need to be a Python guru to create your own interoperable classes/models.

• Model/Serialization separation of concerns:

Page 39: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

Summary• Interoperable VO “pickle”

• Declarative language. Just like Django, SQLAlchemy ( or Hibernate), etc:

no __init__, no clutter

decoupled from implementation

concentrate on science, not VO specs.

no need to be a Python guru to create your own interoperable classes/models.

• Model/Serialization separation of concerns:

Drivers can provide alternative representation formats (e.g. HDF5, FITS)

Page 40: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

The Big Picture

Paul’s IDE

UML Modeler

VO-DML XML

Modeling

Script

Application

Apps

Scripts

Data Access Services

Frameworks

VOTableFITS

Page 41: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

The Big Picture

Paul’s IDE

UML Modeler

VO-DML XML

Modeling

Script

Application

Apps

Scripts

Data Access Services

Frameworks

VOTableFITS

Python/volib

Python/volib Python/volib

Page 42: volib A Python implementation of the VO Data ... - IVOAwiki.ivoa.net/internal/IVOA/InterOpOct2014Applications/VOLibBanff.pdf · with complex data models (client and server side) and

https://github.com/olaurino/volib

Thank you! !

And thanks to Markus Demleitner for his input. !

More information on VO-DML, with examples in DM2, Saturday @ 11