| [email protected] METABASE and PSI, P3G workshop 2009 2009 2009 | Pheno-OM implement, evaluate, refine Morris A. Swertz, Tomasz Adamusiak, Juha Muilu,

|[email protected] and PSI, P3G workshop 2009

2009

2009

|

Pheno-OMimplement, evaluate, refine

Morris A. Swertz, Tomasz Adamusiak, Juha Muilu, members of GEN2PHEN and Geneva workshop, Helen Parkinson

P3G data modelling workshopOctober 1st, Luxembourg

1

|[email protected] Pheno-OM, P3G workshop 2009

2009

2009

|

Use cases

› Use case:› Give overview of equal/partial matching

features between studies› Need to group variables for this (for inferred

features) => › Alternative coding schemes

- So mappings between codes

› How about complicated mappings?

2


2009

2009

|

Pheno-OM

› Simple system for phenotype representation

• minimal model, but not too minimal

• unambiguous entity naming (so not ‘phenotype’)

• work on real data

• multiple investigations in one envelop

• easy to create/parse/convert-into format

• interface/modules that can be adopted by others

• ontology enabled for querying/integration

“Give me all individuals that have deformed hand … across studies … across species”

3


2009

2009

|

Outline

› Implementation procedure- Software generation- Exchange format, db, UI, tools

› Evaluation and model refinement- Data loaded- Model details (and limitations?)

› Future- Mapping to FuGE, XGAP, MAGE-TAB, PaGE- As a module to complementary models- Semantic layers- Barriers to progress

4


2009

2009

|

Implementation procedure

Incremental steps using MOLGENIS toolbox.

5


2009

2009

|

MOLGENIS concept6

Model of a variant<entity name="Experiment" label="Experiment"> <field name="ExperimentID" key="1“ readonly="true" label="ExperimentID(autonum)"/> <field name="Medium" type="xref" xref_field="Medium.name"/> /> <field name="Protocol" label="Experiment Protocol"/> <field name="Temperature" type="int"




Complex research

Reusable software code framework and generators

Reusable software code framework and generators

+

inbreed

100

10.000

1,000,000

100,000

10,000

10,000,00

QTL profiles

network

correlate

genomestrains

individuals

markers

expressions

preprocess

probesmicroarrays

100

hybridize

100,000

genotype genotypes

norm exprs.

10.000

map

Model specifics(biology)

Automatecommon coding(informatics)

Repeat to produce a family of research software

Bespoke infrastructure


2009

2009

|

Locus Specific database

QTL/GWA database

NextGen sequencing

Proteo/Metabolomics

Animal Observations

Growing family, sharing the work7


2009

2009

|

Why? Flexible, Reuse, etc. 8

http://www.molgenis.orgSwertz & Jansen (2007) Nature Reviews Genetics 8, 235-243

http://www.molgenis.org


2009

2009

|

Step 1a: model

1: molgenis_db.xml

506 lines of XML code16 entities, 67 fields


2009

2009

|

Generates 150 files, 30k lines of Java, SQL and R code + docs(tomcat/mysql; hsqldb, psql, jpa/hibernate, jetty in alpha )

0 INFO [myFactory] working dir: D:\Development\Molgenis33Workspace\molgenis4phenotype78 INFO [myFactory] MOLGENIS version 3.3.0-testing94 INFO [myFactory] Using options:model_database = [pheno_db.xml] #File with data structure specification (in MOLGENIS DSL).model_userinterface = pheno_ui.xml #File with user interface specification (in MOLGENIS DSL). Can be same file as model_databaseoutput_src = generated/java #Output-directory for the generated project.output_hand = handwritten/java #Output-directory for the generated project.output_sql = generated/sql #Output-directory for the generated sql files.output_doc = WebContent/doc #Output-directory for the generated documentation.output_type = #Output type of the project, either war (for use in tomcat) or jar (standalone).output_web = WebContent #Output-directory for any generated web resourcesdb_driver = com.mysql.jdbc.Driver #Driver of database. Any JDBC compatible driver should work.db_user = molgenis #Username for database. db_password = xxxxxx #Password for database. db_uri = jdbc:mysql://localhost/pheno #Uri of the database. Default: localhostdb_filepath = attachedfiles #Path where the database should store file attachements. Default: nulldb_jndiname = jdbc/molgenisdb #Used to create a JDBC database resource for the applicationobject_relational_mapping = subclass_per_table #Expert option: Choosing OR strategy. Either 'class_per_table', 'subclass_per_table', 'hierarchy_per_table'. Default: class_per_tablemapper_implementation = multiquery #Expert option: Choosing wether multiquery is used instead of prepared statements. Default: falseexclude_system = true #Expert option: Whether system tables should be excluded from generation. Default: trueforce_molgenis_package = false #Expert option. Whether the generated package should be 'molgenis' or the name specified in the model. Default: falseauth_loginclass = org.molgenis.framework.security.SimpleSecurity #Expert option.verbose = true #This switch turns the verbose-mode on.compile = false #This switch makes the factory also compile (usefull outside IDE).mail_smtp_protocol = #Sets the email protocol, either smtp, smtps or null. Default: null meaning email disabledmail_smtp_hostname = localhost #SMTP host server. Default: localhostmail_smtp_port = 25 #SMTP host server port. Default: 25mail_smtp_user = #SMTP user for authenticated emailing. Default: null.mail_smtp_password = #SMTP user for authenticated emailing. Default: null.

110 INFO [MolgenisLanguage] parsing db-schema from [pheno_db.xml]780 WARN [Entity] [WARNING]: missing key 0 for entity Nameable780 WARN [Entity] [WARNING]: missing key 0 for entity Nameable780 WARN [Entity] [WARNING]: missing key 0 for entity Nameable780 WARN [Entity] [WARNING]: missing key 0 for entity Nameable780 WARN [Entity] [WARNING]: missing key 0 for entity Nameable797 WARN [Entity] [WARNING]: missing key 0 for entity Nameable797 WARN [Entity] [WARNING]: missing key 0 for entity Nameable844 INFO [MolgenisLanguage] parsing ui-schema937 INFO [main] generating ....1717 INFO [TableDocGen] generated WebContent\doc\tabledoc.html2076 INFO [EntityDocGen] generated WebContent\doc\objectmodel.html2436 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram.dot2545 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram-pheno.system.dot2748 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram-pheno.observation.dot2842 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram-pheno.target.dot2998 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram-pheno.variable.dot3138 INFO [DotDocGen] generated WebContent\doc\entity-uml-diagram-pheno.protocol.dot3997 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-minimal-diagram.dot4184 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-diagram-minimal-pheno.system.dot4388 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-diagram-minimal-pheno.observation.dot4606 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-diagram-minimal-pheno.target.dot4731 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-diagram-minimal-pheno.variable.dot4887 INFO [DotDocMinimalGen] generated WebContent\doc\entity-uml-diagram-minimal-pheno.protocol.dot5184 INFO [ClassDocGen] generated WebContent\doc\classmodel.html5293 INFO [InMemoryDatabaseGen] generated generated\java\ui\data\InMemoryDatabase.java5609 INFO [MySqlCreateSubclassPerTableGen] generated generated\sql\create_tables.sql5671 INFO [JDBCDatabaseGen] generated generated\java\ui\JDBCDatabase.java5921 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Identifiable.java5921 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Nameable.java5968 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\OntologySource.java6014 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\OntologyTerm.java6030 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Investigation.java6061 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ObservableFeature.java6124 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ObservedValue.java6170 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ObservedRelationship.java6217 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\InferredValue.java6233 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ObservationTarget.java6280 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Individual.java6311 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Panel.java6326 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\CodeList.java6327 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Code.java6374 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Protocol.java6390 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ProtocolApplication.java6405 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ProtocolParameter.java6437 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\ParameterValue.java6452 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\InferredValue_derivedFrom.java6468 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Panel_individuals.java6483 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Protocol_observableFeatures.java6499 INFO [DataTypeGen] generated generated\java\pheno\core\data\types\Protocol_protocolComponents.java6624 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\OntologySourceMapper.java6655 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\OntologyTermMapper.java6671 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\InvestigationMapper.java6702 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ObservableFeatureMapper.java6733 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ObservedValueMapper.java6780 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ObservedRelationshipMapper.java6827 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\InferredValueMapper.java6842 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ObservationTargetMapper.java6873 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\IndividualMapper.java6889 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\PanelMapper.java6905 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\CodeListMapper.java6936 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\CodeMapper.java6951 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ProtocolMapper.java6983 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ProtocolApplicationMapper.java6998 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ProtocolParameterMapper.java7029 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\ParameterValueMapper.java7045 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\InferredValue_derivedFromMapper.java7061 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\Panel_individualsMapper.java7076 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\Protocol_observableFeaturesMapper.java7092 INFO [MultiqueryMapperGen] generated generated\java\pheno\core\data\mappers\Protocol_protocolComponentsMapper.java7217 INFO [JDBCMetaDatabaseGen] generated generated\java\ui\JDBCMetaDatabase.java7263 INFO [CountPerEntityGen] generated generated\sql\count_per_entity.sql7310 INFO [CountPerTableGen] generated generated\sql\count_per_table.sql7341 INFO [FillMetadataTablesGen] generated generated\sql\insert_metadata.sql7405 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\OntologySourceCsvReader.java7420 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\OntologyTermCsvReader.java7420 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\InvestigationCsvReader.java7436 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ObservableFeatureCsvReader.java7452 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ObservedValueCsvReader.java7467 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ObservedRelationshipCsvReader.java7483 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\InferredValueCsvReader.java7498 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ObservationTargetCsvReader.java7514 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\IndividualCsvReader.java7514 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\PanelCsvReader.java7530 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\CodeListCsvReader.java7545 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\CodeCsvReader.java7545 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ProtocolCsvReader.java7561 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ProtocolApplicationCsvReader.java7561 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ProtocolParameterCsvReader.java7576 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\ParameterValueCsvReader.java7576 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\InferredValue_derivedFromCsvReader.java7592 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\Panel_individualsCsvReader.java7608 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\Protocol_observableFeaturesCsvReader.java7608 INFO [CsvReaderGen] generated generated\java\pheno\core\data\csv\Protocol_protocolComponentsCsvReader.java7748 INFO [REntityGen] generated generated\java\pheno\core\R\OntologySource.R7748 INFO [REntityGen] generated generated\java\pheno\core\R\OntologyTerm.R7764 INFO [REntityGen] generated generated\java\pheno\core\R\Investigation.R7779 INFO [REntityGen] generated generated\java\pheno\core\R\ObservableFeature.R7779 INFO [REntityGen] generated generated\java\pheno\core\R\ObservedValue.R7795 INFO [REntityGen] generated generated\java\pheno\core\R\ObservedRelationship.R7795 INFO [REntityGen] generated generated\java\pheno\core\R\InferredValue.R7810 INFO [REntityGen] generated generated\java\pheno\core\R\ObservationTarget.R7810 INFO [REntityGen] generated generated\java\pheno\core\R\Individual.R7826 INFO [REntityGen] generated generated\java\pheno\core\R\Panel.R7826 INFO [REntityGen] generated generated\java\pheno\core\R\CodeList.R7842 INFO [REntityGen] generated generated\java\pheno\core\R\Code.R7857 INFO [REntityGen] generated generated\java\pheno\core\R\Protocol.R7857 INFO [REntityGen] generated generated\java\pheno\core\R\ProtocolApplication.R7873 INFO [REntityGen] generated generated\java\pheno\core\R\ProtocolParameter.R7873 INFO [REntityGen] generated generated\java\pheno\core\R\ParameterValue.R7888 INFO [REntityGen] generated generated\java\pheno\core\R\InferredValue_derivedFrom.R7888 INFO [REntityGen] generated generated\java\pheno\core\R\Panel_individuals.R7888 INFO [REntityGen] generated generated\java\pheno\core\R\Protocol_observableFeatures.R7904 INFO [REntityGen] generated generated\java\pheno\core\R\Protocol_protocolComponents.R7998 INFO [RApi] generated generated\java\source.R8044 INFO [HtmlFormGen] generated generated\java\pheno\core\html\IdentifiableHtmlForm.java8044 INFO [HtmlFormGen] generated generated\java\pheno\core\html\NameableHtmlForm.java8044 INFO [HtmlFormGen] generated generated\java\pheno\core\html\OntologySourceHtmlForm.java8044 INFO [HtmlFormGen] generated generated\java\pheno\core\html\OntologyTermHtmlForm.java8060 INFO [HtmlFormGen] generated generated\java\pheno\core\html\InvestigationHtmlForm.java8060 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ObservableFeatureHtmlForm.java8076 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ObservedValueHtmlForm.java8076 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ObservedRelationshipHtmlForm.java8076 INFO [HtmlFormGen] generated generated\java\pheno\core\html\InferredValueHtmlForm.java8091 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ObservationTargetHtmlForm.java8091 INFO [HtmlFormGen] generated generated\java\pheno\core\html\IndividualHtmlForm.java8091 INFO [HtmlFormGen] generated generated\java\pheno\core\html\PanelHtmlForm.java8091 INFO [HtmlFormGen] generated generated\java\pheno\core\html\CodeListHtmlForm.java8107 INFO [HtmlFormGen] generated generated\java\pheno\core\html\CodeHtmlForm.java8107 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ProtocolHtmlForm.java8107 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ProtocolApplicationHtmlForm.java8107 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ProtocolParameterHtmlForm.java8122 INFO [HtmlFormGen] generated generated\java\pheno\core\html\ParameterValueHtmlForm.java8122 INFO [HtmlFormGen] generated generated\java\pheno\core\html\InferredValue_derivedFromHtmlForm.java8122 INFO [HtmlFormGen] generated generated\java\pheno\core\html\Panel_individualsHtmlForm.java8122 INFO [HtmlFormGen] generated generated\java\pheno\core\html\Protocol_observableFeaturesHtmlForm.java8138 INFO [HtmlFormGen] generated generated\java\pheno\core\html\Protocol_protocolComponentsHtmlForm.java8138 INFO [MolgenisServletContextGen] generated WebContent\META-INF\context.xml8169 INFO [MolgenisContextListenerGen] generated generated\java\servlet\ContextListener.java8232 INFO [MolgenisServletGen] generated generated\java\MolgenisServlet.java8403 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\InvestigationsForm.java8560 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ObservableFeaturesForm.java8591 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\PanelsForm.java8654 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\Panels\IndividualsForm.java8701 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ObservedValuesForm.java8732 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ProtocolApplicationsForm.java8825 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ProtocolApplications\ProtocolApplicationMenu\ParameterValuesForm.java8857 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ProtocolApplications\ProtocolApplicationMenu\ObservedValuesForm.java8888 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ProtocolApplications\ProtocolApplicationMenu\InferredValuesForm.java9013 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\InferredValuesForm.java9044 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\ObservableFeaturesForm.java9137 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\ObservationTargets\IndividualsForm.java9169 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\ObservationTargets\PanelsForm.java9200 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\ProtocolsForm.java9293 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Protocols\ProtocolMenu\ParametersForm.java9325 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Protocols\ProtocolMenu\ProtocolComponentsForm.java9496 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Ontologies\OntologyTermsForm.java9528 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Ontologies\OntologySourcesForm.java9606 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Ontologies\OntologySources\OntologyTermsForm.java9638 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Ontologies\CodeListsForm.java9700 INFO [FormScreenGen] generated generated\java\ui\screen\TopMenu\Main\Ontologies\CodeLists\CodesForm.java9965 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenuMenu.java10012 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\MainMenu.java10059 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenuMenu.java10152 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\Main\Investigations\InvestigationMenu\ProtocolApplications\ProtocolApplicationMenuMenu.java10230 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\Main\ObservationTargetsMenu.java10293 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\Main\Protocols\ProtocolMenuMenu.java10324 INFO [MenuScreenGen] generated generated\java\ui\screen\TopMenu\Main\OntologiesMenu.java11354 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\Main\ReportPlugin.java11557 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\Main\Ontologies\OntologyManagerPlugin.java11604 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\Model_documentationPlugin.java11604 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\RprojectApiPlugin.java11620 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\HttpApiPlugin.java11635 INFO [PluginScreenGen] generated Molgenis33Workspace\molgenis4phenotype\generated\java\ui\screen\TopMenu\WebServicesApiPlugin.java11651 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\report\InvestigationOverview.ftl11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\OntologyBrowser\OntologyBrowserPlugin.ftl11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\DocumentationScreen.ftl11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\RprojectApiScreen.ftl11823 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\HttpAPiScreen.ftl11823 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\SoapApiScreen.ftl11854 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\report\InvestigationOverview.java12057 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\OntologyBrowser\OntologyBrowserPlugin.java12072 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\DocumentationScreen.java12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\RprojectApiScreen.java12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\HttpAPiScreen.java12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwritten\java\plugin\topmenu\SoapApiScreen.java12103 INFO [MolgenisServletContextGen] generated WebContent\META-INF\context.xml12259 INFO [SoapApiGen] generated generated\java\ui\SoapApi.java12353 INFO [CsvExportGen] generated generated\java\tools\CsvExport.java12431 INFO [CsvImportByNameGen] generated generated\java\tools\CsvImportByName.java12636 INFO [CopyMemoryToDatabaseGen] generated generated\java\ui\tools\CopyMemoryToDatabase.java

Generate (MOLGENIS + Eclipse)10


2009

2009

|

Step 2: evaluate on paper (goto 1)

1: molgenis_db.xml

2: documentation*

*autogenerated

Building on:•FuGE•MAGE-TAB•XGAP•METABASE•PaGE


2009

2009

|

Step 3: evaluate on data (goto 1)

1: molgenis_db.xml

2: documentation*

3: exchange format*

*autogenerated

Tools•Db•CsvImport•CsvExport•constants


2009

2009

|

Step 3: evaluate on data (goto 1)1: molgenis_db.xml

2: documentation*

3: exchange format*

*autogenerated


2009

2009

|

Step 1b: add ui model

1: molgenis_db.xml

50 lines of XML code(forms, menu’s and plugins)


2009

2009

|

Result = db + ui + services + tools

1: molgenis_db.xml

2: brainstorm doc*

3: exchange format*

4: back and frontend*

Demo: http://wwwdev.ebi.ac.uk/microarray-srv/pheno/ Source: https://svn.gene.le.ac.uk/gen2phen/pheno-model

*autogenerated


2009

2009

|16

Connect to R statistics


Workflow ready web-services


UML documentation of your model


Edit & trace your data


Import/export to Excel


plugin your own scripts (OntBrowse)

plugin your own scripts (OntBrowse)

Tech keywords: object oriented data models, multi-platform java, tomcat/glassfish web server, mysql/postgresql database, Eclipse/Netbeans IDE, Java API, WSDL/SOAP API, R-project API, MVC, freemarker templates and css for custom layout, open source.

find.investigation()102 downloaded

obs<-find.observedvalue(43,920 downloaded

#some calculationadd.inferredvalue(res)36 added


2009

2009

|

Todo: semantic layer

1: molgenis_db.xml

2: brainstorm doc*

3: exchange format*

4: back and frontend*

5: ontology browser

*autogenerated

And:•UI for dealing with composite keys•Extend CSV parsers for other formats•Import/Export/Federate wizards to/from data repositories


2009

2009

|

Model evaluation and refinement

Model details after (1) implementation and (2) data loading

18


2009

2009

|

Data model overview

Core› Investigation› ObservableFeature› ObservationTarget › ObservedValue

Feature/Value coding› OntologySource› OntologyTerm

Code CodeList

ObservationTarget subclasses› Individual› Panel

ObservedValue subclasses› ObservedRelationship› InferredValue

Protocol› Protocol› ProtocolApplication› ProtocolParameter› ParameterValue

19


2009

2009

|

Evaluation process

› Loading: 102 studies (MPD, Europhenome, Molpage) 2,042 observable features 42,939 Individuals 287 Panels 196 protocols 140 ontology terms

› Most on mouse, limited on human Getting descriptions possible, data is hard Getting ontology annotated data is harder Recoding to Pheno-OM TAB format is easy

20


2009

2009

|

Core model (required)

• Targets, Features, Values have ‘names’ (Nameable)• Names are unique per investigation • Optional: use ontologyterm for semantic harmonization

21


2009

2009

|

ObservableFeature

› References ontology terms for unambiguous: Feature definition Feature unit

22

OntologyTermTerm=Length

ObservableFeatureName=Length(cm)ontologyReference=LengthUnit=UCU/cm

OntologyTermTerm=UCU/cm

ObservableFeatureName=Length(inch)ontologyReference=LengthUnit=UCU/inch

OntologyTermTerm=UCU/inch


2009

2009

|

ObservableValue23

- time series data- Repeated observations- time relative data (‘age’)

Standard codes/values


2009

2009

|

5x Investigation: redundant?

Trial coordination center use case:Investigation 5:

Reuses ‘standard’ Protocols from Investigation 1 Reuses ‘standard’ Features defined in Investigation 2 Reuses Individuals from Investigation 3 Outsourced ProtocolApplication to Investigation 4 Has values linked Investigation 5 (=self)

24

Protocol

ProtocolApplication

ObservedValue

ObservableFeature

Investigation

ObservationTarget


2009

2009

|

Protocol (optional)

› FuGE/MAGE

25

CRF sections

and forms

When?How?

(by whom?)

Groups observations

(medicin)

PossibleVariationsDeviations

(ontologyRef?)

Alternative protocols

for features

ActualVariationsDeviations

(ontologyRef?)


2009

2009

|

Value package (extension)26

SiblingHousehol

d

High blood pressure

High Cholesterol

How collected

Defined pheno type

Defined pheno value

- Systolic-Diastolic

-Last dinner-Cholesterol


2009

2009

|

Target package (extensions)27

Pedigree(relationship) Cohorts

Other groups(can have values too:

non-smoking, 65yr males)


2009

2009

|

Coding package (extension)28

CodeList < OntologyTermTerm=codelist/sexontologyReference= …

ObservableFeatureName=SexontologyReference=SexUnit=codelist/sex

ObservedValueValue=1ontologyReference=Transsexual, m-> f

Code < OntologyTermCode=1Term=Transsexual, m-> fontologyReference= …

Code List = categorical type of unit = termCode = ontology term, part of listPhenX example


2009

2009

|

Principle is enable, not constrain

› Partial usage- Features without Observations (just exchange designs)- Observations without Protocols (just exchange results)

› Harmonization optional- Use locally unique names- Use external application ontology for harmonization- (but can via reuse of features between investigations)

› Local and external ontologies- Only one term per item to prevent ambiguaty

› Don’t try to be everything to everybody› Semantic maps (synonyms, part of, subclass) outside

29


2009

2009

|

Future

Integrate, integrate, integrate

30


2009

2009

|

Integration on data structure

› Did import of BioMart, MPD, MAGE-TAB› Need import/export to main repositories

31

Pheno-OM FuGE + XGAP MAGE-TAB PaGEInvestigation Investigation Investigation InvestigationObservableFeature Trait Characteristic Observable_featureObservationTarget Subject Source Abstract_observation_targetIndividual Individual NA IndividualPanel Panel NA PanelObservedValue DataElement CharacteristicValue ObservedValueObservedRelationship NA NA NAInferredValue via ProtocolApplication NA ObservedValue (self assoc)OntologySource OntologySource OntologySource Ontology_sourceOntologyTerm OntologyIndividual OntologyTerm Ontology_termProtocol Protocol Protocol Observation_methodProtocolApplication ProtocolApplication ProtocolApplication Observation_methodCode NA Characteristic NACodeList NA NA NA


2009

2009

|

Integration on components

› Other omics phenotypes- XGAP, MAGE, PaGE, Gen2Phen models for

genotype/lsdb etc)

› Other formats- MAGE-TAB extension- XGAP extension- Gen2Phen module assembly- dbGaP/EGA pipelines

32


2009

2009

|

Integration on components

› Other omics phenotypes (XGAP, MAGE, PaGE, Gen2Phen models for genotype/lsdb etc)

33

DATA ELEMENT

TRAIT

SUBJECT

columns

rows

dimension ELEMENT

PROBE-Name-Gene-Chromosme-Locus

PROBE-Name-Gene-Chromosme-Locus

MARKER-Name-Allele-Chromosme-Locus

MARKER-Name-Allele-Chromosme-Locus

MASSPEAK-Name-MZ-RetentionTime

MASSPEAK-Name-MZ-RetentionTime

Panel-Name-Type: CSS, RIL..-Parent Panels

Panel-Name-Type: CSS, RIL..-Parent Panels

INDIVIDUAL-Name-Strain-Mother-Father-Sex

INDIVIDUAL-Name-Strain-Mother-Father-Sex

SAMPLE-Name-Individual-Tissue

SAMPLE-Name-Individual-TissueAnd so on

…

And so on…

And so on…And so on…


2009

2009

|

Genotype data

1. Data model 34

BxD1 BxD2 BxD3 BxD4 BxD5 BxD6 BxD7rs13475697 1 1 0 1 0 1 0rs13475698 1 0 0 0 0 0 1rs13475699 0 0 0 1 0 1 1rs13475700 1 1 1 1 0 1 0rs13475701 1 0 1 0 0 1 1rs2228909 1 1 0 1 0 0 0rs2228910 0 0 1 1 0 0 0rs3022775 0 0 0 1 1 0 1rs3024102 1 0 1 0 0 0 0rs3024103 1 0 0 1 0 0 0rs3024104 0 1 0 0 0 0 0rs3024105 0 0 1 0 0 0 1rs30462182 1 0 0 0 0 0 0rs30522279 0 1 0 0 1 0 0

MARKERS

Subjects: STRAINS

DATA ELEMENTS

Traits:

TRAIT SUBJECTTRAIT SUBJECT

Looking at standards and existing data setsSimple enough for everybody to create


2009

2009

| 35

› QTL› GWAS› Genotype› Expression› MassSpec› NMR› Etc.

› Extends › FuGE

› fuge.sourceforgen.net


2009

2009

|36











plugin your own scripts (R/QTL)

plugin your own scripts (R/QTL)

Tech keywords: object oriented data models, multi-platform java, tomcat/glassfish web server, mysql/postgresql database, Eclipse/Netbeans IDE, Java API, WSDL/SOAP API, R-project API, MVC, freemarker templates and css for custom layout, open source.

m<-find.markers()544 markers downloaded.…library(qtl)#qtl analysis here

add.data(qtl, name = “QTLs”)2,448,000 data elements added.

s tra in .tx tspec ies .tx tp ro toco l.tx tp robe .tx tm arker.tx tinvestiga tion .tx tind iv idua l.tx tgene .tx tda ta .tx tconstan t.p ropertiesda ta

eXtensible Genotype And Phenotype platform http://www.xgap.org


2009

2009

|

Integration on semantics

› Man/mouse mapping ontologies- Integrate data on equivalent features- Usage: R2D server + sparql?

› Semantic search/query expansion- Exploit synonyms, part-of, subclass, equivalence- Usage: super search box

› Barriers to progress- Lack of cross species mapping ontologies- Lack of public, ontology annotated, human data

37

“Give me all individuals that have deformed hand … across studies … across species”


2009

2009

|

Pheno-OM summary

› Simple system for phenotype representation

minimal conceptual model, but not too minimal

unambiguous entity naming (so not ‘phenotype’)

works on real data

exchange multiple investigations in one envelop

easy to create/parse/convert-into format

interface/modules that can be adopted by others

~ontology enabled for querying/integration (NOW)

38

Demo and docs: http://wwwdev.ebi.ac.uk/microarray-srv/pheno/ Source: https://svn.gene.le.ac.uk/gen2phen/pheno-model


2009

2009

|

Questions, suggestions?39


2009

2009

|

Example

› Questionnaire Q1 (protocol)› Observes low/high blood pressures (features)› Using X blood pressure device (protocol)› On patients 200-300 (persons)› At 14 sep 2009 (protocol application)› Resulting in values (125/90, 120/80, 120,75)› In Hg (unit)› At time = x+5,10,15mins (value.timestamp)

40


2009

2009

|

Implementation41

customize... Model file XML

Generate

APIs in Java, R, Web services and HTTP

MyScriptPlugins

FormGen

MenuGenTreeGen

PluginGenMatrixGen

JTypeGenJDBCMapGen

JListGenJReadCsvGen

HSQLGen

JDatabaseGen

MySQLGen

RMatrixGen

WSGen

RListGen

datainfrastructure

user interactioninfrastructure

Communicationinfrastructure


2009

2009

| 42

Documents

| [email protected] METABASE and PSI, P3G workshop 2009 2009 2009 | Pheno-OM implement, evaluate, refine Morris A. Swertz, Tomasz Adamusiak, Juha Muilu,