Towards a (united) federation of Bioinformatics resources

05/01/2023

1

TOWARDS A (UNITED) FEDERATION OF BIOINFORMATICS RESOURCES

Matthew Vaughn @mattdotvaughnDirector, Life Sciences Computing, TACC | Co-PI Cyverse, Araport, Jetstream

Interoperability and Federation Across Bioinformatic Platforms and Resources

Jan 14, 2017

05/01/2023

2

WHY FEDERATE?

Because you can’t do it all or be it all. And would you even want to?

05/01/2023

3

WHY FEDERATE?There’s always some existing or emergent

• Data Set • Database • Visualization Technology • Software Algorithm or Library • Physical Capacity or Capability • Source of funding and support

not in scope for you to directly provide or avail yourself of

Federated infrastructures are TEAM-BUILT

Increase the resiliency of your informatics ecosystem

Leverage all the other brains who have different views of your problem

WHY DON’T WE FEDERATE BY DEFAULT?

05/01/2023

4

Federation requires three things:

• Components confirming to “standardized" schemas, protocols for interaction and usage

• Stably-operated frameworks to handle yeoman’s work of integrating components

05/01/2023

5

05/01/2023

6

WHY DON’T WE FEDERATE BY DEFAULT?

Hey wait.. I said there were three things we needed for federation:

05/01/2023

7

WHY DON’T WE FEDERATE BY DEFAULT?

Hey wait.. I said there were three things we needed for federation:

Planning & Specific Effort

05/01/2023

8

Lab-Born Software

• Immediately responsive• Limited R&D• Resources on hand• Sustainability? What’s

that?

Centrally-Planned Software

• Mindfully built• Better chance for R&D• Dedicated resources• Sustainability? What’s

that?

WHY DON’T WE FEDERATE BY DEFAULT?

Some of the most interesting work is done at the edges of our infrastructure. Their adopting federated access patterns post-hoc means assuming substantial technical debt.

05/01/2023

9

Science applications

Domain-specific services

Establishedsoftware and CI

Physical resources

Federated Storage National CI Virtualization Job

SchedulingSingle

Sign-on

Ease

of U

se

Ease

of R

e-us

e

05/01/2023

10

HOW CAN WE MAKE FEDERATION EASIER?Deeply understand the capabilities of existing integration platforms

• Avoid Not-Invented-Here by adopting the 80% rule• Contribute enhancements, either via active feedback or by coding them• Build on our platforms and make sure they get credit for their role

Identify and adopt existing standards. Contribute where they fall short of our needs• OpenAPI for web service definitions• ISA Framework for experimental metadata• Oauth2 for authorization

Provide tooling and documentation for users with diverse technical backgrounds

• GUI, Forms, Web Services. But also language libraries and SDKs.• Make sure we understand the motivations and constraints of those users• Write cookbooks, not just shopping lists

05/01/2023

11

Jetstream GUI

Agave API Developer Portal

DesignSafe.CI Workbench

Araport Web Services

MAKING FEDERATION WORK REQUIRES THAT WE INCREASE EVERYONE’S

PRODUCTIVITY

05/01/2023

12

@mattdotvaughn www.slideshare.net/mattdotvaughn vaughn@tacc.utexas.edu

05/01/2023

13