178

NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

  • Upload
    others

  • View
    17

  • Download
    0

Embed Size (px)

Citation preview

Page 1: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts
Page 2: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

NISTBIGDATAINTEROPERABILITYFRAMEWORK:VOLUME8,REFERENCEARCHITECTUREINTERFACES

Editor:GregorvonLaszewski,WorkingGroupco-Chair:WoChang

Copyrights andPermissions:Official publications of theNational Institute of Standards andTechnology are not subject to copyright in the United States. Foreign rights are reserved.QuestionsconcerningthepossibilityofcopyrightsinforeigncountriesshouldbereferredtotheOfficeofChiefCounselatNISTviaemailtonistcounsel@nist.gov.

Page 3: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

NISTBIGDATAINTEROPERABILITYFRAMEWORK:VOLUME8,REFERENCEARCHITECTUREINTERFACES

NISTBigDataInteroperabilityFramework:Volume8,ReferenceArchitectureInterfaces1Introduction1.1Background1.2ScopeandObjectivesoftheReferenceArchitecturesSubgroup1.3ReportProduction1.4ReportStructure

2NBDRAInterfaceRequirements2.1High-LevelRequirementsoftheInterfaceApproach2.1.1Technology-andVendor-Agnostic2.1.2SupportofPlug-InComputeInfrastructure2.1.3OrchestrationofInfrastructureandServices2.1.4OrchestrationofBigDataApplicationsandExperiments2.1.5Reusability2.1.6ExecutionWorkloads2.1.7SecurityandPrivacyFabricRequirements

2.2Component-SpecificInterfaceRequirements2.2.1SystemOrchestratorInterfaceRequirements2.2.2DataProviderInterfaceRequirements2.2.3DataConsumerInterfaceRequirements2.2.4BigDataApplicationInterfaceProviderRequirements2.2.5BigDataProviderFrameworkInterfaceRequirements2.2.6BigDataApplicationProvidertoBigDataFrameworkProviderInterface

3SpecificationParadigm3.1HybridandMultipleFrameworks3.2DesignbyResource-OrientedArchitecture3.3DesignbyExample3.4VersionManagement3.5InterfaceCompliancy3.6Refernceimplementations

4Specification4.1Listofspecifications4.2Authentication4.3StatusCodesandErrorResponses4.4Timestamp4.4.1Timestamp

4.5Identity4.5.1Organization4.5.2User4.5.3Account

Page 4: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.5.4PublicKeyStore4.6Variable,Defualt,andAlias4.6.1Alias4.6.2Variables4.6.3Default

4.7DataManagement4.7.1Filestore4.7.2Replica4.7.3Database4.7.4VirtualDirectory

4.8ComputeManagement-VirtualClusters4.8.1VirtualCluster4.8.2NetworkofNodes4.8.3Scheduler4.8.4Queue

4.9ComputeManagement-VirtualMachines4.9.1Image4.9.2Flavor4.9.3VirtualMachine4.9.4Secgroup4.9.5Nic

4.10ComputeManagement-Containers4.10.1Containers

4.11ComputeManagement-MapReduce4.11.1MapReduce

4.12ComputeManagement-Functions4.12.1Microservice

4.13Reservation4.13.1Reservation

4.14DataStreams4.14.1Stream4.14.2Filter

4.15Deployment4.15.1Deployment

5AcronymsandTermsBibliography

Page 5: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

NISTBIGDATAINTEROPERABILITYFRAMEWORK:VOLUME8,REFERENCEARCHITECTUREINTERFACES

Page 6: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts
Page 7: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Abstract

Page 8: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

This document summarizes interfaces that are instrumental for the interaction with Clouds,Containers, andHigh Performance Computing (HPC) systems tomanage virtual clusters tosupport the NIST Big Data Reference Architecture (NBDRA). The REpresentational StateTransfer (REST) paradigm is used to define these interfaces, allowing easy integration andadoptionbyawidevarietyofframeworks.

Big Data is a term used to describe extensive datasets, primarily in the characteristics ofvolume,variety,velocity,and/orvariability.WhileopportunitiesexistwithBigData,thedatacharacteristics can overwhelm traditional technical approaches, and the growth of data isoutpacingscientificandtechnologicaladvancesindataanalytics.ToadvanceprogressinBigData,theNISTBigDataPublicWorkingGroup(NBD-PWG)isworkingtodevelopconsensusonimportantfundamentalconceptsrelatedtoBigData.Theresultsarereportedin theNISTBigDataInteroperabilityFramework(NBDIF) seriesofvolumes.Thisvolume,Volume8,usestheworkperformedbytheNBD-PWGtoidentifyobjectsinstrumentalfortheNISTBigData Reference Architecture (NBDRA) which is introduced in the NBDIF: Volume 6,ReferenceArchitecture.

Keywords

Adoption; barriers; implementation; interfaces; market maturity; organizational maturity;projectmaturity;systemmodernization.

Acknowledgements

This document reflects the contributions and discussions by the membership of the NBD-PWG, cochaired byWoChang (NIST ITL),BobMarcus (ET-Strategies), andChaitanBaru(San Diego Supercomputer Center; National Science Foundation). For all versions, theSubgroupswere led by the followingpeople:NancyGrady (SAIC),NatashaBalac (SDSC),and Eugene Luster (R2AD) for the Definitions and Taxonomies Subgroup; Geoffrey Fox(Indiana University) and Tsegereda Beyene (Cisco Systems) for the Use Cases andRequirements Subgroup; Arnab Roy (Fujitsu), Mark Underwood (Krypton Brothers;Synchrony Financial), and Akhil Manchanda (GE) for the Security and Privacy Subgroup;David Boyd (InCadence Strategic Solutions), Orit Levin (Microsoft), Don Krapohl(Augmented Intelligence), and James Ketner (AT&T) for the Reference ArchitectureSubgroup; and Russell Reinsch (Center for Government Interoperability), David Boyd(InCadenceStrategicSolutions),CarlBuffington(Vistronix),andDanMcClary(Oracle), forthe Standards Roadmap Subgroup, Gregor von Laszewski (Indiana University) for theInterfaceSubgroup.

Thefollowingmilestonereleasesexist:

Version 2.1: A previous volume used the definitions of the schema based onexamplesonly.Itwaseasiertoreadbutonlyincludedthedefinitionoftheresourcesandnottheinteractionwiththeresources.ThisvolumewasinplaceuntilJune2018.Version 2.2: This version was significantly changed and used OpenAPI 2.0 to

Page 9: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

specifytheinterfacesbetweenthevariousservicesandcomponents.Version 3.1.1: The version includes significant improvements of the objectspecificationsbutarestillusingOpenAPI2.0.Version3.2.0:All specificationshavebeenupdated toOpenAPI3.0.2.Significantupdateshavebeendonetoanumberofspecifications.

Theeditorsforthesedocumentsare:

GregorvonLaszewski(IndianaUniversity)WoChang(NIST).

Laurie Aldape (Energetics Incorporated) and Elizabeth Lennon (NIST) provided editorialassistanceacrossallNBDIFvolumes.

NIST SP 1500-9, Draft NIST Big Data Interoperability Framework: Volume 8, ReferenceArchitectureInterfaces,Version2hasbeencollaborativelyauthoredbytheNBD-PWG.Asofthe date of publication, there are over six hundred NBD-PWG participants from industry,academia, and government. Federal agency participants include the National Archives andRecords Administration (NARA), National Aeronautics and Space Administration (NASA),National Science Foundation (NSF), and the U.S. Departments of Agriculture, Commerce,Defense, Energy, Census, Health and Human Services, Homeland Security, Transportation,Treasury,andVeteransAffairs.

NISTwould like toacknowledge thespecificcontributions to thisvolume,duringVersion1and/orVersion2activities.ContributorsaremembersoftheNISTBigDataPublicWorkingGroupwhodedicated great effort to prepare and gave substantial timeon a regular basis toresearchanddevelopmentinsupportofthisdocument.ThisincludesthefollowingNBD-PWGmembers:

GregorvonLaszewski,IndianaUniversityWoChang,NationalInstituteofStandardandTechnology,FugangWang,IndianaUniversityGeoffreyC.Fox,IndianaUniversityShirishJoshi,IndianaUniversityBadiAbdul-Wahid,formerlyIndianaUnivresityAliciaZuniga-Alvarado,ConsultantRobertC.Whetsel,DISA/NBISPratikThakkar,Philips

ExecutiveSummary

The NIST Big Data Interoperability Framework (NBDIF): Volume 8, ReferenceArchitectureInterfacesdocumentwaspreparedbytheNISTBigDataPublicWorkingGroup(NBD-PWG)ReferenceArchitectureSubgroup to identify interfaces in support of theNISTBigDataReferenceArchitecture(NBDRA).Theinterfacedefineresourcesthatarepartofthe

Page 10: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

NBDRA.TheseresourcesareformulatedinOpenAPI3.0.2formatandcanbeeasilyintegratedintoaREpresentationalStateTransfer(REST)frameworkoranobject-basedframework.

TheresourceswerecategorizedingroupsthatareidentifiedbytheNBDRAsetforwardintheNBDIF: Volume 6, Reference Architecture document.While theNBDIF: Volume 3, UseCases and General Requirements document provides application-oriented high-level usecases, theusecasesdefinedin thisdocumentaresubsetsof themandfocuson interfaceusecases.Theinterfaceusecasesarenotmeanttobecompleteexamples,butshowcasewhytheresourcehasbeendefined.Hence,theinterfacesusecasesareonlyrepresentative,anddonotencompasstheentirespectrumofBigDatausage.Alltheinterfaceswereopenlydiscussedintheworkinggroup[1].AdditionstotheinterfacesarewelcomeandtheNBD-PWGisopentodiscussanycontributions.

TheNIST Big Data Interoperability Framework (NBDIF) was released in three versions,whichcorrespond to the threestagesof theNBD-PWGwork.Version3 (currentversion)oftheNBDIFvolumesresultedfromStage3workwithmajoremphasisonthevalidationoftheNBDRAInterfacesandcontentenhancement.Stage3workbuiltuponthefoundationcreatedduringStage 2 andStage 1.The current effort documented in this volume reflects conceptsdevelopedwithintherapidlyevolvingfieldofBigData.Thethreestages(inreverseorder)aimtoachievethefollowingwithrespecttotheNISTBigDataReferenceArchitecture(NBDRA):

Stage 1: Identify the high-level Big Data reference architecture key components,whicharetechnology-,infrastructure-,andvendor-agnostic.Stage2:DefinegeneralinterfacesbetweentheNBDRAcomponents;andStage3:ValidatetheNBDRAbybuildingBigDatageneralapplicationsthroughthegeneralinterfaces;

TheNBDIFconsistsofninevolumes,eachofwhichaddressesaspecifickeytopic,resultingfromtheworkoftheNBD-PWG.Theninevolumesareasfollows:

Volume1,Definitions[2]Volume2,Taxonomies[3]Volume3,UseCasesandGeneralRequirements[4]Volume4,SecurityandPrivacy[5]Volume5,ArchitecturesWhitePaperSurvey[6]Volume6,ReferenceArchitecture[7]Volume7,StandardsRoadmap[8]Volume8,ReferenceArchitectureInterfaces(thisvolume)[9]Volume9,AdoptionandModernization[10]

During Stage 1, Volumes 1 through 7 were conceptualized, organized and written. ThefinalizedVersion1documents canbedownloaded from theV1.0FinalVersionpageof theNBD-PWGwebsite[11].

DuringStage2,theNBD-PWGdevelopedVersion2oftheNBDIFVersion1volumes,withtheexceptionofVolume5,whichcontainedthecompletedarchitecturesurveyworkthatwas

Page 11: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

usedtoinformStage1workof theNBD-PWG.ThegoalsofVersion2weretoenhancetheVersion1content,definegeneralinterfacesbetweentheNBDRAcomponentsbyaggregatinglow-levelinteractionsintohigh-levelgeneralinterfaces,anddemonstratehowtheNBDRAcanbeused.AsaresultoftheStage2work,theneedforNBDIFVolume8andNBDIFVolume9were identified and the two new volumes were created. Version 2 of the NBDIF volumes,resulting from Stage 2 work, can be downloaded from the V2.0 Final Version page of theNBD-PWGwebsite[12].

Thisdocument is theresultofStage3workof theNBD-PWG.Coordinationof thegroupisconductedontheNBD-PWGwebpage[1].

Page 12: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

1INTRODUCTION

1.1BACKGROUND

There is broad agreement among commercial, academic, and government leaders about thepotentialofBigDatatosparkinnovation,fuelcommerce,anddriveprogress.BigDataisthecommontermusedtodescribethedelugeofdataintoday’snetworked,digitized,sensor-laden,and information-drivenworld.The availabilityof vast data resources carries thepotential toanswerquestionspreviouslyoutofreach,includingthefollowing:

Howcanapotentialpandemicreliablybedetectedearlyenoughtointervene?Can new materials with advanced properties be predicted before these materialshaveeverbeensynthesized?Howcanthecurrentadvantageoftheattackeroverthedefenderinguardingagainstcybersecuritythreatsbereversed?

ThereisalsobroadagreementontheabilityofBigDatatooverwhelmtraditionalapproaches.The growth rates for data volumes, speeds, and complexity are outpacing scientific andtechnologicaladvancesindataanalytics,management,transport,anddatauserspheres.

Despite widespread agreement on the inherent opportunities and current limitations of BigData, a lack of consensus on some important fundamental questions continues to confusepotentialusersandstymieprogress.Thesequestionsincludethefollowing:

HowisBigDatadefined?WhatattributesdefineBigDatasolutions?WhatisnewinBigData?WhatisthedifferencebetweenBigDataandbiggerdatathathasbeencollectedforyears?How is Big Data different from traditional data environments and relatedapplications?WhataretheessentialcharacteristicsofBigDataenvironments?Howdotheseenvironmentsintegratewithcurrentlydeployedarchitectures?What are the central scientific, technological, and standardization challenges thatneed to be addressed to accelerate the deployment of robust, secure Big Datasolutions?

Within thiscontext,onMarch29,2012, theWhiteHouseannounced theBigDataResearchandDevelopmentInitiative(TheWhiteHouseOfficeofScienceandTechnologyPolicy,“BigData is a Big Deal,”OSTP Blog, accessed February 21, 2014 [13]. The initiative’s goalsincludehelping toaccelerate thepaceofdiscovery inscienceandengineering,strengtheningnational security, and transforming teaching and learning by improving analysts’ ability toextractknowledgeandinsightsfromlargeandcomplexcollectionsofdigitaldata.

Page 13: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Sixfederaldepartmentsandtheiragenciesannouncedmorethan$200millionincommitmentsspread across more than 80 projects, which aim to significantly improve the tools andtechniques needed to access, organize, and draw conclusions from huge volumes of digitaldata.Theinitiativealsochallengedindustry,researchuniversities,andnonprofitstojoinwiththefederalgovernmenttomakethemostoftheopportunitiescreatedbyBigData.

Motivated by the White House initiative and public suggestions, the National Institute ofStandards and Technology (NIST) accepted the challenge to stimulate collaboration amongindustryprofessionalstofurtherthesecureandeffectiveadoptionofBigData.AsaresultofNIST’s Cloud and Big Data Forum held on January 15–17, 2013, there was strongencouragementforNISTtocreateapublicworkinggroupforthedevelopmentofaBigDataStandardsRoadmap.Forumparticipantsnoted that this roadmapshoulddefineandprioritizeBig Data requirements, including interoperability, portability, reusability, extensibility, datausage,analytics,andtechnologyinfrastructure.Indoingso,theroadmapwouldacceleratetheadoptionofthemostsecureandeffectiveBigDatatechniquesandtechnology.

OnJune19,2013,theNISTBigDataPublicWorkingGroup(NBD-PWG)waslaunchedwithextensive participation by industry, academia, and government from across the nation. Thescope of the NBD-PWG involves forming a community of interests from all sectors—including industry, academia, and government—with the goal of developing consensus ondefinitions,taxonomies,securereferencearchitectures,securityandprivacy,and,fromthese,astandards roadmap. Such a consensus would create a vendor-neutral, technology- andinfrastructure-independentframeworkthatwouldenableBigDatastakeholderstoidentifyanduse the best analytics tools for their processing and visualization requirements on themostsuitable computing platform and cluster, while also allowing added value from Big Dataserviceproviders.

TheNIST Big Data Interoperability Framework (NBDIF) was released in three versions,whichcorrespond to the threestagesof theNBD-PWGwork.Version3 (currentversion)oftheNBDIFvolumesresultedfromStage3workwithmajoremphasisonthevalidationoftheNBDRAInterfacesandcontentenhancement.Stage3workbuiltuponthefoundationcreatedduringStage 2 andStage 1.The current effort documented in this volume reflects conceptsdevelopedwithintherapidlyevolvingfieldofBigData.Thethreestages(inreverseorder)aimtoachievethefollowingwithrespecttotheNISTBigDataReferenceArchitecture(NBDRA).

Stage3:Validate theNBDRAbybuildingBigDatageneralapplicationsthroughthegeneralinterfaces;Stage2:DefinegeneralinterfacesbetweentheNBDRAcomponents;andStage1:Identifythehigh-levelBigDatareferencearchitecturekeycomponents,whicharetechnology-,infrastructure-,andvendor-agnostic.

TheNBDIFconsistsofninevolumes,eachofwhichaddressesaspecifickeytopic,resultingfromtheworkoftheNBD-PWG.Theninevolumesareasfollows:

Volume1,Definitions[2]Volume2,Taxonomies[3]Volume3,UseCasesandGeneralRequirements[4]

Page 14: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Volume4,SecurityandPrivacy[5]Volume5,ArchitecturesWhitePaperSurvey[6]Volume6,ReferenceArchitecture[7]Volume7,StandardsRoadmap[8]Volume8,ReferenceArchitectureInterfaces(thisvolume)[9]Volume9,AdoptionandModernization[10]

During Stage 1, Volumes 1 through 7 were conceptualized, organized and written. ThefinalizedVersion1documents canbedownloaded from theV1.0FinalVersionpageof theNBD-PWGwebsite[11].

DuringStage2,theNBD-PWGdevelopedVersion2oftheNBDIFVersion1volumes,withtheexceptionofVolume5,whichcontainedthecompletedarchitecturesurveyworkthatwasusedtoinformStage1workof theNBD-PWG.ThegoalsofVersion2weretoenhancetheVersion1content,definegeneralinterfacesbetweentheNBDRAcomponentsbyaggregatinglow-levelinteractionsintohigh-levelgeneralinterfaces,anddemonstratehowtheNBDRAcanbeused.AsaresultoftheStage2work,theneedforNBDIFVolume8andNBDIFVolume9were identified and the two new volumes were created. Version 2 of the NBDIF volumes,resulting from Stage 2 work, can be downloaded from the V2.0 Final Version page of theNBD-PWGwebsite[12].

1.2SCOPEANDOBJECTIVESOFTHEREFERENCEARCHITECTURESSUBGROUP

Referencearchitecturesprovide“anauthoritativesourceofinformationaboutaspecificsubjectareathatguidesandconstrainstheinstantiationsofmultiplearchitecturesandsolutions”[14].Referencearchitecturesgenerallyserveasafoundationforsolutionarchitecturesandmayalsobeusedforcomparisonandalignmentofinstantiationsofarchitecturesandsolutions.

ThegoaloftheNBD-PWGReferenceArchitectureSubgroupistodevelopanopenreferencearchitectureforBigDatathatachievesthefollowingobjectives:

Providesacommonlanguageforthevariousstakeholders;Encouragesadherencetocommonstandards,specifications,andpatterns;Provides consistent methods for implementation of technology to solve similarproblemsets;Illustrates and improves understanding of the various Big Data components,processes, and systems, in the context of a vendor- and technology-agnostic BigDataconceptualmodel;ProvidesatechnicalreferenceforU.S.governmentdepartments,agencies,andotherconsumerstounderstand,discuss,categorize,andcompareBigDatasolutions;andFacilitates analysis of candidate standards for interoperability, portability,reusability,andextendibility.

The NBDRA is a high-level conceptual model crafted to serve as a tool to facilitate opendiscussion of the requirements, design structures, and operations inherent in Big Data. The

Page 15: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

NBDRAisintendedtofacilitatetheunderstandingoftheoperationalintricaciesinBigData.ItdoesnotrepresentthesystemarchitectureofaspecificBigDatasystem,butratherisatoolfordescribing, discussing, and developing system-specific architectures using a commonframework of reference. Themodel is not tied to any specific vendor products, services, orreferenceimplementation,nordoesitdefineprescriptivesolutionsthatinhibitinnovation.

TheNBDRAdoesnotaddressthefollowing:

Detailedspecificationsforanyorganization’soperationalsystems;Detailedspecificationsofinformationexchangesorservices;andRecommendationsorstandardsforintegrationofinfrastructureproducts.

The goals of the Subgroupwere realized throughout the three planned phases of theNBD-PWGwork,asoutlinedinSection1.3.

1.3REPORTPRODUCTION

TheNBDIF:Volume8,ReferencesArchitecture Interfaces is one of nine volumes,whoseoverall aims are to define and prioritize Big Data requirements, including interoperability,portability, reusability, extensibility, data usage, analytic techniques, and technologyinfrastructure tosupportsecureandeffectiveadoptionofBigData.Theoverallgoalsof thisvolumearetodefineandspecifyinterfacestoimplementtheBigDataReferenceArchitecture.This volume arose from discussions during theweeklyNBD-PWGconference calls. Topicsincluded in this volumebegan to take form inPhase 2 of theNBD-PWGwork.During thediscussions,theNBD-PWGidentifiedtheneedtospecifyavarietyofinterfaces.

Phase3work,whichbuiltuponthegroundworkdevelopedduringPhase2,includedanearlyspecificationbasedonresourceobjectspecifications thatprovidedasimplifiedversionofanAPIinterfacedesign.

1.4REPORTSTRUCTURE

To enable interoperability between theNBDRAcomponents, a list ofwell-definedNBDRAinterfacesisneeded.Theseinterfacesaredocumentedinthisvolume.Tointroducethem,theNBDRA structure will be followed, focusing on interfaces that allow bootstrapping of theNBDRA.Thedocumentbeginswithasummaryofrequirementsthatwillbeintegratedintoourspecifications. Subsequently, each sectionwill introduce a number of objects that build thecore of the interface addressing a specific aspect of the NBDRA. A selected number ofinterfaceusecaseswillbe showcased tooutlinehow thespecific interfacecanbeused inareference implementationof theNBDRA.Validationof thisapproachcanbeachievedwhileapplyingittotheapplicationusecasesthathavebeengatheredintheNBDIF:Volume3,UseCasesandRequirementsdocument.TheseapplicationusecaseshaveconsiderablycontributedtowardsthedesignoftheNBDRA.Hencetheexpectationisthat:(1)theinterfacescanbeusedto help implement a Big Data architecture for a specific use case; and (2) the proper

Page 16: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

implementation.This approach can facilitate subsequent analysis and comparison of the usecases.

TheorganizationofthisdocumentroughlycorrespondstotheprocessusedbytheNBD-PWGto develop the interfaces. Following the introductory material presented in Section 1, theremainderofthisdocumentisorganizedasfollows:

Section2presentstheinterfacerequirements;Section3presentsthespecificationparadigmthatisused;Section 4 presents several objects grouped by functional use while providing asummarytableofselectedproposedobjectsinSection4.1.

WhileeachNBDIFvolumewascreatedwithaspecificfocuswithinBigData,allvolumesareinterconnected.Duringcreation,thevolumesgaveand/orreceivedinputfromothervolumes.Broad topics (e.g., definition, architecture) may be discussed in several volumes with thediscussioncircumscribedbythevolume’sparticularfocus.ArrowsshowninFigure1indicatethemain flow of input/output.Volumes 2, 3, and 5 (blue circles) are essentially standalonedocumentsthatprovideoutputtoothervolumes(e.g.,toVolume6).Thesevolumescontaintheinitialsituationalawarenessresearch.Volumes4,7,8,and9(greencircles)primarilyreceivedinputfromothervolumesduringthecreationoftheparticularvolume.Volumes1and6(redcircles)were developed using the initial situational awareness research and continued to bemodified based onwork in other volumes. These volumes also provided input to the greencirclevolumes.

Page 17: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Figure1:NBDIFDocumentsNavigationDiagramProvidesContentFlowBetweenVolumes

Page 18: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

2NBDRAINTERFACEREQUIREMENTS

The development of aBigData reference architecture requires a thorough understanding ofcurrent techniques, issues, and concerns. To this end, theNBD-PWGcollected use cases togain an understanding of current applications of BigData, conducted a survey of referencearchitectures to understand commonalitieswithinBigData architectures in use, developed ataxonomy to understand and organize the information collected, and reviewed existingtechnologiesandtrendsrelevant toBigData.Theresultsof theseNBD-PWGactivitieswereused in the development of the NBDRA (Figure 2) and the interfaces presented herein.DetaileddescriptionsoftheseactivitiescanbefoundintheothervolumesoftheNBDIF.

Figure2:NISTBigDataReferenceArchitecture(NBDRA)

Thisvendor-neutral,technology-andinfrastructure-agnosticconceptualmodel,theNBDRA,isshown in Figure 2 and represents a Big Data system composed of five logical functionalcomponents connectedby interoperability interfaces (i.e., services).Two fabrics envelop thecomponents,representingtheinterwovennatureofmanagementandsecurityandprivacywithall five of the components. These two fabrics provide services and functionality to the fivemain roles in the areas specific toBigData and are crucial to anyBigData solution.Note:None of the terminology or diagrams in these documents is intended to be normative or to

Page 19: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

imply any business or deployment model. The terms provider and consumer as used aredescriptiveofgeneralrolesandaremeanttobeinformativeinnature.

TheNBDRA isorganized around fivemajor roles andmultiple sub-roles aligned along twoaxesrepresentingthetwoBigDatavaluechains:theInformationValue(horizontalaxis)andtheInformationTechnology(IT;verticalaxis).AlongtheInformationValueaxis,thevalueiscreatedbydatacollection, integration, analysis, andapplying the results following thevaluechain. Along the IT axis, the value is created by providing networking, infrastructure,platforms,applicationtools,andotherITservicesforhostingofandoperatingtheBigDatainsupport of required data applications. At the intersection of both axes is the Big DataApplication Provider role, indicating that data analytics and its implementation provide thevaluetoBigDatastakeholdersinbothvaluechains.ThetermprovideraspartoftheBigDataApplication Provider andBigData Framework Provider is there to indicate that those rolesprovideorimplementspecificactivitiesandfunctionswithinthesystem.Itdoesnotdesignateaservicemodelorbusinessentity.

TheDATAarrowsinFigure2showtheflowofdatabetweenthesystem’smainroles.Dataflowsbetween the roleseitherphysically (i.e.,byvalue)orbyproviding its locationand themeans to access it (i.e., by reference). The SW arrows show transfer of software tools forprocessing of Big Data in situ. The Service Use arrows represent software programmableinterfaces.WhilethemainfocusoftheNBDRAistorepresenttherun-timeenvironment,allthreetypesofcommunicationsortransactionscanhappenintheconfigurationphaseaswell.Manual agreements (e.g., service-level agreements) and human interactions that may existthroughoutthesystemarenotshownintheNBDRA.

DetailedinformationontheNBDRAconceptualmodelispresentedintheNBDIF:Volume6,ReferenceArchitecturedocument.

Priortooutliningthespecificinterfaces,generalrequirementsareintroducedandtheinterfacesaredefined.

2.1HIGH-LEVELREQUIREMENTSOFTHEINTERFACEAPPROACH

Thissectionfocusesonthehigh-levelrequirementsoftheinterfaceapproachthatareneededtoimplementthereferencearchitecturedepictedinFigure2.

2.1.1Technology-andVendor-Agnostic

Due to themanydifferent tools, services,and infrastructuresavailable in thegeneralareaofBigData,aninterfaceoughttobeasvendor-independentaspossible,while,atthesametime,be able to leveragebest practices.Hence, amethodology is needed that allows extensionofinterfacestoadaptandleverageexistingapproaches,butalsoallowstheinterfacestoprovidemeritineasyspecificationsthatassisttheformulationanddefinitionoftheNBDRA.

Page 20: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

2.1.2SupportofPlug-InComputeInfrastructure

AsBigData isnot justabouthostingdata,butaboutanalyzingdata, the interfacesprovidedhereinmustencapsulatearichinfrastructureenvironmentthatisusedbydatascientists.Thisincludestheabilitytointegrate(orplug-in)variouscomputeresourcesandservicestoprovidethe necessary compute power to analyze the data. These resources and services include thefollowing:

Access to hierarchy of compute resources from the laptop/desktop, servers, dataclusters,andclouds;Theabilitytointegratespecial-purposehardwaresuchasgraphicsprocessingunits(GPUs) and field-programmable gate arrays (FPGAs) that are used in acceleratedanalysisofdata;andThe integration of services includingmicroservices that allow the analysis of thedata by delegating them to hosted or dynamically deployed services on theinfrastructureofchoice.

2.1.3OrchestrationofInfrastructureandServices

Fromreviewof theusecasecollection,presented in theNBDIF:Volume3,UseCasesandGeneralRequirements document [4], theneedarose to address themechanismofpreparingsuitableinfrastructuresforvarioususecases.Asnoteveryinfrastructureissuitedforeveryusecase, a custom infrastructure may be needed. As such, this document is not attempting todeliver a single deployedNBDRA,but allow the setupof an infrastructure that satisfies theparticular use case. To achieve this task, it is necessary to provision software stacks andserviceswhileorchestratingtheirdeploymentandleveraginginfrastructures.Itisnotthefocusof this document to replace existing orchestration software and services, but provide aninterfacetothemtoleveragethemaspartofdefiningandcreatingtheinfrastructure.Variousorchestrationframeworksandservicescouldthereforebeleveraged,evenaspartofthesameframework,andworkinorchestratedfashiontoachievethegoalofpreparinganinfrastructuresuitableforoneormoreapplications.

2.1.4OrchestrationofBigDataApplicationsandExperiments

The creation of the infrastructure suitable for Big Data applications provides the basiccomputing environment. However, Big Data applications may require the creation ofsophisticatedapplicationsaspartofinteractiveexperimentstoanalyzeandprobethedata.Forthis purpose, the applications must be able to orchestrate and interact with experimentsconducted on the data while assuring reproducibility and correctness of the data. For thispurpose,aSystemOrchestrator(eitherthedatascientistsoraserviceactingonbehalfofthedatascientist)isusedasthecommandcentertointeractonbehalfoftheBigDataApplicationProvider to orchestrate dataflow fromDataProvider, carry out theBigData application lifecyclewith thehelp of theBigDataFrameworkProvider, and enable theDataConsumer toconsumeBigDataprocessingresults.Aninterfaceisneededtodescribetheseinteractionsand

Page 21: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

to allow leveraging of experiment management frameworks in scripted fashion. Acustomization of parameters is needed on several levels. On the highest level, application-motivatedparametersareneededtodrivetheorchestrationoftheexperiment.Onlowerlevels,these high-level parameters may drive and create service-level agreements, augmentedspecifications, andparameters that could even lead to theorchestrationof infrastructure andservicestosatisfyexperimentneeds.

2.1.5Reusability

The interfaces provided must encourage reusability of the infrastructure, services, andexperimentsdescribedby them.This includes (1) reusabilityof available analytics packagesandservicesforadoption;(2)deploymentofcustomizableanalyticstoolsandservices;and(3)operational adjustments that allow the services and infrastructure to be adaptedwhile at thesametimeallowingforreproducibleexperimentexecution.

2.1.6ExecutionWorkloads

Oneof the important aspects of distributedBigData services can be that the data served issimply too big to be moved to a different location. Instead, an interface could allow thedescriptionandpackagingofanalyticsalgorithms,andpotentiallyalsotools,asapayloadtoadataservice.Thiscanbebestachieved,notbysendingthedetailedexecution,butbysendingan interface description that describes how such an algorithm or tool can be created on theserver and be executed under security considerations (integrated with authentication andauthorizationinmind).

2.1.7SecurityandPrivacyFabricRequirements

Althoughthefocusofthisdocumentisnotsecurityandprivacy,whicharedocumentedintheNBDIF:Volume4,SecurityandPrivacy[5],theinterfacesdefinedhereinmustbecapableofintegration into a secure reference architecture that supports secure execution, secure datatransfer, and privacy. Consequently, the interfaces defined herein can be augmented withframeworks and solutions that provide such mechanisms. Thus, diverse requirement needsstemming fromdifferent use cases addressing security need to be distinguished.To contrastthatthesecurityrequirementsbetweenapplicationscanvarydrastically,thefollowingexampleisprovided.AlthoughmanyoftheinterfacesandtheirobjectstosupportBigDataapplicationsinphysicsaresimilartothoseinhealthcare,theydifferintheintegrationofsecurityinterfacesand policies. While in physics the protection of data is less of an issue, it is a stringentrequirement in healthcare.Thus, deriving architectural frameworks for bothmayuse largelysimilar components, but addressing security issues will be very different. The security ofinterfaces may be addressed in other documents. In this document, they are considered anadvanced use case showcasing that the validity of the specifications introduced here ispreserved,evenifsecurityandprivacyrequirementsdiffervastlyamongapplicationusecases.

2.2COMPONENT-SPECIFICINTERFACEREQUIREMENTS

Page 22: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

ThissectionsummarizestherequirementsfortheinterfacesoftheNBDRAcomponents.Thefive components are listed in Figure2 and addressed in Section2.2.1 (System OrchestratorInterface Requirements) and Section 2.2.4 (Big Data Application Provider to Big DataFrameworkProviderInterface)ofthisdocument.ThefivemainfunctionalcomponentsoftheNBDRArepresentthedifferenttechnicalroleswithinaBigDatasystemandarethefollowing:

SystemOrchestrator:Definesandintegratestherequireddataapplicationactivitiesintoanoperationalverticalsystem(seeSection2.2.1);DataProvider: Introducesnewdataor informationfeeds into theBigDatasystem(seeSection2.2.2);DataConsumer:IncludesendusersorothersystemsthatusetheresultsoftheBigDataApplicationProvider(seeSection2.2.3);Big Data Application Provider: Executes a data life cycle to meet security andprivacy requirements as well as System Orchestrator-defined requirements (seeSection2.2.4);Big Data Framework Provider: Establishes a computing framework in which toexecute certain transformation applications while protecting the privacy andintegrityofdata(seeSection2.2.5);andBig Data Application Provider to Framework Provider Interface: Defines aninterfacebetweentheapplicationspecificationandtheprovider(seeSection2.2.6).

2.2.1SystemOrchestratorInterfaceRequirements

TheSystemOrchestrator role includesdefining and integrating the requireddata applicationactivities into an operational vertical system. Typically, the SystemOrchestrator involves acollection of more specific roles, performed by one or more actors, which manage andorchestrate the operation of theBigData system. These actorsmay be human components,software components, or some combination of the two. The function of the SystemOrchestratoristoconfigureandmanagetheothercomponentsoftheBigDataarchitecturetoimplementoneormoreworkloadsthatthearchitectureisdesignedtoexecute.TheworkloadsmanagedbytheSystemOrchestratormaybeassigning/provisioningframeworkcomponentstoindividualphysicalorvirtualnodesatthelowerlevel,orprovidingagraphicaluserinterfacethat supports the specification of workflows linking together multiple applications andcomponentsat thehigherlevel.TheSystemOrchestratormayalso, throughtheManagementFabric, monitor the workloads and system to confirm that specific quality of servicerequirements ismet for eachworkload, andmay elastically assign and provision additionalphysicalorvirtualresourcestomeetworkloadrequirementsresultingfromchanges/surgesinthe data or number of users/transactions. The interface to the SystemOrchestratormust becapable of specifying the task of orchestration the deployment, configuration, and theexecution of applications within the NBDRA. A simple vendor-neutral specification tocoordinate the various parts either as simple parallel language tasks or as a workflowspecification isneeded to facilitate theoverallcoordination. Integrationofexisting toolsandservicesintotheSystemOrchestratorasextensibleinterfacesisdesirable.

Page 23: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

2.2.2DataProviderInterfaceRequirements

TheDataProviderroleintroducesnewdataorinformationfeedsintotheBigDatasystemfordiscovery,access,andtransformationbytheBigDatasystem.Newdatafeedsaredistinctfromthedataalreadyinusebythesystemandresidinginthevarioussystemrepositories.Similartechnologiescanbeusedtoaccessbothnewdatafeedsandexistingdata.TheDataProvideractorscanbeanythingfromasensor,toahumaninputtingdatamanually,toanotherBigDatasystem.Interfacesfordataprovidersmustbeabletospecifyadataprovidersoitcanbelocatedbyadataconsumer.Italsomustincludeenoughdetailstoidentifytheservicesofferedsotheycan be pragmatically reused by consumers. Interfaces to describe pipes and filters must beaddressed.

2.2.3DataConsumerInterfaceRequirements

Like theDataProvider, theroleofDataConsumerwithin theNBDRAcanbeanactualenduseroranothersystem.Inmanyways,thisroleisthemirrorimageoftheDataProvider,withthe entire Big Data framework appearing like a Data Provider to the Data Consumer. TheactivitiesassociatedwiththeDataConsumerroleincludethefollowing:

SearchandRetrieve,Download,AnalyzeLocally,Reporting,Visualization,andDatatoUseforTheirOwnProcesses.

Theinterfaceforthedataconsumermustbeabletodescribetheconsumingservicesandhowtheyretrieveinformationorleveragedataconsumers.

2.2.4BigDataApplicationInterfaceProviderRequirements

TheBigDataApplicationProviderroleexecutesaspecificsetofoperationsalongthedatalifecycle to meet the requirements established by the System Orchestrator, as well as meetingsecurity and privacy requirements. The Big Data Application Provider is the architecturecomponent that encapsulates the business logic and functionality to be executed by thearchitecture.TheinterfacestodescribeBigDataapplicationsincludeinterfacesforthevarioussubcomponentsincludingcollections,preparation/curation,analytics,visualization,andaccess.Someoftheinterfacesusedinthesesubcomponentscanbereusedfromotherinterfaces,whichare introduced in other sections of this document. Where appropriate, application-specificinterfaceswillbeidentifiedandexamplesprovidedwithafocusonusecasesasidentifiedintheNBDIF:Volume3,UseCasesandGeneralRequirements.

2.2.4.1Collection

Page 24: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Ingeneral, thecollectionactivityof theBigDataApplicationProviderhandles the interfacewith theData Provider. Thismay be a general service, such as a file server or web serverconfiguredbytheSystemOrchestrator toacceptorperformspecificcollectionsofdata,or itmaybeanapplication-specificservicedesignedtopulldataorreceivepushesofdatafromtheData Provider. Since this activity is receiving data at a minimum, it must store/buffer thereceiveddatauntil it ispersistedthroughtheBigDataFrameworkProvider.Thispersistenceneed not be to physical media but may simply be to an in-memory queue or other serviceprovidedby theprocessingframeworksof theBigDataFrameworkProvider.Thecollectionactivity is likelywhere theextractionportionof theExtract,Transform,Load(ETL)/Extract,Load,Transform (ELT) cycle is performed.At the initial collection stage, sets of data (e.g.,datarecords)ofsimilarstructurearecollected(andcombined), resultinginuniformsecurity,policy, and other considerations. Initial metadata is created (e.g., subjects with keys areidentified)tofacilitatesubsequentaggregationorlook-upmethods.

2.2.4.2Preparation

The preparation activity iswhere the transformation portion of theETL/ELT cycle is likelyperformed, although analytics activity will also likely perform advanced parts of thetransformation. Tasks performed by this activity could include data validation (e.g.,checksums/hashes, format checks), cleaning (e.g., eliminating bad records/fields), outlierremoval,standardization,reformatting,orencapsulating.Thisactivityisalsowheresourcedatawill frequently be persisted to archive storage in the Big Data Framework Provider andprovenancedatawillbeverifiedorattached/associated.Verificationorattachmentmayincludeoptimizationofdatathroughmanipulations(e.g.,deduplication)andindexingtooptimizetheanalytics process. This activity may also aggregate data from different Data Providers,leveragingmetadatakeystocreateanexpandedandenhanceddataset.

2.2.4.3Analytics

TheanalyticsactivityoftheBigDataApplicationProviderincludestheencodingofthelow-level business logic of theBigData system (with higher-level business process logic beingencoded by the System Orchestrator). The activity implements the techniques to extractknowledge from the data based on the requirements of the vertical application. Therequirementsspecifythedataprocessingalgorithmstoproducenewinsightsthatwilladdressthetechnicalgoal.Theanalyticsactivitywillleveragetheprocessingframeworkstoimplementtheassociatedlogic.Thistypicallyinvolvestheactivityprovidingsoftwarethatimplementstheanalytic logic to the batch and/or streaming elements of the processing framework forexecution. The messaging/communication framework of the Big Data Framework Providermaybeusedtopassdataorcontrolfunctionstotheapplicationlogicrunningintheprocessingframeworks.Theanalyticlogicmaybebrokenupintomultiplemodulestobeexecutedbytheprocessing frameworks which communicate, through the messaging/communicationframework, with each other and other functions instantiated by the Big Data ApplicationProvider.

Page 25: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

2.2.4.4Visualization

The visualization activity of the Big Data Application Provider prepares elements of theprocesseddataand theoutputof theanalyticactivity forpresentation to theDataConsumer.The objective of this activity is to format and present data in such a way as to optimallycommunicatemeaningandknowledge.Thevisualizationpreparationmayinvolveproducingatext-basedreportorrenderingtheanalyticresultsassomeformofgraphic.Theresultingoutputmay be a static visualization and may simply be stored through the Big Data FrameworkProvider for later access. However, the visualization activity frequently interacts with theaccessactivity, theanalyticsactivity,and theBigDataFrameworkProvider (processingandplatform) to provide interactive visualization of the data to the Data Consumer based onparameters provided to the access activity by theDataConsumer.The visualization activitymay be completely application-implemented, leverage one or more application libraries, ormay use specialized visualization processing frameworks within the Big Data FrameworkProvider.

2.2.4.5Access

The access activity within the Big Data Application Provider is focused on thecommunication/interaction with the Data Consumer. Like the collection activity, the accessactivitymaybeagenericservicesuchasawebserverorapplicationserverthatisconfiguredbytheSystemOrchestratortohandlespecificrequestsfromtheDataConsumer.Thisactivitywould interfacewith thevisualizationandanalyticactivities to respond to requests from theDataConsumer (whomaybeaperson)anduses theprocessingandplatformframeworks toretrievedata to respond toDataConsumer requests. Inaddition, theaccessactivityconfirmsthat descriptive and administrative metadata and metadata schemes are captured andmaintainedforaccessbytheDataConsumerandasdataistransferredtotheDataConsumer.TheinterfacewiththeDataConsumermaybesynchronousorasynchronousinnatureandmayuseapullorpushparadigmfordatatransfer.

2.2.5BigDataProviderFrameworkInterfaceRequirements

DataforBigDataapplicationsaredeliveredthroughdataproviders.Theycanbeeitherlocalproviders,datacontributedbyauser,ordistributeddataproviders,dataontheInternet.Thisinterfacemustbeabletoprovidethefollowingfunctionality:

Interfacestofiles,Interfacestovirtualdatadirectories,Interfacestodatastreams,andInterfacestodatafilters.

2.2.5.1InfrastructuresInterfaceRequirements

ThisBigDataFrameworkProviderelementprovidesalltheresourcesnecessarytohost/runthe

Page 26: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

activitiesoftheothercomponentsoftheBigDatasystem.Typically,theseresourcesconsistofsomecombinationofphysicalresources,whichmayhost/supportsimilarvirtualresources.TheNBDRAneedsinterfacesthatcanbeusedtodealwiththeunderlyinginfrastructuretoaddressnetworking,computing,andstorage.

2.2.5.2PlatformsInterfaceRequirements

Aspartof theNBDRAplatforms, interfacesareneeded thatcanaddressplatformneedsandservicesfordataorganization,datadistribution,indexedstorage,andfilesystems.

2.2.5.3ProcessingInterfaceRequirements

The processing frameworks for Big Data provide the necessary infrastructure software tosupport implementation of applications that candealwith the volume, velocity, variety, andvariabilityofdata.Processingframeworksdefinehowthecomputationandprocessingofthedataisorganized.BigDataapplicationsrelyonvariousplatformsandtechnologiestomeetthechallenges of scalable data analytics and operation.A requirement is the ability to interfaceeasily with computing services that offer specific analytics services, batch processingcapabilities,interactiveanalysis,anddatastreaming.

2.2.5.4CrosscuttingInterfaceRequirements

Severalcrosscutting interface requirementswithin theBigDataFrameworkProvider includemessaging, communication, and resourcemanagement. Often these servicesmay be hiddenfrom explicit interface use as they are part of larger systems that expose higher-levelfunctionality through their interfaces. However, such interfaces may also be exposed on alower level in case finer-grained control is needed.The need for such crosscutting interfacerequirements will be extracted from the NBDIF: Volume 3, Use Cases and GeneralRequirementsdocument.

2.2.5.5Messaging/CommunicationsFrameworks

Messaging and communications frameworks have their roots in the High PerformanceComputing environments long popular in the scientific and research communities.Messaging/CommunicationsFrameworksweredevelopedtoprovideapplicationprogramminginterfaces(APIs)forthereliablequeuing,transmission,andreceiptofdata.

2.2.5.6ResourceManagementFramework

As Big Data systems have evolved and becomemore complex, and as businesses work toleveragelimitedcomputationandstorageresourcestoaddressabroaderrangeofapplicationsand business challenges, the requirement to effectively manage those resources has grownsignificantly.Whiletoolsforresourcemanagementandelasticcomputinghaveexpandedand

Page 27: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

maturedinresponsetotheneedsofcloudprovidersandvirtualizationtechnologies,BigDataintroduces unique requirements for these tools. However, BigData frameworks tend to fallmoreintoadistributedcomputingparadigm,whichpresentsadditionalchallenges.

2.2.6BigDataApplicationProvidertoBigDataFrameworkProviderInterface

TheBigDataFrameworkProvidertypicallyconsistsofoneormorehierarchicallyorganizedinstancesofthecomponentsintheNBDRAITvaluechain(Figure2).Thereisnorequirementthatallinstancesatagivenlevelinthehierarchybeofthesametechnology.Infact,mostBigData implementations are hybrids that combine multiple technology approaches to provideflexibility ormeet the complete range of requirements,which are driven from theBigDataApplicationProvider.

Page 28: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

3SPECIFICATIONPARADIGM

Thissectionsummarizestheelementaryspecificationparadigm.

3.1HYBRIDANDMULTIPLEFRAMEWORKS

To avoid vendor lock-in, Big Data systems must be able to deal with hybrid and multipleframeworks.ThisisnotonlytrueforClouds,containers,DevOps,butalsoforcomponentsoftheNBDRA.

3.2DESIGNBYRESOURCE-ORIENTEDARCHITECTURE

Aresource-orientedarchitecturerepresentsasoftwarearchitectureandprogrammingparadigmfor designing and developing software in the form of resources. It is often associated withREpresentationalStateTransfer(REST) interfaces.The resourcesare softwarecomponentswhich can be reused in concrete reference implementations. The service specification isconductedwithOpenAPI,allowingusetoprovideitinaverygeneralformthatisindependentof the framework or computer language in which the services can be specified. Note thatOpenAPIdefinesservicesinRESTThepreviousversiononlyspecifiedtheresourceobjects.

3.3DESIGNBYEXAMPLE

To accelerate discussion among theNBD-PWGmembers, contributors to this document areencouragedtoalsoprovidetheNBD-PWGwithexamples.

3.4VERSIONMANAGEMENT

PreviousworkthatshapedthecurrentversionofthisvolumesandaredocumentedInGitHub[15]withpriorversionsofVolume8[16][17]andCloudmesh[18]insupportoftheNISTBigDataArchitectureFramework[1].

During the design phase and development period of each version of this document,enhancements are managed through GitHub and community contributions are managed viaGitHubissues.Thisallowspreservationofthehistoryofthisdocument.Whenanewversionisready,theversionwillbetaggedinGitHub.Olderversionswill,throughthisprocess,alsobeavailableashistoricaldocuments.DiscussionsaboutobjectsinwrittenformarecommunicatedasGitHubissues.

3.5INTERFACECOMPLIANCY

Due to the easy extensibility of the resource objects specified in this document and their

Page 29: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

interfaces, it is important to introduce a terminology that allows the definition of interfacecompliancy.Wedefinethreelevelsofinterfacecomplianceasfollows:

FullCompliance:Thesearereferenceimplementationsthatprovidefullcompliancetotheobjectsdefinedinthisdocument.Aversionnumberisaddedtoassurethatthesnapshot in time of the objects is associated with the version. A full complientframeworkimplementsallobjects.

Partial Compliance: These are reference implementations that provide partialcompliancetotheobjectsdefinedinthisdocument.Aversionnumberwillisaddedtoassurethatthesnapshotintimeoftheobjectsisassociatedwiththeversion.Thisreference implementation implementsapartial listof theobjectsand interfaces.Adocument is to be added that specifies the differences to a full complientimplementation.

Extended Compliance: In addition to full and partial compliance additionalresources can be identified while documenting additional resource objects andinterfaces that are not included in the current specification. The extendedcomplience document can lead to additional improvements of the currentspecification.

3.6REFERNCEIMPLEMENTATIONS

Documents generated during a reference implementation can be forwarded to theReferenceArchitecture Subgroup for further discussion and for possible futuremodifications based onadditionalpracticaluserfeedback.

Page 30: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4SPECIFICATION

The specifications in this section are provided through an automated document creationprocess using the actual OpenAPI specifications yaml files as the source. All OpenAPIspecificationslocatedinthecloudmesh/cloudmesh-nist/spec/directoryinGitHub[19].

Limitationsofthecurrentimplementationareasfollows.ItisademonstrationthatshowcasesthegenerationofafullyfunctioningRESTservicebasedonthespecificationsprovidedinthisdocument.However,itisexpectedthatscalability,distributionofservices,andotheradvancedoptionsneedtobeaddressedbasedonapplicationrequirements.

4.1LISTOFSPECIFICATIONS

The following table lists the current set of resource objects that are defined in this draft.AdditionalobjectsarealsoavailableinGitHub[19].

Table1showsthelistofcurrentlyincludedspecificationinthisversionofthedocument.

Table1:SpecificationsService Version Date Reference Section

Alias 3.2.0 17-06-2019 ☁� Section4.6.1Account 3.2.0 17-06-2019 ☁� Section4.5.3Containers 3.2.0 17-06-2019 ☁� Section4.10.1Database 3.2.0 17-06-2019 ☁� Section4.7.3Default 3.2.0 17-06-2019 ☁� Section4.6.3Deployment 3.2.0 17-06-2019 ☁� Section4.15.1File 3.2.0 17-06-2019 ☁� Section4.7.1Filter 3.2.0 17-06-2019 ☁� Section4.14.2Flavor 3.2.0 17-06-2019 ☁� Section4.9.2Image 3.2.0 17-06-2019 ☁� Section4.9.1MapReduce 3.2.0 17-06-2019 ☁� Section4.11.1Microservice 3.2.0 17-06-2019 ☁� Section4.12.1Nic 3.2.0 17-06-2019 ☁� Section4.9.5NetworkofNodes 3.2.0 17-06-2019 ☁� Section4.8.2Organization 3.2.0 17-06-2019 ☁� Section4.5.1PublicKeyStore 3.2.0 17-06-2019 ☁� Section4.5.4Scheduler 3.2.0 17-06-2019 ☁� Section4.8.4

Page 31: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Replica 3.2.0 17-06-2019 ☁� Section4.7.2Reservation 3.2.0 17-06-2019 ☁� Section4.13.1Queue 3.2.0 17-06-2019 ☁� Section4.8.3Secgroup 3.2.0 17-06-2019 ☁� Section4.9.4Stream 3.2.0 17-06-2019 ☁� Section4.14.1Timestamp 3.2.0 17-06-2019 ☁� Section4.4.1User 3.2.0 17-06-2019 ☁� Section4.5.2Variables 3.2.0 17-06-2019 ☁� Section4.6.2VirtualCluster 3.2.0 17-06-2019 ☁� Section4.8.1VirtualDirectory 3.2.0 17-06-2019 ☁� Section4.7.4VirtualMachine 3.2.0 17-06-2019 ☁� Section4.9.3

Figure3showstheproviderviewofthespecificationresources.

Figure3:Providerview

Figure4showstheresourcesviewofthespecificationresources.

Page 32: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Figure4:Resourceview

4.2AUTHENTICATION

Mechanisms are not included in this specification to manage authentication to externalservices. However, the working group has shown multiple solutions to this as part ofcloudmesh.Thisincludestheposibilityofa

Localconfigurationfile:Aconfigurationfileismanagedlocallytoallowaccesstotheclouds.Itisthedesigner’sresponsibilitynottoexposesuchcredentials.Sessionbasedauthentication:Nopasswordsarestoredintheconfigurationfileandaccessisgrantedonapersessionbasiswherethepasswordneedstobeentered.Service based authentication: The authentication is delegated to an externalprocess. The service that acts on behalf of the user needs to have access to theappropriatecloudprovidercredentials

Anexampleforaconfigurationfileisprovidedat[20].

4.3STATUSCODESANDERRORRESPONSES

Incaseofanerrororasuccessful response, the responseheadercontainsaHTTPcode(seehttps://tools.ietf.org/html/rfc7231).Theresponsebodyusuallycontainsthefollowing:

TheHTTPresponsecode;

AnaccompanyingmessagefortheHTTPresponsecode;and

Afieldorobjectwheretheerroroccurred.

Page 33: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Table1:HTTPResponseCodes

HTTPResponse OperationD escription

200Ok GET,PUT,DELETE Noerror,operationsuccessful.201Created POST Successfulcreationofaresource.204NoContent GET,PUT,DELETE Successfulbutnocontent.

400BadRequest

GET,POST,PUT,DELETE Therequestcouldnotbeunderstood.

401Unauthorized

GET,POST,PUT,DELETE Usermustauthorize.

403Forbidden

GET,POST,PUT,DELETE

Therequesthasbeenrefusedduetoauthorizationfailure.

404NotFound

GET,POST,PUT,DELETE Therequestedresourcecouldnotbefound.

405NotAllowed

GET,POST,PUT,DELETE Themethodisnotallowedontheresource.

500ServerError GET,POSTPUT InternalServererror.

InthespecificationsuchresponsesareindicatedandifansimpleresponseisreturnedthetermMessageisused.

Resources

4.4TIMESTAMP

Timestampscanbeusedinconjunctionwithandyserversideimplementationoftheinterfaces.It can be useful to return information about when a particular resource has been created,updated, or accessed. To simplify the specification in the document we have not explicitlylistedthatatimestampispartofthereource,butwecanassumeitmaybeaddedaspartoftheserviceimplementation.Toobtainanexampletimestampasimplegetfunctionisprovided.

4.4.1Timestamp

Data often needs to be time stamped to indicate when it has been accessed, created, ormodified.Allobjectsdefined in thisdocumentwillhave, in their finalversion,a timestamp.Thedate-timestringisdefinedinRFC3339.

Page 34: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.4.1.1SchemaTimestamp

Reference:☁�

Property Type Description

accessed string(date-time) Thetimestampwhentheobjectwaslastaccessed

created string(date-time) Thetimestampwhentheobjectwascreated

modified string(date-time) Thetimestampwhentheobjectwasmodified

4.4.1.2Paths

HTTP Path Summaryget /timestamp Returnsthetimestamp

4.4.1.2.1/timestamp

4.4.1.2.1.1GET/timestamp

Returnsthetimestamp

Responses

Code Description Schema200 Thecurrenttimeanddate string

4.4.1.3timestamp.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Timestamp

description:|-

Dataoftenneedstobetimestampedtoindicatewhenithasbeen

accessed,created,ormodified.Allobjectsdefinedinthis

documentwillhave,intheirfinalversion,atimestamp.

Thedate-timestringisdefinedin

[RFC3339](https://xml2rfc.ietf.org/public/rfc/html/rfc3339.html#anchor14).

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

Page 35: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

-url:/cloudmesh/v3

paths:

/timestamp:

get:

summary:Returnsthetimestamp

description:Returnsthetimestamp

responses:

'200':

description:Thecurrenttimeanddate

content:

application/json:

schema:

type:string

example:1985-04-12T23:20:50.52Z

components:

schemas:

Timestamp:

type:object

description:thetimestamp

properties:

accessed:

type:string

format:date-time

description:Thetimestampwhentheobjectwaslastaccessed

example:1985-04-12T23:20:50.52Z

created:

type:string

format:date-time

description:Thetimestampwhentheobjectwascreated

example:1985-04-12T23:20:50.52Z

modified:

type:string

format:date-time

description:Thetimestampwhentheobjectwasmodified

example:1985-04-12T23:20:50.52Z

4.5IDENTITY

Aspartofservicesanidentityoftenneedstobespecified.Inaddition,suchpersons[21] areoften part of groups.Thus, three important terms related to the identity are distinguished asfollows:

Organization: The information representing an Organization that manages a BigDataService(Section4.5.1)Group:Agroup that apersonmaybelong to that is important todefine access toservices(includedinSection4.5.1)User:Theinformationidentifyingtheprofileofaperson(Section4.5.2)

4.5.1Organization

An important concept in many services is the management of a group of users in anorganization.Withinanorganizationwedistinguishdifferentgroupsofusers.Groupscanbeusedtocharachterizerolesuserscanfulfill.Userscanbelongtomultiplegroups.Suchgroupscanalsobeusedtospecifyaccessrightstoservices.

4.5.1.1SchemaOrganization

Reference:☁�

Property Type Descriptionname string Nameoftheorganization

Page 36: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

users array[User] Listofusers

4.5.1.2SchemaGroup

Reference:☁�

Property Type Descriptionname string Thenameofthegroupdescription string Thedescriptionofthegroupusers array[string] Theusernamesthataremembersofthegroup

4.5.1.3Paths

HTTP Path Summary

get /organization Returnsalistoforganizations

get /organization/{name} Returnsthenamedorganization

put /organization/{name} Uploadsanorganizationtothelistoforganizations

delete /organization/{name} Deletesthenamedorganization

get /organization/{name}/user Returnsallusersoftheorganization

get /organization/{name}/user/{user} Returnsthespecificuserofthatorganization

put /organization/{name}/user/{user} Updatesoraddsauserintheorganization

delete /organization/{name}/user/{user} Deleteanuserintheorganization

get /organization/{name}/group/ Returnsallgroupnames

get /organization/{name}/group/{group} Returnsthespecificgroupofthatorganization

put /organization/{name}/group/{group} Updatesoraddsagroupintheorganization

delete /organization/{name}/group/{group} Deleteagroupintheorganization

Page 37: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

put /organization/{name}/group/{group}/{user} Updatesoraddsausernametothegroup

delete /organization/{name}/group/{group}/{user} Deleteauserinthegroup

4.5.1.3.1/organization

4.5.1.3.1.1GET/organization

Returnsalistofallorganizations

Responses

Code Description Schema200 Thelistoforganizations array[Organization]401 Notauthorized String

4.5.1.3.2/organization/{name}

4.5.1.3.2.1GET/organization/{name}

Returnsanorganizationbyname

Responses

Code Description Schema200 Retruningtheinformationoftheorganization Organization401 Notauthorized String404 Thenamedorganizationcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True String

4.5.1.3.2.2PUT/organization/{name}

Uploadsanorganizationtothelistoforganizations

Responses

Page 38: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Organizationcreatedorupdated String401 Notauthorized String404 Theorganizationcouldnotbefound String

RequestBody

Locatedin Description Required SchemaBody Theorganizationtobeuploaded True Organization

4.5.1.3.2.3DELETE/organization/{name}

Deletesanorganizationbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedorganizationcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True String

4.5.1.3.3/organization/{name}/user

4.5.1.3.3.1GET/organization/{name}/user

Returnsallusersoftheorganization

Responses

Code Description Schema200 Theorganization Organization401 Notauthorized String

Page 39: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True String

4.5.1.3.4/organization/{name}/user/{user}

4.5.1.3.4.1GET/organization/{name}/user/{user}

Returnsthespecificuserofthatorganization

Responses

Code Description Schema200 Theuser User401 Notauthorized String404 Theorganizationorusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True Stringuser path Theusername True String

4.5.1.3.4.2PUT/organization/{name}/user/{user}

Updatesoraddsauserintheorganization

Responses

Code Description Schema200 Useraddedsucessfully String401 Notauthorized String404 Theorganizationorusercouldnotbefound\ String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True String

Page 40: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

user path Theusername True String

RequestBody

Locatedin Description Required SchemaBody Theusertobeuploaded True User

4.5.1.3.4.3DELETE/organization/{name}/user/{user}

Deleteanuserintheorganization

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Theorganizationorusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True Stringuser path Theusername True String

4.5.1.3.5/organization/{name}/group/

4.5.1.3.5.1GET/organization/{name}/group/

Returnsallgroupnames

Responses

Code Description Schema

200 Returningtheinformationofthegroup array[String]400 Nogroupfound String401 Notauthorized String404 Theorganizationorgroupcouldnotbefound String

Page 41: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True String

4.5.1.3.6/organization/{name}/group/{group}

4.5.1.3.6.1GET/organization/{name}/group/{group}

Returnsthespecificgroupofthatorganization

Responses

Code Description Schema200 Thegroup Group401 Notauthorized String404 Theorganizationorgroupcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True Stringgroup path Thegroupname True String

4.5.1.3.6.2PUT/organization/{name}/group/{group}

Updatesoraddsagroupintheorganization

Responses

Code Description Schema200 Groupaddedsucessfully String401 Notauthorized String

404 Theorganizationorgroupcouldnotbefound String

Parameters

Name Locatedin Description Required Schema

Page 42: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

name path Thenameofthegroup True Stringgroup path Thegroupname True String

4.5.1.3.6.3DELETE/organization/{name}/group/{group}

Deleteagroupintheorganization

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Theorganizationorgroupcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True Stringgroup path Thegroupname True String

4.5.1.3.7/organization/{name}/group/{group}/{user}

4.5.1.3.7.1PUT/organization/{name}/group/{group}/{user}

Updatesoraddsausernametothegroup

Responses

Code Description Schema200 Useraddedsucessfully String401 Notauthorized String404 Theorganization,group,orusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthegroup True Stringgroup path Thegroupname True String

Page 43: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

user path Theusername True String

4.5.1.3.7.2DELETE/organization/{name}/group/{group}/{user}

Deleteauserinthegroup

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Theorganization,group,orusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheorganization True Stringgroup path Thegroupname True Stringuser path Theusername True String

4.5.1.4organization.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Organization

description:|-

Animportantconceptinmanyservicesisthemanagementofagroup

ofusersinanorganization.Withinanorganizationwedistinguish

differentgroupsofusers.Groupscanbeusedtocharachterizeroles

userscanfulfill.Userscanbelongtomultiplegroups.Suchgroupscan

alsobeusedtospecifyaccessrightstoservices.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/organization:

get:

tags:

-Organization

summary:Returnsalistoforganizations

description:Returnsalistofallorganizations

operationId:cloudmesh.organization.list

responses:

'200':

description:Thelistoforganizations

content:

application/json:

schema:

type:array

Page 44: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

items:

$ref:'#/components/schemas/Organization'

'401':

description:Notauthorized

/organization/{name}:

get:

tags:

-Organization

summary:Returnsthenamedorganization

description:Returnsanorganizationbyname

operationId:cloudmesh.organization.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

responses:

'200':

description:Retruningtheinformationoftheorganization

content:

application/json:

schema:

$ref:'#/components/schemas/Organization'

'401':

description:Notauthorized

'404':

description:Thenamedorganizationcouldnotbefound

put:

tags:

-Organization

summary:Uploadsanorganizationtothelistoforganizations

description:Uploadsanorganizationtothelistoforganizations

operationId:cloudmesh.organization.add

requestBody:

description:Theorganizationtobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Organization'

responses:

'200':

description:Organizationcreatedorupdated

'401':

description:Notauthorized

'404':

description:Theorganizationcouldnotbefound

delete:

tags:

-Organization

summary:Deletesthenamedorganization

description:Deletesanorganizationbyname

operationId:cloudmesh.organization.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedorganizationcouldnotbefound

/organization/{name}/user:

get:

tags:

-Organization

summary:Returnsallusersoftheorganization

description:Returnsallusersoftheorganization

operationId:cloudmesh.organization.user.list

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

responses:

'200':

description:Theorganization

content:

application/json:

schema:

Page 45: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

$ref:"#/components/schemas/Organization"

'401':

description:Notauthorized

/organization/{name}/user/{user}:

get:

tags:

-Organization

summary:Returnsthespecificuserofthatorganization

description:Returnsthespecificuserofthatorganization

operationId:cloudmesh.organization.user.get_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:user

description:Theusername

in:path

required:true

schema:

type:string

responses:

'200':

description:Theuser

content:

application/json:

schema:

$ref:"user.yaml#/components/schemas/User"

'401':

description:Notauthorized

'404':

description:Theorganizationorusercouldnotbefound

put:

tags:

-Organization

summary:Updatesoraddsauserintheorganization

description:Updatesoraddsauserintheorganization

operationId:cloudmesh.organization.user.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:user

description:Theusername

in:path

required:true

schema:

type:string

requestBody:

description:Theusertobeuploaded

required:true

content:

application/json:

schema:

$ref:'user.yaml#/components/schemas/User'

responses:

'200':

description:Useraddedsucessfully

'401':

description:Notauthorized

'404':

description:Theorganizationorusercouldnotbefound\

delete:

tags:

-Organization

summary:Deleteanuserintheorganization

description:Deleteanuserintheorganization

operationId:cloudmesh.organization.user.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:user

description:Theusername

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

Page 46: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

'401':

description:Notauthorized

'404':

description:Theorganizationorusercouldnotbefound

/organization/{name}/group/:

get:

tags:

-Organization

summary:Returnsallgroupnames

description:Returnsallgroupnames

operationId:cloudmesh.organization.group.list

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

responses:

'200':

description:Returningtheinformationofthegroup

content:

application/json:

schema:

type:array

items:

type:string

'400':

description:Nogroupfound

'401':

description:Notauthorized

'404':

description:Theorganizationorgroupcouldnotbefound

/organization/{name}/group/{group}:

get:

tags:

-Organization

summary:Returnsthespecificgroupofthatorganization

description:Returnsthespecificgroupofthatorganization

operationId:cloudmesh.organization.group.get_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:group

description:Thegroupname

in:path

required:true

schema:

type:string

responses:

'200':

description:Thegroup

content:

application/json:

schema:

$ref:"#/components/schemas/Group"

'401':

description:Notauthorized

'404':

description:Theorganizationorgroupcouldnotbefound

put:

tags:

-Organization

summary:Updatesoraddsagroupintheorganization

description:Updatesoraddsagroupintheorganization

operationId:cloudmesh.organization.group.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthegroup

-name:group

description:Thegroupname

in:path

required:true

schema:

type:string

responses:

'200':

description:Groupaddedsucessfully

'401':

description:Notauthorized

'404':

Page 47: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Theorganizationorgroupcouldnotbefound

delete:

tags:

-Organization

summary:Deleteagroupintheorganization

description:Deleteagroupintheorganization

operationId:cloudmesh.organization.greop.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:group

description:Thegroupname

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Theorganizationorgroupcouldnotbefound

/organization/{name}/group/{group}/{user}:

put:

tags:

-Organization

summary:Updatesoraddsausernametothegroup

description:Updatesoraddsausernametothegroup

operationId:cloudmesh.organization.group.user.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthegroup

-name:group

description:Thegroupname

in:path

required:true

schema:

type:string

-name:user

description:Theusername

in:path

required:true

schema:

type:string

responses:

'200':

description:Useraddedsucessfully

'401':

description:Notauthorized

'404':

description:Theorganization,group,orusercouldnotbefound

delete:

tags:

-Organization

summary:Deleteauserinthegroup

description:Deleteauserinthegroup

operationId:cloudmesh.organization.greop.delete.user

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheorganization

-name:group

description:Thegroupname

in:path

required:true

schema:

type:string

-name:user

description:Theusername

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

Page 48: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Notauthorized

'404':

description:Theorganization,group,orusercouldnotbefound

components:

schemas:

Organization:

type:object

properties:

name:

description:Nameoftheorganization

type:string

users:

description:Listofusers

type:array

items:

$ref:"user.yaml#/components/schemas/User"

Group:

type:object

description:Thegroups

properties:

name:

type:string

description:Thenameofthegroup

description:

type:string

description:Thedescriptionofthegroup

users:

description:Theusernamesthataremembersofthegroup

type:array

items:

type:string

4.5.2User

Servicesneedtospecifywhichusershaveaccesstothem.Userinformationcanbereusedinotherservicesandorganizedinavirtualorganization.Ausercanbeaddedtoanamedlistofuserswithinthisorganization.Agroupassociatedwiththeusercanbeusedtoaugmentuserstobepartofoneormoregroups.

4.5.2.1SchemaUser

Reference:☁�

Property Type Descriptionusername string Theuniqueusernameassociatedwiththeuserfirstname string Thefirstnameoftheuserlastname string Thelastnameoftheuseremail string Theemailoftheusercomment string Acommentregardingtheuserpublickey string Thepublickeyoftheuser

4.5.2.2Paths

HTTP Path Summaryget /user Returnsalistofusers

Page 49: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

get /user/{name} Returnsthenameduserput /user/{name} Uploadsausertothelistofusersdelete /user/{name} Deletesthenameduser

4.5.2.2.1/user

4.5.2.2.1.1GET/user

Returnsalistofallusers

Responses

Code Description Schema200 Thelistofusers array[User]401 Notauthorized String

4.5.2.2.2/user/{name}

4.5.2.2.2.1GET/user/{name}

Returnsanuserbyname

Responses

Code Description Schema200 Returningtheinformationoftheuser User401 Notauthorized String404 Thenamedusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheuser True String

4.5.2.2.2.2PUT/user/{name}

Uploadsausertothelistofusers

Responses

Page 50: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Userupdated String401 Notauthorized String404 Thenamedusercouldnotbefound String

RequestBody

Locatedin Description Required SchemaBody Theusertobeuploaded True User

4.5.2.2.2.3DELETE/user/{name}

Deletesanuserbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedusercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheuser True String

4.5.2.3user.yaml

openapi:"3.0.2"

info:

version:"3.2.0"

x-date:17-06-2019

x-status:defined

title:User

description:|-

Servicesneedtospecifywhichusershaveaccesstothem.User

informationcanbereusedinotherservicesandorganizedinavirtual

organization.Ausercanbeaddedtoanamedlistofuserswithinthis

organization.Agroupassociatedwiththeusercanbeusedtoaugment

userstobepartofoneormoregroups.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:CloudmeshUser

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

Page 51: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

paths:

/user:

get:

tags:

-User

summary:Returnsalistofusers

description:Returnsalistofallusers

operationId:cloudmesh.user.list

responses:

'200':

description:Thelistofusers

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/User'

'401':

description:Notauthorized

/user/{name}:

get:

tags:

-User

summary:Returnsthenameduser

description:Returnsanuserbyname

operationId:cloudmesh.user.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheuser

responses:

'200':

description:Returningtheinformationoftheuser

content:

application/json:

schema:

$ref:'#/components/schemas/User'

'401':

description:Notauthorized

'404':

description:Thenamedusercouldnotbefound

put:

tags:

-User

summary:Uploadsausertothelistofusers

description:Uploadsausertothelistofusers

operationId:cloudmesh.user.add

requestBody:

description:Theusertobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/User'

responses:

'200':

description:Userupdated

'401':

description:Notauthorized

'404':

description:Thenamedusercouldnotbefound

delete:

tags:

-User

summary:Deletesthenameduser

description:Deletesanuserbyname

operationId:cloudmesh.user.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheuser

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedusercouldnotbefound

components:

schemas:

User:

type:object

properties:

Page 52: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

username:

type:string

description:Theuniqueusernameassociatedwiththeuser

firstname:

type:string

description:Thefirstnameoftheuser

lastname:

type:string

description:Thelastnameoftheuser

email:

type:string

description:Theemailoftheuser

comment:

type:string

description:Acommentregardingtheuser

publickey:

type:string

description:Thepublickeyoftheuser

4.5.3Account

Tochargetheuseofresourcesaccountingcanbeused.Accountingcanbeimplementedonavariety of resources, such as users, groups or organizations. It is up to the implementer toproviderulesandcostforit.Ifneededvvmultipleaccountingresourcescanbeimplemented.

4.5.3.1SchemaAccount

Reference:☁�

Property Type Descriptionname string nameofaccountdescription string thepurposeoftheaccountcharge integer Thecurrentchareoftheaccount

unit string theunitinwhichtheaccountischargedandthechargevalueisstored

4.5.3.2Paths

HTTP Path Summaryget /account Returnsatheaccountsget /account/{name} Returnsthenamedaccountput /account/{name} Setthevalueofaaccountdelete /account/{name} Deletesthenamedaccount

4.5.3.2.1/account

4.5.3.2.1.1GET/account

Returnstheaccounts

Page 53: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Thelistofaccounts array[Account]401 Notauthorized String

4.5.3.2.2/account/{name}

4.5.3.2.2.1GET/account/{name}

Returnsthenamedaccount

Responses

Code Description Schema200 Returningtheinformationoftheaccount Account401 Notauthorized String404 Thenamedaccountcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheaccount True String

4.5.3.2.2.2PUT/account/{name}

Setthevalueofthenamedaccount

Responses

Code Description Schema200 Accountupdatedorcreated String400 Errorupdatingaccount String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Theaccountanditsvalue True Account

Page 54: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.5.3.2.2.3DELETE/account/{name}

Deletesaaccountbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedaccountcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameoftheaccount True String

4.5.3.3account.yaml

```{include=./spec/account.yaml}

4.5.4PublicKeyStore

Many services and frameworks use Secure Shell (SSH) keys to authenticate. This serviceallowstheconvenientstorageofthepublickeys.

4.5.4.1SchemaKey

Reference:☁�

Property Type Descriptionname string Thenameofthepublickeyvalue string Thevalueofthepublickeykind string Thekeykindsuchasrsa,dsa

group string Anoptionalgroupnameallowingtogroupkeystocreatecustomkeygroupswithinthepublickeystore

comment string Acommentforthepublickeyuri string Theuriofthepublickeyifany

Page 55: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

fingerprint string Thefingerprintofthepublickey

4.5.4.2Paths

HTTP Path Summaryget /key Returnsalistofkeysget /key/{name} Returnsthenamedkeyput /key/{name} Setakeydelete /key/{name} Deletesthenamedkey

4.5.4.2.1/key

4.5.4.2.1.1GET/key

Returnsalistofallkeys

Responses

Code Description Schema200 Thelistofkeys array[Key]401 Notauthorized String

4.5.4.2.2/key/{name}

4.5.4.2.2.1GET/key/{name}

Returnsakeybyname

Responses

Code Description Schema200 Returningtheinformationofthekey Key401 Notauthorized String404 Thenamedkeycouldnotbefound String

Parameters

Name Locatedin Description Required Schema

Page 56: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

name path Thenameofthekey True String

4.5.4.2.2.2PUT/key/{name}

Setsthenamedkey

Responses

Code Description Schema200 Keyupdated String401 Notauthorized String404 Thenamedkeycouldnotbefound String

RequestBody

Locatedin Description Required SchemaBody Thenewkeytocreate True Key

4.5.4.2.2.3DELETE/key/{name}

Deletesakeybyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedkeycouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthekey True String

4.5.4.3publickeystore.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:PublicKeyStore

Page 57: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:|-

ManyservicesandframeworksuseSecureShell(SSH)keysto

authenticate.Thisserviceallowstheconvenientstorageofthe

publickeys.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/key:

get:

tags:

-Key

summary:Returnsalistofkeys

description:Returnsalistofallkeys

operationId:cloudmesh.key.list

responses:

'200':

description:Thelistofkeys

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Key'

'401':

description:Notauthorized

/key/{name}:

get:

tags:

-Key

summary:Returnsthenamedkey

description:Returnsakeybyname

operationId:cloudmesh.key.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthekey

responses:

'200':

description:Returningtheinformationofthekey

content:

application/json:

schema:

$ref:'#/components/schemas/Key'

'401':

description:Notauthorized

'404':

description:Thenamedkeycouldnotbefound

put:

tags:

-Key

summary:Setakey

description:Setsthenamedkey

operationId:cloudmesh.key.add

requestBody:

description:Thenewkeytocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Key'

responses:

'200':

description:Keyupdated

'401':

description:Notauthorized

'404':

description:Thenamedkeycouldnotbefound

delete:

tags:

-Key

summary:Deletesthenamedkey

description:Deletesakeybyname

operationId:cloudmesh.key.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

Page 58: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

description:Thenameofthekey

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedkeycouldnotbefound

components:

schemas:

Key:

type:object

description:thepublickey

properties:

name:

type:string

description:Thenameofthepublickey

value:

type:string

description:Thevalueofthepublickey

kind:

type:string

description:Thekeykindsuchasrsa,dsa

group:

type:string

description:Anoptionalgroupnameallowingtogroupkeystocreate

customkeygroupswithinthepublickeystore

comment:

type:string

description:Acommentforthepublickey

uri:

type:string

description:Theuriofthepublickeyifany

fingerprint:

type:string

description:Thefingerprintofthepublickey

4.6VARIABLE,DEFUALT,ANDALIAS

4.6.1Alias

Oftenauserhas thedesire tocreateacustomnameforanobject.Analiasallowstodothatwhilewhile assosication auser defined name oralias to a previouly used name.The aliasescouldbesharedwithotherusers.Anamecouldhaveoneormorealiases.

4.6.1.1SchemaAlias

Reference:☁�

Property Type Descriptionname string Thenameofthealiassource string Theoriginaluniqueobjectname

4.6.1.2Paths

HTTP Path Summaryget /alias Returnsalistofaliases

get /alias/{name} Returnsthenamedalias

Page 59: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

put /alias/{name} Setanaliasdelete /alias/{name} Deletesthenamedalias

4.6.1.2.1/alias

4.6.1.2.1.1GET/alias

Returnsalistofallaliases

Responses

Code Description Schema200 Thelistofaliasses array[Alias]400 Noaliasfound String401 Notauthorized String

4.6.1.2.2/alias/{name}

4.6.1.2.2.1GET/alias/{name}

Returnsanaliasbyname

Responses

Code Description Schema200 Returningtheinformationofthealias Alias401 Notauthorized String404 Thenamedaliascouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthealias True String

4.6.1.2.2.2PUT/alias/{name}

Setsthenamedalias

Responses

Page 60: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Aliasupdatedorcreated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewaliastocreate True Alias

4.6.1.2.2.3DELETE/alias/{name}

Deletesanaliasbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedaliascouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthealias True String

4.6.1.3alias.yaml

openapi:'3.0.2'

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Alias

description:|-

Oftenauserhasthedesiretocreateacustomnameforanobject.An

aliasallowstodothatwhilewhileassosicationauserdefinednameor

*alias*toaprevioulyusedname.Thealiasescouldbesharedwithother

users.Anamecouldhaveoneormorealiases.

termsOfService:'https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt'

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/alias:

get:

Page 61: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

tags:

-Alias

summary:Returnsalistofaliases

description:Returnsalistofallaliases

operationId:cloudmesh.alias.list

responses:

'200':

description:Thelistofaliasses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Alias'

'400':

description:Noaliasfound

'401':

description:Notauthorized

/alias/{name}:

get:

tags:

-Alias

summary:Returnsthenamedalias

description:Returnsanaliasbyname

operationId:cloudmesh.alias.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthealias

responses:

'200':

description:Returningtheinformationofthealias

content:

application/json:

schema:

$ref:'#/components/schemas/Alias'

'401':

description:Notauthorized

'404':

description:Thenamedaliascouldnotbefound

put:

tags:

-Alias

summary:Setanalias

description:Setsthenamedalias

operationId:cloudmesh.alias.add

requestBody:

description:Thenewaliastocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Alias'

responses:

'200':

description:Aliasupdatedorcreated

'401':

description:Notauthorized

delete:

tags:

-Alias

summary:Deletesthenamedalias

description:Deletesanaliasbyname

operationId:cloudmesh.alias.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthealias

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedaliascouldnotbefound

components:

schemas:

Alias:

type:object

description:thealias

properties:

name:

type:string

Page 62: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Thenameofthealias

source:

type:string

description:Theoriginaluniqueobjectname

4.6.2Variables

Variablesareasimplestringkeyvaluestoragetostoresimplevalues.Eachvariablecanhaveadatatype,sothatitcanbeusedforserializationintootherformats.Internallytheyarestorredasstrings.

4.6.2.1SchemaVariable

Reference:☁�

Property Type Descriptionname string Nameofthevariablevalue string Valueofthevariabledescription string Adescriptionofthevariable

datatype string Thedatatypeofthevariablewhichcanbeusedforserialization

4.6.2.2Paths

HTTP Path Summaryget /variable Returnsathevariablesget /variable/{name} Returnsthenamedvariableput /variable/{name} Setthevalueofavariabledelete /variable/{name} Deletesthenamedvariable

4.6.2.2.1/variable

4.6.2.2.1.1GET/variable

Returnsthevariables

Responses

Code Description Schema200 Thelistofvariables array[Variable]400 Novariablefound String

Page 63: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.6.2.2.2/variable/{name}

4.6.2.2.2.1GET/variable/{name}

Returnsthenamedvariable

Responses

Code Description Schema200 Returningtheinformationofthevariable Variable401 Notauthorized String404 Thenamedvariablecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevariable True String

4.6.2.2.2.2PUT/variable/{name}

Setthevalueofthenamedvariable

Responses

Code Description Schema200 Variableupdatedorcreated String400 Errorupdatingvariable String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thevariableanditsvalue True Variable

4.6.2.2.2.3DELETE/variable/{name}

Deletesavariablebyname

Page 64: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedvariablecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevariable True String

4.6.2.3variables.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Variables

description:|-

Variablesareasimplestringkeyvaluestoragetostoresimple

values.Eachvariablecanhaveadatatype,sothatitcanbeusedfor

serializationintootherformats.Internallytheyarestorredasstrings.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/variable:

get:

tags:

-Variable

summary:Returnsathevariables

description:Returnsthevariables

operationId:cloudmesh.variable.list

responses:

'200':

description:Thelistofvariables

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Variable'

'400':

description:Novariablefound

/variable/{name}:

get:

tags:

-Variable

summary:Returnsthenamedvariable

description:Returnsthenamedvariable

operationId:cloudmesh.variable.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevariable

responses:

'200':

Page 65: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Returningtheinformationofthevariable

content:

application/json:

schema:

$ref:'#/components/schemas/Variable'

'401':

description:Notauthorized

'404':

description:Thenamedvariablecouldnotbefound

put:

tags:

-Variable

summary:Setthevalueofavariable

description:Setthevalueofthenamedvariable

operationId:cloudmesh.variable.add

requestBody:

description:Thevariableanditsvalue

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Variable'

responses:

'200':

description:Variableupdatedorcreated

'400':

description:Errorupdatingvariable

'401':

description:Notauthorized

delete:

tags:

-Variable

summary:Deletesthenamedvariable

description:Deletesavariablebyname

operationId:cloudmesh.variable.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevariable

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedvariablecouldnotbefound

components:

schemas:

Variable:

type:object

description:thevariables

properties:

name:

type:string

description:Nameofthevariable

value:

type:string

description:Valueofthevariable

description:

type:string

description:Adescriptionofthevariable

datatype:

type:string

description:Thedatatypeofthevariablewhichcanbeusedfor

serialization

4.6.3Default

Adefaultisaspecialvariablethathasacontextassociatedwithit.Thisallowsonetodefinevalues that can be easily retrieved based on the associated context. For example, a defaultcouldbetheimagenameforacloudwherethecontextisdefinedbythecloudname.

4.6.3.1SchemaDefault

Reference:☁�

Page 66: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Property Type Descriptionname string Thenameofthedefaultvalue string Thevalueofthedefaultcontext string Thecontextofthedefault

4.6.3.2Paths

HTTP Path Summaryget /default Returnsalistofdefaultsget /default/{name} Returnsthenameddefaultput /default/{name} Setadefaultdelete /default/{name} Deletesthenameddefault

4.6.3.2.1/default

4.6.3.2.1.1GET/default

Returnsalistofalldefaults

Responses

Code Description Schema200 Thelistofdefaults array[Default]401 Notauthorized String

4.6.3.2.2/default/{name}

4.6.3.2.2.1GET/default/{name}

Returnsadefaultbyname

Responses

Code Description Schema200 Returningtheinformationofthedefault Default401 Notauthorized String404 Thenameddefaultcouldnotbefound String

Page 67: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameofthedefault True String

4.6.3.2.2.2PUT/default/{name}

Setsthenameddefault

Responses

Code Description Schema200 Defaultupdatedorcreated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewdefaulttocreate True Default

4.6.3.2.2.3DELETE/default/{name}

Deletesadefaultbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenameddefaultcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthedefault True String

4.6.3.3default.yaml

openapi:"3.0.2"

info:

Page 68: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Default

description:|-

Adefaultisaspecialvariablethathasacontextassociatedwith

it.Thisallowsonetodefinevaluesthatcanbeeasilyretrieved

basedontheassociatedcontext.Forexample,adefaultcouldbe

theimagenameforacloudwherethecontextisdefinedbythe

cloudname.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/default:

get:

tags:

-Default

summary:Returnsalistofdefaults

description:Returnsalistofalldefaults

operationId:cloudmesh.default.list

responses:

'200':

description:Thelistofdefaults

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Default'

'401':

description:Notauthorized

/default/{name}:

get:

tags:

-Default

summary:Returnsthenameddefault

description:Returnsadefaultbyname

operationId:cloudmesh.default.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthedefault

responses:

'200':

description:Returningtheinformationofthedefault

content:

application/json:

schema:

$ref:'#/components/schemas/Default'

'401':

description:Notauthorized

'404':

description:Thenameddefaultcouldnotbefound

put:

tags:

-Default

summary:Setadefault

description:Setsthenameddefault

operationId:cloudmesh.default.add

requestBody:

description:Thenewdefaulttocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Default'

responses:

'200':

description:Defaultupdatedorcreated

'401':

description:Notauthorized

delete:

tags:

-Default

summary:Deletesthenameddefault

description:Deletesadefaultbyname

operationId:cloudmesh.default.delete_by_name

Page 69: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthedefault

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenameddefaultcouldnotbefound

components:

schemas:

Default:

type:object

description:thedefaults

properties:

name:

type:string

description:Thenameofthedefault

example:"image"

value:

type:string

description:Thevalueofthedefault

example:"m1.medium"

context:

type:string

description:Thecontextofthedefault

example:"cloud.vm.flavor"

4.7DATAMANAGEMENT

4.7.1Filestore

Afilestoreisaresourceallowingstorageofdataasatraditionalfile.Afilestorecancontaninanynumberoffileswithadditionalattributesdescribingthefile.Afilestoreislocatedonthephysical server. Itcontainsaccess to thecontentof the file.Thiscontrastsvirtualdirectoriesthatarejustpointerstofiles,whichcouldincludefileslocatedindifferentfilestores.Avirtualdirectoryalsodoesnotcontainthecontentofthefile,butjustapointerwheretofindthefile.

4.7.1.1SchemaFile

Reference:☁�

Property Type Descriptionname string Thenameofthefileendpoint string Thelocationofthefilechecksum string Thechecksumofthefilesize integer Thesizeofthefileinbytecontent string(binary) thecontentofthefile

4.7.1.2Paths

HTTP Path Summary

Page 70: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

get /file Returnsalistoffilesinthefilestoreget /file/{name} Returnsthenamedfileinthefilestoreput /file/{name} Uploadsafiletothelistoffilesinthefilestoredelete /file/{name} Deletesthenamedfileinthefilestore

4.7.1.2.1/file

4.7.1.2.1.1GET/file

Returnsalistofallfiles

Responses

Code Description Schema200 Thelistoffiles array[File]401 Notauthorized String

4.7.1.2.2/file/{name}

4.7.1.2.2.1GET/file/{name}

Returnsanfilebynameinthefilestore

Responses

Code Description Schema200 Returningtheinformationofthefilestore File401 Notauthorized String404 Thenamedfilecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefile True String

4.7.1.2.2.2PUT/file/{name}

Uploadsafiletothelistoffilesinthefilestore

Page 71: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Fileupdatedorcreated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thefiletobeuploaded True File

4.7.1.2.2.3DELETE/file/{name}

Deletesanfilebyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedfilecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefile True String

4.7.1.3filestore.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:File

description:|-

Afilestoreisaresourceallowingstorageofdataasatraditionalfile.

Afilestorecancontaninanynumberoffileswithadditionalattributes

describingthefile.Afilestoreislocatedonthephysicalserver.It

containsaccesstothecontentofthefile.Thiscontrastsvirtual

directoriesthatarejustpointerstofiles,whichcouldincludefiles

locatedindifferentfilestores.Avirtualdirectoryalsodoesnot

containthecontentofthefile,butjustapointerwheretofindthefile.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

Page 72: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/file:

get:

tags:

-File

summary:Returnsalistoffilesinthefilestore

description:Returnsalistofallfiles

operationId:cloudmesh.file.list

responses:

'200':

description:Thelistoffiles

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/File'

'401':

description:Notauthorized

/file/{name}:

get:

tags:

-File

summary:Returnsthenamedfileinthefilestore

description:Returnsanfilebynameinthefilestore

operationId:cloudmesh.file.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefile

responses:

'200':

description:Returningtheinformationofthefilestore

content:

application/json:

schema:

$ref:'#/components/schemas/File'

'401':

description:Notauthorized

'404':

description:Thenamedfilecouldnotbefound

put:

tags:

-File

summary:Uploadsafiletothelistoffilesinthefilestore

description:Uploadsafiletothelistoffilesinthefilestore

operationId:cloudmesh.file.add

requestBody:

description:Thefiletobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/File'

responses:

'200':

description:Fileupdatedorcreated

'401':

description:Notauthorized

delete:

tags:

-File

summary:Deletesthenamedfileinthefilestore

description:Deletesanfilebyname

operationId:cloudmesh.file.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefile

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedfilecouldnotbefound

components:

schemas:

File:

type:object

description:anobjectrepresentingafile

Page 73: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

properties:

name:

type:string

description:Thenameofthefile

endpoint:

type:string

description:Thelocationofthefile

checksum:

type:string

description:Thechecksumofthefile

size:

type:integer

description:Thesizeofthefileinbyte

content:

type:string

format:binary

description:thecontentofthefile

4.7.2Replica

In many distributed systems, it is important that a file can be replicated among differentsystemstoprovidefasteraccess.Itisimportanttoprovideamechanismtotracethepedigreeofthe filewhilepointing to itsoriginal source.A replicawillpoint toa file ina file storeandstorethecontentsinthefilestoreinsteadofthereplica.Thereplicaisjustapointer.

4.7.2.1SchemaReplica

Reference:☁�

Property Type Descriptionname string Thenameofthereplicafilename string Theoriginalfilenameendpoint string Thelocationofthefilechecksum string Thechecksumofthefilesize integer Thesizeofthefileinbyte

4.7.2.2Paths

HTTP Path Summaryget /replica Returnsalistofreplicasget /replica/{name} Returnsthenamedreplicaput /replica/{name} Uploadsareplicatothelistofreplicasdelete /replica/{name} Deletesthenamedreplica

4.7.2.2.1/replica

4.7.2.2.1.1GET/replica

Page 74: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returnsalistofallreplicas

Responses

Code Description Schema200 Thelistofreplicas array[Replica]401 Notauthorized String

4.7.2.2.2/replica/{name}

4.7.2.2.2.1GET/replica/{name}

Returnsanreplicabyname

Responses

Code Description Schema200 Returningtheinformationofthereplica Replica401 Notauthorized String404 Thenamedreplicacouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthereplica True String

4.7.2.2.2.2PUT/replica/{name}

Uploadsareplicatothelistofreplicas

Responses

Code Description Schema200 Replicaupdatedorcreated String401 Notauthorized String

RequestBody

Locatedin Description Required Schema

Page 75: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Body Thereplicatobeuploaded True Replica

4.7.2.2.2.3DELETE/replica/{name}

Deletesanreplicabyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedreplicacouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthereplica True String

4.7.2.3replica.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Replica

description:|-

Inmanydistributedsystems,itisimportantthatafilecanbe

replicatedamongdifferentsystemstoprovidefasteraccess.Itis

importanttoprovideamechanismtotracethepedigreeofthefile

whilepointingtoitsoriginalsource.Areplicawillpointtoafilein

afilestoreandstorethecontentsinthefilestoreinsteadofthe

replica.Thereplicaisjustapointer.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/replica:

get:

tags:

-Replica

summary:Returnsalistofreplicas

description:Returnsalistofallreplicas

operationId:cloudmesh.replica.list

responses:

'200':

description:Thelistofreplicas

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Replica'

'401':

description:Notauthorized

Page 76: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

/replica/{name}:

get:

tags:

-Replica

summary:Returnsthenamedreplica

description:Returnsanreplicabyname

operationId:cloudmesh.replica.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthereplica

responses:

'200':

description:Returningtheinformationofthereplica

content:

application/json:

schema:

$ref:'#/components/schemas/Replica'

'401':

description:Notauthorized

'404':

description:Thenamedreplicacouldnotbefound

put:

tags:

-Replica

summary:Uploadsareplicatothelistofreplicas

description:Uploadsareplicatothelistofreplicas

operationId:cloudmesh.replica.add

requestBody:

description:Thereplicatobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Replica'

responses:

'200':

description:Replicaupdatedorcreated

'401':

description:Notauthorized

delete:

tags:

-Replica

summary:Deletesthenamedreplica

description:Deletesanreplicabyname

operationId:cloudmesh.replica.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthereplica

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedreplicacouldnotbefound

components:

schemas:

Replica:

type:object

description:Anentryrepresentingafilereplicarecord

properties:

name:

type:string

description:Thenameofthereplica

filename:

type:string

description:Theoriginalfilename

endpoint:

type:string

description:Thelocationofthefile

checksum:

type:string

description:Thechecksumofthefile

size:

type:integer

description:Thesizeofthefileinbyte

4.7.3Database

Page 77: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Thedatabasespecificationallowstoregisteradatabaseandperformelementaryoperationstousethisdatabase.Wedistinguishactionsrelatedtotheregistration,theaddingofaschema,theinsertionofdataandthequeryofdata.Thedatabaseisdefinedbyanameanendpoint(e.g.,host,port),andaprotocolused(e.g.,SQL,MongoDB,graphgl,andothers).

4.7.3.1SchemaDatabase

Reference:☁�

Property Type Descriptionname string Nameofthedatabasedescription string Descriptionofthedatabaseendpoint string Endpointofthedatabasekind string thekindofthedatabase

4.7.3.2SchemaSchema

Reference:☁�

Property Type Descriptionname string Nameofthedatabasedescription string Descriptionofthedatabasekind string Thekindofthedefinitioncontent string Theschemaassociatedwiththetableorcollection

4.7.3.3SchemaRecord

Reference:☁�

Property Type Descriptionstatus string Thestatusofthereurnresult string Theresultofthequesryinjsonstringformat

4.7.3.4SchemaQuery

Reference:☁�

Property Type Description

Page 78: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

status string Thequerystring

4.7.3.5Paths

HTTP Path Summaryget /database Returnsalldatabasesget /database/{name}/schema Getthelistoftheschemaput /database/{name}/schema Uploadaschema

delete /database/{name}/schema Deletesadatabasefromthelistofdatabases

get /database/{name} Querythenameddatabaseput /database/{name} adddatatothetableorcollectiondelete /database/{name} Deletetheobjectsmatchingthequery

4.7.3.5.1/database

4.7.3.5.1.1GET/database

Returnsalldatabases

Responses

Code Description Schema200 Listofdatabases array[Database]401 Notauthorized String404 Nameddatabasenotfound String

4.7.3.5.2/database/{name}/schema

4.7.3.5.2.1GET/database/{name}/schema

Responses

Code Description Schema200 successfullyreturnedtheschema array[Schema]401 Notauthorized String404 Nameddatabasenotfound String

Page 79: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Nameoftheschema True String

4.7.3.5.2.2PUT/database/{name}/schema

Responses

Code Description Schema200 successfullyreturnedthelist Schema401 Notauthorized String404 Nameddatabasenotfound String

Parameters

Name Locatedin Description Required Schemaname path Nameofthedatabase True String

4.7.3.5.2.3DELETE/database/{name}/schema

Deletesadatabasefromthelistofdatabases

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Nameddatabasenotfound String

Parameters

Name Locatedin Description Required Schemaname path Nameofthedatabase True String

4.7.3.5.3/database/{name}

4.7.3.5.3.1GET/database/{name}

Page 80: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Querythenameddatabase

Responses

Code Description Schema200 Successfullquery array[Record]401 Notauthorized String404 Nameddatabasenotfound String

Parameters

Name Locatedin Description Required Schemaname path Nameofthedatabase True Stringquery query DatabaseQuery True Query

4.7.3.5.3.2PUT/database/{name}

Responses

Code Description Schema200 successfullyuploaded Record401 Notauthorized String404 Nameddatabasenotfound String

Parameters

Name Locatedin Description Required Schemaname path Nameofthedatabase True String

RequestBody

Locatedin Description Required SchemaBody Recordtobeuploaded True Record

4.7.3.5.3.3DELETE/database/{name}

Responses

Page 81: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Successfullquery array[Record]401 Notauthorized String404 Nameddatabasenotfound String

Parameters

Name Locatedin Description Required Schemaname path Nameofthedatabase True Stringquery query DatabaseQuery True Query

4.7.3.6database.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Database

description:|-

Thedatabasespecificationallowstoregisteradatabaseandperform

elementaryoperationstousethisdatabase.Wedistinguishactions

relatedtotheregistration,theaddingofaschema,theinsertionof

dataandthequeryofdata.Thedatabaseisdefinedbyanameanendpoint

(e.g.,host,port),andaprotocolused(e.g.,SQL,MongoDB,graphgl,and

others).

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/database:

get:

tags:

-"DatabaseRegistry"

summary:Returnsalldatabases

description:Returnsalldatabases

operationId:cloudmesh.database.get

responses:

'200':

description:Listofdatabases

content:

application/json:

schema:

type:array

items:

$ref:"#/components/schemas/Database"

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

/database/{name}/schema:

get:

tags:

-"DatabaseSchema"

summary:Getthelistoftheschema

description:""

operationId:"cloudmesh.database.get.schema"

parameters:

-name:name

description:Nameoftheschema

in:path

required:true

schema:

Page 82: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

responses:

'200':

description:"successfullyreturnedtheschema"

content:

application/json:

schema:

type:array

items:

$ref:"#/components/schemas/Schema"

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

put:

tags:

-"DatabaseSchema"

summary:"Uploadaschema"

description:""

operationId:"cloudmesh.database.put.schema"

parameters:

-name:name

description:Nameofthedatabase

in:path

required:true

schema:

type:string

responses:

'200':

description:"successfullyreturnedthelist"

content:

application/json:

schema:

$ref:"#/components/schemas/Schema"

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

delete:

tags:

-"DatabaseRegistry"

summary:Deletesadatabasefromthelistofdatabases

description:Deletesadatabasefromthelistofdatabases

operationId:cloudmesh.database.delete

parameters:

-name:name

description:Nameofthedatabase

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

/database/{name}:

get:

tags:

-"DatabaseData"

summary:Querythenameddatabase

description:Querythenameddatabase

operationId:"cloudmesh.database.data.get"

parameters:

-name:name

description:Nameofthedatabase

in:path

required:true

schema:

type:string

-in:query

name:query

description:DatabaseQuery

required:true

schema:

$ref:'#/components/schemas/Query'

responses:

'200':

description:Successfullquery

content:

application/json:

schema:

type:array

items:

$ref:"#/components/schemas/Record"

'401':

description:Notauthorized

Page 83: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

'404':

description:Nameddatabasenotfound

put:

tags:

-"DatabaseData"

summary:"adddatatothetableorcollection"

description:""

operationId:"cloudmesh.database.data.put"

parameters:

-name:name

description:Nameofthedatabase

in:path

required:true

schema:

type:string

requestBody:

description:Recordtobeuploaded

required:true

content:

application/json:

schema:

$ref:"#/components/schemas/Record"

responses:

'200':

description:"successfullyuploaded"

content:

application/json:

schema:

$ref:"#/components/schemas/Record"

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

delete:

tags:

-"DatabaseData"

summary:"Deletetheobjectsmatchingthequery"

description:""

operationId:"cloudmesh.database.data.delete"

parameters:

-name:name

description:Nameofthedatabase

in:path

required:true

schema:

type:string

-name:query

description:DatabaseQuery

in:query

required:true

schema:

$ref:'#/components/schemas/Query'

responses:

'200':

description:Successfullquery

content:

application/json:

schema:

type:array

items:

$ref:"#/components/schemas/Record"

'401':

description:Notauthorized

'404':

description:Nameddatabasenotfound

components:

schemas:

Database:

type:object

description:Definesadatabaseobjectasanentry

properties:

name:

type:string

description:Nameofthedatabase

description:

type:string

description:Descriptionofthedatabase

endpoint:

type:string

description:Endpointofthedatabase

kind:

type:string

description:thekindofthedatabase

Schema:

type:object

description:Definesadatabase

properties:

name:

type:string

Page 84: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Nameofthedatabase

description:

type:string

description:Descriptionofthedatabase

kind:

type:string

description:Thekindofthedefinition

content:

type:string

description:Theschemaassociatedwiththetableorcollection

Record:

type:object

description:Theresultofaquery

properties:

status:

type:string

description:Thestatusofthereurn

result:

type:string

description:Theresultofthequesryinjsonstringformat

Query:

type:object

description:Thequery

properties:

status:

type:string

description:Thequerystring

4.7.4VirtualDirectory

Avirtualdirectoryisacollectionoffiles,replicas,streamsorothervirtualdirectories.

4.7.4.1SchemaVirtualdirectory

Reference:☁�

Property Type Descriptionname string Thenameofthevirtualdirectorydescription string Descriptionofthevirtualdirectoryhost string Remotehostofthevirtualdirectorylocation string Remotelocation,e.g.,adirectorywithfullpathonahostprotocol string Accessprotocol(e.g.HTTP,FTP,SSH,etc.)credential object Credentialtoaccess

4.7.4.2Paths

HTTP Path Summary

get /virtualdirectory Returnsalistofvirtualdirectories

get /virtualdirectory/{name} Returnsthenamedvirtualdirectory

put /virtualdirectory/{name} Uploadsavirtualdirectorytothelistofvirtualdirectories

Page 85: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

delete /virtualdirectory/{name} Deletesthenamedvirtualdirectory

get /virtualdirectory/{name}/{filename} Returnsthespecificfileofthatvirtualdirectory

put /virtualdirectory/{name}/{filename} Updatesoraddsavirtualfileinthevirtualdirectory

delete /virtualdirectory/{name}/{filename} Deleteanuserinthevirtualdirectory

4.7.4.2.1/virtualdirectory

4.7.4.2.1.1GET/virtualdirectory

Returnsalistofallvirtualdirectories

Responses

Code Description Schema200 Thelistofvirtualdirectories array[Virtualdirectory]401 Notauthorized String

4.7.4.2.2/virtualdirectory/{name}

4.7.4.2.2.1GET/virtualdirectory/{name}

Returnsanvirtualdirectorybyname

Responses

Code Description Schema200 Returningtheinformationofthevirtualdirectory Virtualdirectory401 Notauthorized String404 Thenamedvirtualdirectorycouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualdirectory True String

Page 86: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.7.4.2.2.2PUT/virtualdirectory/{name}

Uploadsavirtualdirectorytothelistofvirtualdirectories

Responses

Code Description Schema200 Virtualdirectoryupdatedorcreated String401 Notauthorized String404 Thenamedvirtualdirectorycouldnotbefound String

RequestBody

Locatedin Description Required SchemaBody Thevirtualdirectorytobeuploaded True Virtualdirectory

4.7.4.2.2.3DELETE/virtualdirectory/{name}

Deletesanvirtualdirectorybyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedvirtualdirectorycouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualdirectory True String

4.7.4.2.3/virtualdirectory/{name}/{filename}

4.7.4.2.3.1GET/virtualdirectory/{name}/{filename}

Returnsthespecificfileofthatvirtualdirectory

Responses

Page 87: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 uploadsucessful File401 Notauthorized String404 Thenamedvirtualdirectoryorfilecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualdirectory True Stringfilename path Thefilename True String

4.7.4.2.3.2PUT/virtualdirectory/{name}/{filename}

Updatesoraddsavirtualfileinthevirtualdirectory

Responses

Code Description Schema200 Useraddedsucessfully String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualdirectory True Stringfilename path Thefilename True String

RequestBody

Locatedin Description Required SchemaBody Theusertobeuploaded True File

4.7.4.2.3.3DELETE/virtualdirectory/{name}/{filename}

Deleteanuserinthevirtualdirectory

Responses

Page 88: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedvirtualdirectoryorfilecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualdirectory True Stringfilename path Thefilename True String

4.7.4.3virtualdirectory.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:VirtualDirectory

description:|-

Avirtualdirectoryisacollectionoffiles,replicas,streamsorother

virtualdirectories.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroupService

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/virtualdirectory:

get:

tags:

-Virtualdirectory

summary:Returnsalistofvirtualdirectories

description:Returnsalistofallvirtualdirectories

operationId:cloudmesh.virtualdirectory.list

responses:

'200':

description:Thelistofvirtualdirectories

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Virtualdirectory'

'401':

description:Notauthorized

/virtualdirectory/{name}:

get:

tags:

-Virtualdirectory

summary:Returnsthenamedvirtualdirectory

description:Returnsanvirtualdirectorybyname

operationId:cloudmesh.virtualdirectory.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualdirectory

responses:

'200':

description:Returningtheinformationofthevirtualdirectory

Page 89: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

content:

application/json:

schema:

$ref:'#/components/schemas/Virtualdirectory'

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectorycouldnotbefound

put:

tags:

-Virtualdirectory

summary:Uploadsavirtualdirectorytothelistofvirtualdirectories

description:Uploadsavirtualdirectorytothelistofvirtualdirectories

operationId:cloudmesh.virtualdirectory.add

requestBody:

description:Thevirtualdirectorytobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Virtualdirectory'

responses:

'200':

description:Virtualdirectoryupdatedorcreated

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectorycouldnotbefound

delete:

tags:

-Virtualdirectory

summary:Deletesthenamedvirtualdirectory

description:Deletesanvirtualdirectorybyname

operationId:cloudmesh.virtualdirectory.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualdirectory

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectorycouldnotbefound

/virtualdirectory/{name}/{filename}:

get:

tags:

-Virtualdirectory

summary:Returnsthespecificfileofthatvirtualdirectory

description:Returnsthespecificfileofthatvirtualdirectory

operationId:cloudmesh.virtualdirectory.file.get_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualdirectory

-name:filename

description:Thefilename

in:path

required:true

schema:

type:string

responses:

'200':

description:uploadsucessful

content:

application/json:

schema:

$ref:"filestore.yaml#/components/schemas/File"

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectoryorfilecouldnotbefound

put:

tags:

-Virtualdirectory

summary:Updatesoraddsavirtualfileinthevirtualdirectory

description:Updatesoraddsavirtualfileinthevirtualdirectory

operationId:cloudmesh.virtualdirectory.file.add

parameters:

-name:name

in:path

required:true

Page 90: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

schema:

type:string

description:Thenameofthevirtualdirectory

-name:filename

description:Thefilename

in:path

required:true

schema:

type:string

requestBody:

description:Theusertobeuploaded

required:true

content:

application/json:

schema:

$ref:"filestore.yaml#/components/schemas/File"

responses:

'200':

description:Useraddedsucessfully

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectoryorfilecouldnotbefound

delete:

tags:

-Virtualdirectory

summary:Deleteanuserinthevirtualdirectory

description:Deleteanuserinthevirtualdirectory

operationId:cloudmesh.virtualdirectory.file.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualdirectory

-name:filename

description:Thefilename

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedvirtualdirectoryorfilecouldnotbefound

components:

schemas:

Virtualdirectory:

type:object

description:thevirtualdirectory

properties:

name:

description:Thenameofthevirtualdirectory

type:string

description:

description:Descriptionofthevirtualdirectory

type:string

host:

description:Remotehostofthevirtualdirectory

type:string

location:

description:Remotelocation,e.g.,adirectorywithfullpathona

host

type:string

protocol:

description:Accessprotocol(e.g.HTTP,FTP,SSH,etc.)

type:string

credential:

description:Credentialtoaccess

type:object

4.8COMPUTEMANAGEMENT-VIRTUALCLUSTERS

4.8.1VirtualCluster

AVirtualClusterismodeledasmanagernode,andoneormorecomputenodes.Themanager

Page 91: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

node usually serves as a login node and can be accessed from outside via a public IP. Thecompute nodes are connected to themanager node via a private, usually high performance(high throughputand lowlatency)networkand thusaccessibleonlyfromthemanagernode.To use the virtual cluster, login to themanager node, and from there one can login to anycomputenode,orsubmitajobtorunonthecomputenodes.

4.8.1.1SchemaVirtualcluster

Reference:☁�

Property Type Descriptionname string Thenameofthevirtualclusterdescription string Adescriptionofthevirtualclusterowner string Usernameoftheownerofthevirtualclustermanager Node Managernodeofthevirtualclusternodes array[Node] Listofnodesofthevirtualcluster

4.8.1.2SchemaNode

Reference:☁�

Property Type Descriptionname string Nameofthenodestate string Powerstateofthenodencpu integer NumberofvirtualCPUsofthenoderam string RAMsizeofthenodedisk string Disksizeofthenodenics array[NIC] Listofnetworkinterfacesofthenode

4.8.1.3SchemaNIC

Reference:☁�

Property Type Descriptionmac string MACaddressofthenodeip string IPaddressofthenode

4.8.1.4Paths

Page 92: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

HTTP Path Summaryget /virtualcluster Returnsalistofvirtualclustersget /virtualcluster/{name} Returnsthenamedvirtualcluster

put /virtualcluster/{name} Uploadsanvirtualclustertothelistofvirtualclusters

delete /virtualcluster/{name} Deletesthenamedvirtualclusterget /virtualcluster/{name}/{node} Nodeofthenamedvirtualcluster

put /virtualcluster/{name}/{node} Updatesoraddsanodetothevirtualcluster

delete /virtualcluster/{name}/{node} Deleteanodeinthevirtualcluster

4.8.1.4.1/virtualcluster

4.8.1.4.1.1GET/virtualcluster

Returnsalistofallvirtualclusters

Responses

Code Description Schema200 Thelistofvirtualclusters array[Virtualcluster]401 Notauthorized String

4.8.1.4.2/virtualcluster/{name}

4.8.1.4.2.1GET/virtualcluster/{name}

Returnsanvirtualclusterbyname

Responses

Code Description Schema200 Returningtheinformationofthevirtualcluster Virtualcluster401 Notauthorized String404 Thenamedvirtualclustercouldnotbefound String

Parameters

Page 93: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Name Locatedin Description Required Schemaname path Thenameofthevirtualcluster True String

4.8.1.4.2.2PUT/virtualcluster/{name}

Uploadsanvirtualclustertothelistofvirtualclusters

Responses

Code Description Schema200 Virtualclusterupdatedorcreated String401 Notauthorized String404 Thenamedvirtualclustercouldnotbefound String

RequestBody

Locatedin Description Required SchemaBody Thevirtualclustertobeuploaded True Virtualcluster

4.8.1.4.2.3DELETE/virtualcluster/{name}

Deletesanvirtualclusterbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedvirtualclustercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualcluster True String

4.8.1.4.3/virtualcluster/{name}/{node}

4.8.1.4.3.1GET/virtualcluster/{name}/{node}

Page 94: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returns the specific node of the named virtual cluster. If the node name is manager, themanagernodeisused.Acomputenodecannotbenamedmanager

Responses

Code Description Schema200 Nodeinfo Node401 Notauthorized String404 Thenamedvirtualclusterornodecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualcluster True Stringnode path Thenodename True String

4.8.1.4.3.2PUT/virtualcluster/{name}/{node}

Updatesoraddsanodetothevirtualcluster.Ifthenodenameismanager,themanagernodeisuploaded.

Responses

Code Description Schema200 Nodeaddedsucessfully String401 Notauthorized String404 Thenamedvirtualclusterornodecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualcluster True Stringnode path Thenodename True String

RequestBody

Locatedin Description Required SchemaBody Thenodetobeuploaded True Node

Page 95: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.8.1.4.3.3DELETE/virtualcluster/{name}/{node}

Deleteanodeinthevirtualcluster

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedvirtualclusterornodecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthevirtualcluster True Stringnode path Thenodename True String

4.8.1.5virtualcluster.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:VirtualCluster

description:|-

AVirtualClusterismodeledasmanagernode,andoneormore

computenodes.Themanagernodeusuallyservesasaloginnodeand

canbeaccessedfromoutsideviaapublicIP.Thecomputenodesare

connectedtothemanagernodeviaaprivate,usuallyhighperformance(high

throughputandlowlatency)networkandthusaccessibleonlyfromthe

managernode.Tousethevirtualcluster,logintothemanagernode,and

fromthereonecanlogintoanycomputenode,orsubmitajobtorunonthe

computenodes.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroupService

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/virtualcluster:

get:

tags:

-Virtualcluster

summary:Returnsalistofvirtualclusters

description:Returnsalistofallvirtualclusters

operationId:cloudmesh.virtualcluster.list

responses:

'200':

description:Thelistofvirtualclusters

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Virtualcluster'

'401':

description:Notauthorized

Page 96: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

/virtualcluster/{name}:

get:

tags:

-Virtualcluster

summary:Returnsthenamedvirtualcluster

description:Returnsanvirtualclusterbyname

operationId:cloudmesh.virtualcluster.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualcluster

responses:

'200':

description:Returningtheinformationofthevirtualcluster

content:

application/json:

schema:

$ref:'#/components/schemas/Virtualcluster'

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclustercouldnotbefound

put:

tags:

-Virtualcluster

summary:Uploadsanvirtualclustertothelistofvirtualclusters

description:Uploadsanvirtualclustertothelistofvirtualclusters

operationId:cloudmesh.virtualcluster.add

requestBody:

description:Thevirtualclustertobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Virtualcluster'

responses:

'200':

description:Virtualclusterupdatedorcreated

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclustercouldnotbefound

delete:

tags:

-Virtualcluster

summary:Deletesthenamedvirtualcluster

description:Deletesanvirtualclusterbyname

operationId:cloudmesh.virtualcluster.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualcluster

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclustercouldnotbefound

/virtualcluster/{name}/{node}:

get:

tags:

-Virtualcluster

summary:Nodeofthenamedvirtualcluster

description:Returnsthespecificnodeofthenamedvirtualcluster.If

thenodenameismanager,themanagernodeisused.A

computenodecannotbenamedmanager

operationId:cloudmesh.virtualcluster.node.get_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualcluster

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

responses:

'200':

Page 97: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Nodeinfo

content:

application/json:

schema:

$ref:"#/components/schemas/Node"

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclusterornodecouldnotbefound

put:

tags:

-Virtualcluster

summary:Updatesoraddsanodetothevirtualcluster

description:Updatesoraddsanodetothevirtualcluster.If

thenodenameismanager,themanagernodeisuploaded.

operationId:cloudmesh.virtualcluster.node.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualcluster

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

requestBody:

description:Thenodetobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Node'

responses:

'200':

description:Nodeaddedsucessfully

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclusterornodecouldnotbefound

delete:

tags:

-Virtualcluster

summary:Deleteanodeinthevirtualcluster

description:Deleteanodeinthevirtualcluster

operationId:cloudmesh.virtualcluster.node.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthevirtualcluster

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedvirtualclusterornodecouldnotbefound

components:

schemas:

Virtualcluster:

type:object

properties:

name:

description:Thenameofthevirtualcluster

type:string

description:

type:string

description:Adescriptionofthevirtualcluster

owner:

type:string

description:Usernameoftheownerofthevirtualcluster

manager:

description:Managernodeofthevirtualcluster

$ref:"#/components/schemas/Node"

nodes:

description:Listofnodesofthevirtualcluster

type:array

Page 98: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

items:

$ref:"#/components/schemas/Node"

Node:

type:object

properties:

name:

type:string

description:Nameofthenode

state:

type:string

description:Powerstateofthenode

ncpu:

type:integer

description:NumberofvirtualCPUsofthenode

ram:

type:string

description:RAMsizeofthenode

disk:

type:string

description:Disksizeofthenode

nics:

type:array

description:Listofnetworkinterfacesofthenode

items:

$ref:"#/components/schemas/NIC"

NIC:

type:object

properties:

mac:

type:string

description:MACaddressofthenode

ip:

type:string

description:IPaddressofthenode

4.8.2NetworkofNodes

ANetwork of Nodes (NON) contains a number of compute nodes that are connected by anetwork and can be reached from each other. The concept is a generalization of the termNetworkofWorkstations.IncontrasttoaVirtualClusteritdoesnothaveadedicatedmanagernode. Network of nodes can be real or virtual. The same security context can be used toauthenticatetoallnodesinthenetworkofnodes.Thisistypicallydonewithapublickeystoreinwhichallkeysarestoredthatallowaccesstothenodes.

4.8.2.1SchemaNon

Reference:☁�

Property Type Descriptionname string Thenameofthenetworkofnodesdescription string Adescriptionofthenetworkofnodesnodes array[Node] Listofnodesofthenetworkofnodes

4.8.2.2Paths

HTTP Path Summaryget /non Returnsalistofnetworkofnodess

Uploadsanetworkofnodestothelistof

Page 99: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

put /non/{name} networkofnodess

get /non/{name}/publickeystore Returnstheinformationofthekeystoredelete /non/{name}/publickeystore Deletesthekeystoreput /non/{name}/publickeystore Addsakeystoreget /non/{name}/node Returnsthenamednetworkofnodesdelete /non/{name}/node Deletesthenamednetworkofnodesget /non/{name}/node/{node} Nodeofthenamednetworkofnodes

put /non/{name}/node/{node} Updatesoraddsanodetothenetworkofnodes

delete /non/{name}/node/{node} Deleteanodeinthenetworkofnodes

4.8.2.2.1/non

4.8.2.2.1.1GET/non

Returnsalistofallnetworkofnodess

Responses

Code Description Schema200 Thelistofnetworkofnodess array[Non]401 Notauthorized String

4.8.2.2.2/non/{name}

4.8.2.2.2.1PUT/non/{name}

Uploadsanetworkofnodestothelistofnetworkofnodess

Responses

Code Description Schema200 Networkofnodesupdatedorcreated. String400 Errorupdatingnetworkofnodes String401 Notauthorized String

RequestBody

Page 100: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Locatedin Description Required SchemaBody Thenetworkofnodestobeuploaded True Non

4.8.2.2.3/non/{name}/publickeystore

4.8.2.2.3.1GET/non/{name}/publickeystore

Returnsanetworkofnodesbyname

Responses

Code Description Schema200 Returningtheinformationofthenetworkofnodes string401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

4.8.2.2.3.2DELETE/non/{name}/publickeystore

Deletesanetworkofnodesbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

4.8.2.2.3.3PUT/non/{name}/publickeystore

Updatesoraddsanodetothenetworkofnodes.

Page 101: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Nodekeystoreaddedsucessfully String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

4.8.2.2.4/non/{name}/node

4.8.2.2.4.1GET/non/{name}/node

Returnsanetworkofnodesbyname

Responses

Code Description Schema200 Returningtheinformationofthenode Non401 Notauthorized String404 Thenamednetworkofnodescouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

4.8.2.2.4.2DELETE/non/{name}/node

Deletesanetworkofnodesbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamednetworkofnodescouldnotbefound String

Page 102: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

4.8.2.2.5/non/{name}/node/{node}

4.8.2.2.5.1GET/non/{name}/node/{node}

Returnsthespecificnodeofthenamednetworkofnodes.

Responses

Code Description Schema200 Nodeinfo Node401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True Stringnode path Thenodename True String

4.8.2.2.5.2PUT/non/{name}/node/{node}

Updatesoraddsanodetothenetworkofnodes

Responses

Code Description Schema200 Nodeaddedsucessfully String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True String

Page 103: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

node path Thenodename True String

RequestBody

Locatedin Description Required SchemaBody Thenodetobeuploaded True Node

4.8.2.2.5.3DELETE/non/{name}/node/{node}

Deleteanodeinthenetworkofnodes

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthenetworkofnodes True Stringnode path Thenodename True String

4.8.2.3non.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:NetworkofNodes

description:|-

ANetworkofNodes(NON)containsanumberofcomputenodesthatare

connectedbyanetworkandcanbereachedfromeachother.Theconceptisa

generalizationofthetermNetworkofWorkstations.Incontrasttoa

VirtualClusteritdoesnothaveadedicatedmanagernode.Networkof

nodescanberealorvirtual.Thesamesecuritycontextcanbeusedto

authenticatetoallnodesinthenetworkofnodes.Thisistypicallydone

withapublickeystoreinwhichallkeysarestoredthatallowaccessto

thenodes.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroupService

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/non:

get:

tags:

Page 104: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

-Networkofnodes

summary:Returnsalistofnetworkofnodess

description:Returnsalistofallnetworkofnodess

operationId:cloudmesh.non.list

responses:

'200':

description:Thelistofnetworkofnodess

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Non'

'401':

description:Notauthorized

/non/{name}:

put:

tags:

-Networkofnodes

summary:Uploadsanetworkofnodestothelistofnetworkofnodess

description:Uploadsanetworkofnodestothelistofnetworkofnodess

operationId:cloudmesh.non.add

requestBody:

description:Thenetworkofnodestobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Non'

responses:

'200':

description:Networkofnodesupdatedorcreated.

'400':

description:Errorupdatingnetworkofnodes

'401':

description:Notauthorized

/non/{name}/publickeystore:

get:

tags:

-Non

summary:Returnstheinformationofthekeystore

description:Returnsanetworkofnodesbyname

operationId:cloudmesh.non.keystore.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

responses:

'200':

description:Returningtheinformationofthenetworkofnodes

content:

application/json:

schema:

type:string

description:theendpointofthepublickeystore

'401':

description:Notauthorized

delete:

tags:

-Networkofnodes

summary:Deletesthekeystore

description:Deletesanetworkofnodesbyname

operationId:cloudmesh.non.keystore.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

put:

tags:

-Networkofnodes

summary:Addsakeystore

description:Updatesoraddsanodetothenetworkofnodes.

operationId:cloudmesh.non.keystore.add

parameters:

-name:name

in:path

required:true

schema:

type:string

Page 105: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Thenameofthenetworkofnodes

responses:

'200':

description:Nodekeystoreaddedsucessfully

'401':

description:Notauthorized

/non/{name}/node:

get:

tags:

-Non

summary:Returnsthenamednetworkofnodes

description:Returnsanetworkofnodesbyname

operationId:cloudmesh.non.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

responses:

'200':

description:Returningtheinformationofthenode

content:

application/json:

schema:

$ref:'#/components/schemas/Non'

'401':

description:Notauthorized

'404':

description:Thenamednetworkofnodescouldnotbefound

delete:

tags:

-Networkofnodes

summary:Deletesthenamednetworkofnodes

description:Deletesanetworkofnodesbyname

operationId:cloudmesh.non.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamednetworkofnodescouldnotbefound

/non/{name}/node/{node}:

get:

tags:

-Networkofnodes

summary:Nodeofthenamednetworkofnodes

description:Returnsthespecificnodeofthenamednetworkofnodes.

operationId:cloudmesh.non.node.get_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

responses:

'200':

description:Nodeinfo

content:

application/json:

schema:

$ref:"virtualcluster.yaml#/components/schemas/Node"

'401':

description:Notauthorized

put:

tags:

-Networkofnodes

summary:Updatesoraddsanodetothenetworkofnodes

description:Updatesoraddsanodetothenetworkofnodes

operationId:cloudmesh.non.node.add

parameters:

-name:name

in:path

Page 106: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

requestBody:

description:Thenodetobeuploaded

required:true

content:

application/json:

schema:

$ref:"virtualcluster.yaml#/components/schemas/Node"

responses:

'200':

description:Nodeaddedsucessfully

'401':

description:Notauthorized

delete:

tags:

-Networkofnodes

summary:Deleteanodeinthenetworkofnodes

description:Deleteanodeinthenetworkofnodes

operationId:cloudmesh.non.node.delete

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkofnodes

-name:node

description:Thenodename

in:path

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

components:

schemas:

Non:

type:object

properties:

name:

description:Thenameofthenetworkofnodes

type:string

description:

type:string

description:Adescriptionofthenetworkofnodes

nodes:

description:Listofnodesofthenetworkofnodes

type:array

items:

$ref:"virtualcluster.yaml#/components/schemas/Node"

4.8.3Scheduler

Aschedulerallowstocontroltheexecutionoftasksbasedonapolicy.Schedulersmayallowtheassignmentofdifferntpoliciestodefinetheorderofthetasks.Aschedulerreturnsthenexttasktobeexecuted.Taskscanbeaddedanddeleted.

4.8.3.1SchemaTask

Reference:☁�

Property Type Descriptionname string nameofthescheduler

Page 107: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

user string theusernamethetaskbelongstodescription string Thedescriptionofthetaskkind string Thekindofthetask

4.8.3.2SchemaPolicy

Reference:☁�

Property Type Descriptionname string nameoftheschedulerpolicydescription string Thedescriptionofthepolicykind string Thekindofthepolicyparameters string parameterstodefinethebehaviourofthescheduler

4.8.3.3Paths

HTTP Path Summaryget /task Returnsalistoftasksget /task/{name} Returnsthenamedtaskput /task/{name} Uploadsatasktothelistoftasksdelete /task/{name} Deletesthenamedtaskget /policy Returnsthepolicyfoundput /policy Uploadsthepolicy

4.8.3.3.1/task

4.8.3.3.1.1GET/task

Returnsalistofalltasks

Responses

Code Description Schema200 Thelistoftasks array[Task]401 Notauthorized String

4.8.3.3.2/task/{name}

Page 108: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.8.3.3.2.1GET/task/{name}

Returnsantaskbyname

Responses

Code Description Schema200 Returningtheinformationofthetask Task401 Notauthorized String404 Thenamedtaskcouldnotbefound String

Parameters

Name Locatedin Description Required Schema

name path Thenameofthetask True String

operation query Showthetaskbutdonotremoveitfromthequeue False String

4.8.3.3.2.2PUT/task/{name}

Uploadsatasktothelistoftasks

Responses

Code Description Schema200 Taskupdated String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthetask True String

RequestBody

Locatedin Description Required SchemaBody Thetasktobeuploaded True Task

Page 109: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.8.3.3.2.3DELETE/task/{name}

Deletesantaskbyname

Responses

Code Description Schema200 Deletionsuccessful String400 Errortodeletethetask String401 Notauthorized String404 Thenamedtaskcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthetask True String

4.8.3.3.3/policy

4.8.3.3.3.1GET/policy

Returnsthepolicy

Responses

Code Description Schema200 Thepolicy array[Policy]401 Notauthorized String

4.8.3.3.3.2PUT/policy

Uploadsatasktothelistoftasks

Responses

Code Description Schema200 Taskupdated String400 Errorupdating String401 Notauthorized String

Page 110: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

RequestBody

Locatedin Description Required SchemaBody Thepolicytobeuploaded True Policy

4.8.3.4scheduler.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Queue

description:|-

Aschedulerallowstocontroltheexecutionoftasksbasedonapolicy.

Schedulersmayallowtheassignmentofdifferntpoliciestodefinethe

orderofthetasks.Aschedulerreturnsthenexttasktobeexecuted.

Taskscanbeaddedanddeleted.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3/scheduler

paths:

/task:

get:

tags:

-Scheduler

summary:Returnsalistoftasks

description:Returnsalistofalltasks

operationId:cloudmesh.scheduler.task.list

responses:

'200':

description:Thelistoftasks

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Task'

'401':

description:Notauthorized

/task/{name}:

get:

tags:

-Scheduler

summary:Returnsthenamedtask

description:Returnsantaskbyname

operationId:cloudmesh.scheduler.task.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

-in:query

name:operation

description:Showthetaskbutdonotremoveitfromthequeue

schema:

type:string

enum:

-info

responses:

'200':

description:Returningtheinformationofthetask

content:

application/json:

schema:

$ref:'#/components/schemas/Task'

'401':

description:Notauthorized

'404':

description:Thenamedtaskcouldnotbefound

Page 111: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

put:

tags:

-Scheduler

summary:Uploadsatasktothelistoftasks

description:Uploadsatasktothelistoftasks

operationId:cloudmesh.scheduler.task.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

requestBody:

description:Thetasktobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Task'

responses:

'200':

description:Taskupdated

'401':

description:Notauthorized

delete:

tags:

-Scheduler

summary:Deletesthenamedtask

description:Deletesantaskbyname

operationId:cloudmesh.scheduler.task.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

responses:

'200':

description:Deletionsuccessful

'400':

description:Errortodeletethetask

'401':

description:Notauthorized

'404':

description:Thenamedtaskcouldnotbefound

/policy:

get:

tags:

-Scheduler

summary:Returnsthepolicyfound

description:Returnsthepolicy

operationId:cloudmesh.scheduler.policy.list

responses:

'200':

description:Thepolicy

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Policy'

'401':

description:Notauthorized

put:

tags:

-Scheduler

summary:Uploadsthepolicy

description:Uploadsatasktothelistoftasks

operationId:cloudmesh.scheduler.policy.add

requestBody:

description:Thepolicytobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Policy'

responses:

'200':

description:Taskupdated

'400':

description:Errorupdating

'401':

description:Notauthorized

components:

schemas:

Task:

type:object

description:thescheduler

Page 112: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

properties:

name:

type:string

description:nameofthescheduler

user:

type:string

description:theusernamethetaskbelongsto

description:

type:string

description:Thedescriptionofthetask

kind:

type:string

description:Thekindofthetask

Policy:

type:object

description:Thepolicyofthescheduler

properties:

name:

type:string

description:nameoftheschedulerpolicy

description:

type:string

description:Thedescriptionofthepolicy

kind:

type:string

description:Thekindofthepolicy

parameters:

type:string

description:parameterstodefinethebehaviourofthescheduler

4.8.4Queue

Thequeueisaspecialschedulerthatallowstaskstobescheduledwitledoingqueuepolicies,suchasLIFO,FIFO,andsoon.Aqueue returns thenext task tobeexecuted.Taskscanbeaddedanddeleted.

4.8.4.1SchemaTask

Reference:☁�

Property Type Descriptionname string Nameofthescheduleruser string Theusernamethetaskbelongstodescription string Thedescriptionofthetaskkind string Thekindofthetaskcontent string Thecontentofthetask

4.8.4.2SchemaPolicy

Reference:☁�

Property Type Descriptionname string Nameoftheschedulerpolicydescription string Thedescriptionofthepolicykind string Thekindofthepolicy

Page 113: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

parameters string parameterstodefinethebehaviourofthescheduler

4.8.4.3Paths

HTTP Path Summaryget /task Returnsalistoftasksget /task/{name} Returnsthenamedtaskput /task/{name} Uploadsatasktothelistoftasksdelete /task/{name} Deletesthenamedtaskget /policy Returnsthepolicyput /policy Uploadsthepolicy

4.8.4.3.1/task

4.8.4.3.1.1GET/task

Returnsalistofalltasks

Responses

Code Description Schema200 Thelistoftasks array[Task]400 Notasksfound String401 Notauthorized String

4.8.4.3.2/task/{name}

4.8.4.3.2.1GET/task/{name}

Returnsantaskbyname

Responses

Code Description Schema200 Returningtheinformationofthetask Task400 Notaskfound String401 Notauthorized String404 Thenamedtaskcouldnotbefound String

Page 114: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameofthetask True Stringoperation query ERROR:descriptionmissing False String

4.8.4.3.2.2PUT/task/{name}

Uploadsatasktothelistoftasks

Responses

Code Description Schema200 Taskupdated String400 Errorupdatingtask. String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthetask True String

RequestBody

Locatedin Description Required SchemaBody Thetasktobeuploaded True Task

4.8.4.3.2.3DELETE/task/{name}

Deletesantaskbyname

Responses

Code Description Schema200 Deletionsuccessful String400 Notaskfound String401 Notauthorized String

Page 115: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

404 Thenamedtaskcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthetask True String

4.8.4.3.3/policy

4.8.4.3.3.1GET/policy

Returnsthepolocy

Responses

Code Description Schema200 Thepolicy array[Policy]400 Notasksfound String401 Notauthorized String

4.8.4.3.3.2PUT/policy

Uploadsatasktothelistoftasks

Responses

Code Description Schema200 Taskupdated String400 Errorupdatingtask String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody ThePolicy True Policy

4.8.4.4queue.yaml

openapi:"3.0.2"

info:

Page 116: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Scheduler

description:|-

Thequeueisaspecialschedulerthatallowstaskstobescheduled

witledoingqueuepolicies,suchasLIFO,FIFO,andsoon.

Aqueuereturnsthenexttasktobeexecuted.Taskscanbeadded

and

deleted.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3/scheduler

paths:

/task:

get:

tags:

-Task

summary:Returnsalistoftasks

description:Returnsalistofalltasks

operationId:cloudmesh.task.list

responses:

'200':

description:Thelistoftasks

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Task'

'400':

description:Notasksfound

'401':

description:Notauthorized

/task/{name}:

get:

tags:

-Task

summary:Returnsthenamedtask

description:Returnsantaskbyname

operationId:cloudmesh.task.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

-in:query

name:operation

schema:

type:string

enum:

-info

-pop

responses:

'200':

description:Returningtheinformationofthetask

content:

application/json:

schema:

$ref:'#/components/schemas/Task'

'400':

description:Notaskfound

'401':

description:Notauthorized

'404':

description:Thenamedtaskcouldnotbefound

put:

tags:

-Task

summary:Uploadsatasktothelistoftasks

description:Uploadsatasktothelistoftasks

operationId:cloudmesh.task.add

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

Page 117: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

requestBody:

description:Thetasktobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Task'

responses:

'200':

description:Taskupdated

'400':

description:Errorupdatingtask.

'401':

description:Notauthorized

delete:

tags:

-Task

summary:Deletesthenamedtask

description:Deletesantaskbyname

operationId:cloudmesh.task.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthetask

responses:

'200':

description:Deletionsuccessful

'400':

description:Notaskfound

'401':

description:Notauthorized

'404':

description:Thenamedtaskcouldnotbefound

/policy:

get:

tags:

-Task

summary:Returnsthepolicy

description:Returnsthepolocy

operationId:cloudmesh.task.policy.list

responses:

'200':

description:Thepolicy

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Policy'

'400':

description:Notasksfound

'401':

description:Notauthorized

put:

tags:

-Task

summary:Uploadsthepolicy

description:Uploadsatasktothelistoftasks

operationId:cloudmesh.task.policy.add

requestBody:

description:ThePolicy

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Policy'

responses:

'200':

description:Taskupdated

'400':

description:Errorupdatingtask

'401':

description:Notauthorized

components:

schemas:

Task:

type:object

description:Thescheduler

properties:

name:

type:string

description:Nameofthescheduler

user:

type:string

description:Theusernamethetaskbelongsto

description:

Page 118: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

description:Thedescriptionofthetask

kind:

type:string

description:Thekindofthetask

content:

type:string

description:Thecontentofthetask

Policy:

type:object

description:Thepolicyofthescheduler

properties:

name:

type:string

description:Nameoftheschedulerpolicy

description:

type:string

description:Thedescriptionofthepolicy

kind:

type:string

description:Thekindofthepolicy

parameters:

type:string

description:parameterstodefinethebehaviourofthescheduler

4.9COMPUTEMANAGEMENT-VIRTUALMACHINES

Thissectionsummarizesabasicinterfacespecificationofvirtualmachines.

4.9.1Image

To execute virtual machines, we need an image that specifies the details of the operatingsystem.

4.9.1.1SchemaImage

Reference:☁�

Property Type Descriptionname string Auniquenameoftheimagecloud string Thenameofthecloudlabel string Alabelthatcanbedefinedbytheuserfortheimagedescription string AdescriptionfortheimageosType string TheOStypeoftheimageosVersion string TheOSversionoftheimagestatus string Thestatusoftheimageprogress integer Theloadingprogresspercentageoftheimagevisibility string Thevisibilityoftheimagerequirement Requirements Minimumrequirementtoruntheimage

4.9.1.2SchemaRequirements

Page 119: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Reference:☁�

Property Type Descriptionsize integer Minimumdisksizeinbytesrequiredfortheimageram integer Minimumramsizeinbytestoruntheimagecpu string CPUrequiredtoruntheimagecores integer Minimumnumberofcores

4.9.1.3Paths

HTTP Path Summaryget /image/{cloud} Returnsalistofimagesforthecloudget /image/{cloud}/{name} Returnsthenamedimageput /image/{cloud}/{name} Addaimagedelete /image/{cloud}/{name} Deletesthenamedimage

4.9.1.3.1/image/{cloud}

4.9.1.3.1.1GET/image/{cloud}

Returnsalistofallimages

Responses

Code Description Schema200 Thelistofimages array[Image]401 Notauthorized String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

4.9.1.3.2/image/{cloud}/{name}

4.9.1.3.2.1GET/image/{cloud}/{name}

Returnsaimagebyname

Page 120: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Returningtheinformationoftheimage Image401 Notauthorized String404 Thenamedimagecouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameoftheimage True String

4.9.1.3.2.2PUT/image/{cloud}/{name}

Setsthenamedimage

Responses

Code Description Schema200 Imageupdatedorcreated String401 Notauthorized String404 Thenamedimagecouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

RequestBody

Locatedin Description Required SchemaBody Theimagetoaddormodify True Image

4.9.1.3.2.3DELETE/image/{cloud}/{name}

Deletesaimagebyname

Page 121: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedimagecouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameoftheimage True String

4.9.1.4image.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Image

description:|-

Toexecutevirtualmachines,weneedanimagethatspecifiesthe

detailsoftheoperatingsystem.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/image/{cloud}:

get:

tags:

-Image

summary:Returnsalistofimagesforthecloud

description:Returnsalistofallimages

operationId:cloudmesh.image.list

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

responses:

'200':

description:Thelistofimages

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Image'

'401':

description:Notauthorized

/image/{cloud}/{name}:

get:

tags:

-Image

summary:Returnsthenamedimage

description:Returnsaimagebyname

operationId:cloudmesh.image.find_by_name

Page 122: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

parameters:

-name:cloud

in:path

description:Thenameofthecloud

required:true

schema:

type:string

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheimage

responses:

'200':

description:Returningtheinformationoftheimage

content:

application/json:

schema:

$ref:'#/components/schemas/Image'

'401':

description:Notauthorized

'404':

description:Thenamedimagecouldnotbefound

put:

tags:

-Image

summary:Addaimage

description:Setsthenamedimage

operationId:cloudmesh.image.add

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

requestBody:

description:Theimagetoaddormodify

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Image'

responses:

'200':

description:Imageupdatedorcreated

'401':

description:Notauthorized

'404':

description:Thenamedimagecouldnotbefound

delete:

tags:

-Image

summary:Deletesthenamedimage

description:Deletesaimagebyname

operationId:cloudmesh.image.delete_by_name

parameters:

-name:cloud

description:Thenameofthecloud

in:path

required:true

schema:

type:string

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheimage

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedimagecouldnotbefound

components:

schemas:

Image:

type:object

properties:

name:

type:string

description:Auniquenameoftheimage

cloud:

type:string

description:Thenameofthecloud

label:

Page 123: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

description:Alabelthatcanbedefinedbytheuserfortheimage

description:

type:string

description:Adescriptionfortheimage

osType:

type:string

description:TheOStypeoftheimage

osVersion:

type:string

description:TheOSversionoftheimage

status:

type:string

description:Thestatusoftheimage

progress:

type:integer

description:Theloadingprogresspercentageoftheimage

visibility:

description:Thevisibilityoftheimage

type:string

requirement:

$ref:"#/components/schemas/Requirements"

description:Minimumrequirementtoruntheimage

Requirements:

type:object

properties:

size:

type:integer

description:Minimumdisksizeinbytesrequiredfortheimage

ram:

type:integer

description:Minimumramsizeinbytestoruntheimage

cpu:

type:string

description:CPUrequiredtoruntheimage

cores:

type:integer

description:Minimumnumberofcores

4.9.2Flavor

The flavor specifies elementary information about a virtualmachine or compute node. Thisinformationincludesname,id,label,ramsize,swapsize,diskspace,availabilityofephemeraldisk,availablebandwidth,pricevalue,cloudname.Flavorsandthecorrespondinginformationareessentialtosizeavirtualclusterappropriately.

4.9.2.1SchemaFlavor

Reference:☁�

Property Type Descriptionname string Nameoftheflavorid string Theidoftheflavorforthenamedcloudlabel string Alabelthatausercansetforthisflavordescription string Adescriptionfortheflavorram integer NumberofbytesusedfortheimageinRAMswap integer NumberofbytesusedfortheimageinSWAPdisk integer Numberofbytesusedforthediskephemeral_disk boolean Specifieswhethertheflavorfeaturesanephemeraldiskbandwidth integer Bandwidthofthenode

Page 124: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

price number Pricefortheflavorcloud string Nameofthecloudthisflavorisusedin

4.9.2.2Paths

HTTP Path Summaryget /flavor/{cloud} Returnsalistofflavorsforthecloudget /flavor/{cloud}/{name} Returnsthenamedflavorput /flavor/{cloud}/{name} Addaflavordelete /flavor/{cloud}/{name} Deletesthenamedflavor

4.9.2.2.1/flavor/{cloud}

4.9.2.2.1.1GET/flavor/{cloud}

Returnsalistofallflavors

Responses

Code Description Schema200 Thelistofflavors array[Flavor]401 Notauthorized String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

4.9.2.2.2/flavor/{cloud}/{name}

4.9.2.2.2.1GET/flavor/{cloud}/{name}

Returnsaflavorbyname

Responses

Code Description Schema200 Returningtheinformationoftheflavor Flavor

Page 125: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

401 Notauthorized String404 Thenamedflavorcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameoftheflavor True String

4.9.2.2.2.2PUT/flavor/{cloud}/{name}

Setsthenamedflavor

Responses

Code Description Schema200 Flavorupdated String401 Notauthorized String404 Thenamedflavorcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

RequestBody

Locatedin Description Required SchemaBody Theflavortoaddormodify True Flavor

4.9.2.2.2.3DELETE/flavor/{cloud}/{name}

Deletesaflavorbyname

Responses

Code Description Schema200 Deletionsuccessful String

Page 126: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

401 Notauthorized String404 Thenamedflavorcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameoftheflavor True String

4.9.2.3flavor.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Flavor

description:|-

Theflavorspecifieselementaryinformationaboutavirtualmachine

orcomputenode.Thisinformationincludesname,id,label,ramsize,

swapsize,diskspace,availabilityofephemeraldisk,available

bandwidth,pricevalue,cloudname.Flavorsandthecorresponding

informationareessentialtosizea

virtualclusterappropriately.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/flavor/{cloud}:

get:

tags:

-Flavor

summary:Returnsalistofflavorsforthecloud

description:Returnsalistofallflavors

operationId:cloudmesh.flavor.list

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

responses:

'200':

description:Thelistofflavors

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Flavor'

'401':

description:Notauthorized

/flavor/{cloud}/{name}:

get:

tags:

-Flavor

summary:Returnsthenamedflavor

description:Returnsaflavorbyname

operationId:cloudmesh.flavor.find_by_name

parameters:

-name:cloud

in:path

description:Thenameofthecloud

required:true

schema:

Page 127: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheflavor

responses:

'200':

description:Returningtheinformationoftheflavor

content:

application/json:

schema:

$ref:'#/components/schemas/Flavor'

'401':

description:Notauthorized

'404':

description:Thenamedflavorcouldnotbefound

put:

tags:

-Flavor

summary:Addaflavor

description:Setsthenamedflavor

operationId:cloudmesh.flavor.add

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

requestBody:

description:Theflavortoaddormodify

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Flavor'

responses:

'200':

description:Flavorupdated

'401':

description:Notauthorized

'404':

description:Thenamedflavorcouldnotbefound

delete:

tags:

-Flavor

summary:Deletesthenamedflavor

description:Deletesaflavorbyname

operationId:cloudmesh.flavor.delete_by_name

parameters:

-name:cloud

description:Thenameofthecloud

in:path

required:true

schema:

type:string

-name:name

in:path

required:true

schema:

type:string

description:Thenameoftheflavor

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedflavorcouldnotbefound

components:

schemas:

Flavor:

type:object

description:Theflavor

properties:

name:

type:string

description:Nameoftheflavor

id:

type:string

description:Theidoftheflavorforthenamedcloud

label:

type:string

description:Alabelthatausercansetforthisflavor

description:

type:string

description:Adescriptionfortheflavor

Page 128: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

ram:

type:integer

description:NumberofbytesusedfortheimageinRAM

swap:

type:integer

description:NumberofbytesusedfortheimageinSWAP

disk:

type:integer

description:Numberofbytesusedforthedisk

ephemeral_disk:

type:boolean

description:Specifieswhethertheflavorfeaturesanephemeraldisk

bandwidth:

type:integer

description:Bandwidthofthenode

price:

type:number

description:Pricefortheflavor

cloud:

type:string

description:Nameofthecloudthisflavorisusedin

4.9.3VirtualMachine

Vmisusedtomanagevirtualmachines.

4.9.3.1SchemaVm

Reference:☁�

Property Type Descriptionprovider string Nameoftheprovidername string theuniquenameofthevirtualmachineimage string theimagenameforthevirtualmachineflavor string theflavornameforthevirtualmachineregion string anoptionalregionstate string Thestateofthevirtualmachineprivate_ips string TheprivateIPspublic_ips string ThepublicIPSmetadata string Themetadatapassedalongtothevirtualmachine

4.9.3.2Paths

HTTP Path Summaryget /vm/{cloud} Returnsalistofvirtualmachinesforthecloudget /vm/{cloud}/{name} Returnsthenamedvirtualmachineput /vm/{cloud}/{name} Addavirtualmachinedelete /vm/{cloud}/{name} Deletesthenamedvirtualmachine

Page 129: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.9.3.2.1/vm/{cloud}

4.9.3.2.1.1GET/vm/{cloud}

Returnsalistofallvirtualmachines

Responses

Code Description Schema200 Thelistofvirtualmachines array[Vm]400 NoVmfound String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

4.9.3.2.2/vm/{cloud}/{name}

4.9.3.2.2.1GET/vm/{cloud}/{name}

Returnsavirtualmachinebyname

Responses

Code Description Schema200 Returningtheinformationofthevirtualmachine Vm400 Errorupdatingvirtualmachine String401 Notauthorized String404 Thenamedvirtualmachineorcloudcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameofthevirtualmachine True String

4.9.3.2.2.2PUT/vm/{cloud}/{name}

Page 130: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Setsthenamedvirtualmachine

Responses

Code Description Schema200 Vmupdated String400 Errorupdatingvirtualmachine String401 Notauthorized String404 Thenamedvirtualmachineorcloudcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True String

RequestBody

Locatedin Description Required SchemaBody Thevirtualmachinetoaddormodify True Vm

4.9.3.2.2.3DELETE/vm/{cloud}/{name}

Deletesavirtualmachinebyname

Responses

Code Description Schema200 Deletionsuccessful String400 Errorupdatingvirtualmachine String401 Notauthorized String404 Thenamedvirtualmachineorcloudcouldnotbefound String

Parameters

Name Locatedin Description Required Schemacloud path Thenameofthecloud True Stringname path Thenameofthevirtualmachine True String

Page 131: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.9.3.3vm.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:VirtualMachine

description:|-

Vmisusedtomanagevirtualmachines.

termsOfService:https://github.com/cloudmesh-community/nist/blob/master/LICENSE.txt

contact:

name:NISTBDRAInterfaceSubgroupService

url:https://cloudmesh-community.github.io/nist/spec/

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/vm/{cloud}:

get:

tags:

-Vm

summary:Returnsalistofvirtualmachinesforthecloud

description:Returnsalistofallvirtualmachines

operationId:cloudmesh.vm.list

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

responses:

'200':

description:Thelistofvirtualmachines

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Vm'

'400':

description:NoVmfound

'401':

description:Notauthorized

/vm/{cloud}/{name}:

get:

tags:

-Vm

summary:Returnsthenamedvirtualmachine

description:Returnsavirtualmachinebyname

operationId:cloudmesh.vm.find_by_name

parameters:

-name:cloud

in:path

description:Thenameofthecloud

required:true

schema:

type:string

-name:name

in:path

description:Thenameofthevirtualmachine

required:true

schema:

type:string

responses:

'200':

description:Returningtheinformationofthevirtualmachine

content:

application/json:

schema:

$ref:'#/components/schemas/Vm'

'400':

description:Errorupdatingvirtualmachine

'401':

description:Notauthorized

'404':

description:Thenamedvirtualmachineorcloudcouldnotbefound

put:

tags:

-Vm

summary:Addavirtualmachine

description:Setsthenamedvirtualmachine

Page 132: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

operationId:cloudmesh.vm.add

parameters:

-name:cloud

in:path

required:true

schema:

type:string

description:Thenameofthecloud

requestBody:

description:Thevirtualmachinetoaddormodify

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Vm'

responses:

'200':

description:Vmupdated

'400':

description:Errorupdatingvirtualmachine

'401':

description:Notauthorized

'404':

description:Thenamedvirtualmachineorcloudcouldnotbefound

delete:

tags:

-Vm

summary:Deletesthenamedvirtualmachine

description:Deletesavirtualmachinebyname

operationId:cloudmesh.vm.delete_by_name

parameters:

-name:cloud

description:Thenameofthecloud

in:path

required:true

schema:

type:string

-name:name

in:path

description:Thenameofthevirtualmachine

required:true

schema:

type:string

responses:

'200':

description:Deletionsuccessful

'400':

description:Errorupdatingvirtualmachine

'401':

description:Notauthorized

'404':

description:Thenamedvirtualmachineorcloudcouldnotbefound

components:

schemas:

Vm:

type:object

properties:

provider:

type:string

description:Nameoftheprovider

name:

type:string

description:theuniquenameofthevirtualmachine

image:

type:string

description:theimagenameforthevirtualmachine

flavor:

type:string

description:theflavornameforthevirtualmachine

region:

type:string

description:anoptionalregion

state:

type:string

description:Thestateofthevirtualmachine

private_ips:

type:string

description:TheprivateIPs

public_ips:

type:string

description:ThepublicIPS

metadata:

type:string

description:Themetadatapassedalongtothevirtualmachine

4.9.4Secgroup

Page 133: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Asecuritygroupdefinestheincomingandoutgoingsecurityruleswhichcanthenbeassignedtoanode.Theconnectiontoandfromthenodewillbedeterminedbythesecuritygrouprules,in addition to any other possible rules applied on network devices or from the instance’sfirewall settings. A security group may have one or multiple rules and a node may beassociatedwithoneormoresecuritygroups.

4.9.4.1SchemaSecgroup

Reference:☁�

Property Type Descriptionname string Nameofthesecuritygroupdescription string Describeswhatthesecuritygroupisforrules array[SecGroupRule] ListofSecuritygrouprules

4.9.4.2SchemaSecGroupRule

Reference:☁�

Property Type Descriptionname string Uniquenameoftheruleingress boolean ThedefinedsecuritygroupruleisforingressifTrueegress boolean ThedefinedsecuritygroupruleisforegressifTrue

remote_group string NameofthegroupiftheruleisdefinedbygroupinsteadofIPrange

protocol string TheprotocolusedsuchasTCP,UDP,ICMPfrom_port integer Portrangestartingportto_port integer Portrangeendingportcidr string ThesourceordestinationnetworkinCIDRnotation,

4.9.4.3Paths

HTTP Path Summaryget /secgroup Returnsallsecuritygroupsget /secgroup/{name} Returnthesecuritygroupbynamepost /secgroup/{name} Createthenamedsecuritygroup

Getanexistingrulefromthespecified

Page 134: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

get /secgroup/{name}/rule/{rule} securitygroup

put /secgroup/{name}/rule/{rule} Createorupdatespecifiedsecuritygroup

delete /secgroup/{name}/rule/{rule} Deleteanexistingrulefromthespecifiedsecuritygroup

4.9.4.3.1/secgroup

4.9.4.3.1.1GET/secgroup

Returnsallsecuritygroups

Responses

Code Description Schema200 securitygroupinformation array[Secgroup]401 Notauthorized String

4.9.4.3.2/secgroup/{name}

4.9.4.3.2.1GET/secgroup/{name}

Returnthesecuritygroupbyname

Responses

Code Description Schema200 securitygroupinformation Secgroup401 Notauthorized String404 Thenamedsecuritygroupcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path nameofthesecuritygroup True String

4.9.4.3.2.2POST/secgroup/{name}

Page 135: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Createanewnamedsecuritygroup

Responses

Code Description Schema201 Created String400 Thegroupcouldnotbecreated String401 Notauthorized String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthesecuritygrouptocreate True String

4.9.4.3.3/secgroup/{name}/rule/{rule}

4.9.4.3.3.1GET/secgroup/{name}/rule/{rule}

Createanewruleinsecuritygroup

Responses

Code Description Schema200 Thesecuritygroupruledefinitioninfo Secgrouprule401 Notauthorized String404 Thenamedsecuritygrouporrolecouldnotbefound String

Parameters

Name Locatedin Description Required Schema

name path Thenamedofthesecuritygroupfromwhichtherulewillbedeleted True String

rule path Theruletobeadded True String

4.9.4.3.3.2PUT/secgroup/{name}/rule/{rule}

Createanewruleinsecuritygroup

Page 136: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Created String401 Notauthorized String404 Thenamedsecuritygrouporrolecouldnotbefound String

Parameters

Name Locatedin Description Required Schema

name path Thenameofthenewsecuritygrouptocreate True String

RequestBody

Locatedin Description Required SchemaBody Thenewsecuritygroupruletocreate True SecGroupRule

4.9.4.3.3.3DELETE/secgroup/{name}/rule/{rule}

Createanewruleinsecuritygroup

Responses

Code Description Schema200 Deletedsucessfully String401 Notauthorized String404 Thenamedsecuritygrouporrolecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenamedsecgroup True Stringrule path Thesecgrouprule True String

4.9.4.4secgroup.yaml

Page 137: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Secgroup

description:|-

Asecuritygroupdefinestheincomingandoutgoingsecurityrules

whichcanthenbeassignedtoanode.Theconnectiontoandfromthenode

willbedeterminedbythesecuritygrouprules,inadditiontoanyother

possiblerulesappliedonnetworkdevicesorfromtheinstance'sfirewall

settings.Asecuritygroupmayhaveoneormultiplerulesandanodemaybe

associatedwithoneormoresecuritygroups.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/secgroup:

get:

tags:

-Securitygroup

summary:Returnsallsecuritygroups

description:Returnsallsecuritygroups

operationId:cloudmesh.secgroup.get

responses:

'200':

description:securitygroupinformation

content:

application/json:

schema:

type:array

items:

$ref:"#/components/schemas/Secgroup"

'401':

description:Notauthorized

/secgroup/{name}:

get:

tags:

-Securitygroup

summary:Returnthesecuritygroupbyname

description:Returnthesecuritygroupbyname

operationId:cloudmesh.secgroup.get_by_name

parameters:

-name:name

description:nameofthesecuritygroup

in:path

required:true

schema:

type:string

responses:

'200':

description:securitygroupinformation

content:

application/json:

schema:

$ref:"#/components/schemas/Secgroup"

'401':

description:Notauthorized

'404':

description:Thenamedsecuritygroupcouldnotbefound

post:

tags:

-Securitygroup

summary:Createthenamedsecuritygroup

description:Createanewnamedsecuritygroup

operationId:cloudmesh.secgroup.create

parameters:

-in:path

name:name

required:true

description:Thenameofthesecuritygrouptocreate

schema:

type:string

responses:

'201':

description:Created

'400':

description:Thegroupcouldnotbecreated

'401':

description:Notauthorized

/secgroup/{name}/rule/{rule}:

get:

Page 138: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

tags:

-Securitygroup

summary:Getanexistingrulefromthespecifiedsecuritygroup

description:Createanewruleinsecuritygroup

operationId:cloudmesh.secgroup.get_rule

parameters:

-in:path

name:name

required:true

description:Thenamedofthesecuritygroupfromwhichtherulewillbedeleted

schema:

type:string

-in:path

name:rule

required:true

description:Theruletobeadded

schema:

type:string

responses:

'200':

description:Thesecuritygroupruledefinitioninfo

content:

application/json:

schema:

$ref:"#/components/schemas/SecGroupRule"

'401':

description:Notauthorized

'404':

description:Thenamedsecuritygrouporrolecouldnotbefound

put:

tags:

-Securitygroup

summary:Createorupdatespecifiedsecuritygroup

description:Createanewruleinsecuritygroup

operationId:cloudmesh.secgroup.add_rule

parameters:

-in:path

name:name

required:true

description:Thenameofthenewsecuritygrouptocreate

schema:

type:string

requestBody:

description:Thenewsecuritygroupruletocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/SecGroupRule'

responses:

'200':

description:Created

'401':

description:Notauthorized

'404':

description:Thenamedsecuritygrouporrolecouldnotbefound

delete:

tags:

-Securitygroup

summary:Deleteanexistingrulefromthespecifiedsecuritygroup

description:Createanewruleinsecuritygroup

operationId:cloudmesh.secgroup.delete_rule

parameters:

-in:path

name:name

required:true

description:Thenamedsecgroup

schema:

type:string

-in:path

name:rule

required:true

description:Thesecgrouprule

schema:

type:string

responses:

'200':

description:Deletedsucessfully

'401':

description:Notauthorized

'404':

description:Thenamedsecuritygrouporrolecouldnotbefound

components:

schemas:

Secgroup:

type:object

description:thesecuritygroupobject

properties:

name:

Page 139: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

type:string

description:Nameofthesecuritygroup

description:

type:string

description:Describeswhatthesecuritygroupisfor

rules:

type:array

description:ListofSecuritygrouprules

items:

$ref:"#/components/schemas/SecGroupRule"

SecGroupRule:

type:object

description:securitygrouprule

properties:

name:

type:string

description:Uniquenameoftherule

ingress:

type:boolean

description:ThedefinedsecuritygroupruleisforingressifTrue

egress:

type:boolean

description:ThedefinedsecuritygroupruleisforegressifTrue

remote_group:

type:string

description:Nameofthegroupiftheruleisdefinedbygroup

insteadofIPrange

protocol:

type:string

description:TheprotocolusedsuchasTCP,UDP,ICMP

example:TCP

from_port:

type:integer

description:Portrangestartingport

to_port:

type:integer

description:Portrangeendingport

cidr:

type:string

description:ThesourceordestinationnetworkinCIDRnotation,

example:129.79.0.0/16

4.9.5Nic

AresourcestoreNetworkInterfaceController(NIC)information.

4.9.5.1SchemaNic

Reference:☁�

Property Type Descriptionname string Nameofthenetworkinterfacecontrollerkind string Kindofthenetworkinterfacecontroller(wifi,WAN,…)mac string Themacaddressip string TheIPaddressmask string Thenetworkmaskbroadcast string Thebroadcastaddressgateway string Thegatewayaddressmtu integer TheMTUoftheNICbandwidth integer Thebandwidthinbps

Page 140: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.9.5.2Paths

HTTP Path Summaryget /nic Returnsalistofnetworkinterfacecontrollersget /nic/{name} Returnsthenamednetworkinterfacecontrollerput /nic/{name} Setanetworkinterfacecontrollerdelete /nic/{name} Deletesthenamednetworkinterfacecontroller

4.9.5.2.1/nic

4.9.5.2.1.1GET/nic

Returnsalistofallnetworkinterfacecontrollers

Responses

Code Description Schema200 Thelistofnetworkinterfacecontrollers array[Nic]401 Notauthorized String

4.9.5.2.2/nic/{name}

4.9.5.2.2.1GET/nic/{name}

Returnsanetworkinterfacecontrollerbyname

Responses

Code Description Schema200 Returningtheinformationofthenetworkinterfacecontroller Nic401 Notauthorized String404 Thenamednetworkinterfacecontrollercouldnotbefound String

Parameters

Name Locatedin Description Required Schema

name path Thenameofthenetworkinterfacecontroller True String

Page 141: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.9.5.2.2.2PUT/nic/{name}

Setsthenamednetworkinterfacecontroller

Responses

Code Description Schema200 Nicupdated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewnictocreateorupdate True Nic

4.9.5.2.2.3DELETE/nic/{name}

Deletesanetworkinterfacecontrollerbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamednetworkinterfacecontrollercouldnotbefound String

Parameters

Name Locatedin Description Required Schema

name path Thenameofthenetworkinterfacecontroller True String

4.9.5.3nic.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Nic

description:|-

AresourcestoreNetworkInterfaceController(NIC)information.

Page 142: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/nic:

get:

tags:

-Nic

summary:Returnsalistofnetworkinterfacecontrollers

description:Returnsalistofallnetworkinterfacecontrollers

operationId:cloudmesh.nic.list

responses:

'200':

description:Thelistofnetworkinterfacecontrollers

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Nic'

'401':

description:Notauthorized

/nic/{name}:

get:

tags:

-Nic

summary:Returnsthenamednetworkinterfacecontroller

description:Returnsanetworkinterfacecontrollerbyname

operationId:cloudmesh.nic.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkinterfacecontroller

responses:

'200':

description:Returningtheinformationofthenetworkinterfacecontroller

content:

application/json:

schema:

$ref:'#/components/schemas/Nic'

'401':

description:Notauthorized

'404':

description:Thenamednetworkinterfacecontrollercouldnotbefound

put:

tags:

-Nic

summary:Setanetworkinterfacecontroller

description:Setsthenamednetworkinterfacecontroller

operationId:cloudmesh.nic.add

requestBody:

description:Thenewnictocreateorupdate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Nic'

responses:

'200':

description:Nicupdated

'401':

description:Notauthorized

delete:

tags:

-Nic

summary:Deletesthenamednetworkinterfacecontroller

description:Deletesanetworkinterfacecontrollerbyname

operationId:cloudmesh.nic.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthenetworkinterfacecontroller

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

Page 143: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Thenamednetworkinterfacecontrollercouldnotbefound

components:

schemas:

Nic:

type:object

description:Thenetworkinterfacecontroller

properties:

name:

type:string

description:Nameofthenetworkinterfacecontroller

kind:

type:string

description:Kindofthenetworkinterfacecontroller(wifi,WAN,...)

mac:

type:string

description:Themacaddress

ip:

type:string

description:TheIPaddress

mask:

type:string

description:Thenetworkmask

broadcast:

type:string

description:Thebroadcastaddress

gateway:

type:string

description:Thegatewayaddress

mtu:

type:integer

description:TheMTUoftheNIC

bandwidth:

type:integer

description:Thebandwidthinbps

4.10COMPUTEMANAGEMENT-CONTAINERS

4.10.1Containers

Numerousdifferentcontainersare likely tobecreatedandhandlingthembecomesmoreandmore time consuming as their number increases. This service helps to solve that issue bystoringcontainersandtheircorrespondinginformation.

4.10.1.1SchemaContainer

Reference:☁�

Property Type Descriptionname string Nameofthecontainerversion string Versionofthecontainerlabel string Labelofthecontainertype string Typeofthecontainerdefinition string DefinitionormanifestofthecontainerimgURI string URIofthecontainertags array[string] Tagsofthecontainer

4.10.1.2Paths

Page 144: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

HTTP Path Summaryget /container Returnsalistofcontainersget /container/{name} Returnsthenamedcontainerput /container/{name} Setancontainerdelete /container/{name} Deletesthenamedcontainer

4.10.1.2.1/container

4.10.1.2.1.1GET/container

Returnsalistofallcontainers

Responses

Code Description Schema200 Thelistofcontainerses array[Container]401 Notauthorized String

4.10.1.2.2/container/{name}

4.10.1.2.2.1GET/container/{name}

Returnsancontainerbyname

Responses

Code Description Schema200 Returningtheinformationofthecontainer Container400 NoContainerfound String401 Notauthorized String404 Thenamedcontainercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthecontainer True String

4.10.1.2.2.2PUT/container/{name}

Page 145: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Setsthenamedcontainer

Responses

Code Description Schema200 Containerupdated String401 Notauthorized String400 Errorupdatingcontainer String

RequestBody

Locatedin Description Required SchemaBody Thenewcontainertocreate True Container

4.10.1.2.2.3DELETE/container/{name}

Deletesancontainerbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedcontainercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthecontainer True String

4.10.1.3containers.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Containers

description:|-

Numerousdifferentcontainersarelikelytobecreatedandhandlingthem

becomesmoreandmoretimeconsumingastheirnumberincreases.Thisservice

helpstosolvethatissuebystoringcontainersandtheircorresponding

information.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

Page 146: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/container:

get:

tags:

-Container

summary:Returnsalistofcontainers

description:Returnsalistofallcontainers

operationId:cloudmesh.container.list

responses:

'200':

description:Thelistofcontainerses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Container'

'401':

description:Notauthorized

/container/{name}:

get:

tags:

-Container

summary:Returnsthenamedcontainer

description:Returnsancontainerbyname

operationId:cloudmesh.container.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthecontainer

responses:

'200':

description:Returningtheinformationofthecontainer

content:

application/json:

schema:

$ref:'#/components/schemas/Container'

'400':

description:NoContainerfound

'401':

description:Notauthorized

'404':

description:Thenamedcontainercouldnotbefound

put:

tags:

-Container

summary:Setancontainer

description:Setsthenamedcontainer

operationId:cloudmesh.container.add

requestBody:

description:Thenewcontainertocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Container'

responses:

'200':

description:Containerupdated

'401':

description:Notauthorized

'400':

description:Errorupdatingcontainer

delete:

tags:

-Container

summary:Deletesthenamedcontainer

description:Deletesancontainerbyname

operationId:cloudmesh.container.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthecontainer

responses:

'200':

description:Deletionsuccessful

Page 147: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

'401':

description:Notauthorized

'404':

description:Thenamedcontainercouldnotbefound

components:

schemas:

Container:

type:object

description:Arecordrepresentingacontainer

properties:

name:

type:string

description:Nameofthecontainer

version:

type:string

description:Versionofthecontainer

label:

type:string

description:Labelofthecontainer

type:

type:string

description:Typeofthecontainer

definition:

type:string

description:Definitionormanifestofthecontainer

imgURI:

type:string

description:URIofthecontainer

tags:

type:array

description:Tagsofthecontainer

items:

type:string

4.11COMPUTEMANAGEMENT-MAPREDUCE

4.11.1MapReduce

Aservicetostoretheinformationofamapreducedeploymentdefinition.AlloftheattributesarestoredasStrings.

4.11.1.1SchemaMap

Reference:☁�

Property Type Descriptionname string Thenameofthemapfunctionkind string Thekindinwhichthespecificationisprovidedcontent string Thekindinwhichthespecificationisprovided

4.11.1.2SchemaReduce

Reference:☁�

Property Type Descriptionname string Thenameofthereducefunctionkind string Thekindinwhichthespecificationisprovided

Page 148: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

content string Thekindinwhichthespecificationisprovided

4.11.1.3SchemaData

Reference:☁�

Property Type Descriptionname string Thenameofthedatacontent string Thecontentoftehedata

4.11.1.4Paths

HTTP Path Summary

get /mapreduce Returnsthedataidentifiedbythemapreduceresource

get /mapreduce/{name} Returnsthedataidentifiedbythemapandfunction

put /mapreduce/map/{name} Createorupdatethemapfunction

get /mapreduce/map/{name} Returnsthedataidentifiedbythemapfunction

put /mapreduce/reduce/{name} Createorupdatethereducefunction

get /mapreduce/reduce/{name} Returnsthedataidentifiedbythereducefunction

4.11.1.4.1/mapreduce

4.11.1.4.1.1GET/mapreduce

Returnsthedataidentifiedbythemapreduceresource

Responses

Code Description Schema200 mapreducenames array[String]401 Notauthorized String

4.11.1.4.2/mapreduce/{name}

4.11.1.4.2.1GET/mapreduce/{name}

Page 149: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returnsthedataidentifiedbythereducefunction.

Responses

Code Description Schema200 Thedataidentifiedbyreduce array[Data]401 Notauthorized String404 Theresourcecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefunction True String

4.11.1.4.3/mapreduce/map/{name}

4.11.1.4.3.1PUT/mapreduce/map/{name}

Createorupdatethemapfunction

Responses

Code Description Schema200 Themapfunctionwascreatedorupdated String401 Notauthorized String404 Theresourcecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefunction True String

RequestBody

Locatedin Description Required SchemaBody Thenewdefaulttocreate True Map

4.11.1.4.3.2GET/mapreduce/map/{name}

Page 150: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returnsthedataidentifiedbythemapfunction

Responses

Code Description Schema200 Thedataidentifiedbymap array[Data]401 Notauthorized String404 Theresourcecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefunction True String

4.11.1.4.4/mapreduce/reduce/{name}

4.11.1.4.4.1PUT/mapreduce/reduce/{name}

Createorupdatethereducefunction

Responses

Code Description Schema200 Thereducefunctionwascreatedorupdated String401 Notauthorized String404 Theresourcecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefunction True String

RequestBody

Locatedin Description Required SchemaBody Thenewdefaulttocreate True Reduce

4.11.1.4.4.2GET/mapreduce/reduce/{name}

Page 151: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returnsthedataidentifiedbythereducefunction

Responses

Code Description Schema200 Thedataidentifiedbyreduce array[Data]401 Notauthorized String404 Theresourcecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefunction True String

4.11.1.5mapreduce.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:MapReduce

description:|-

Aservicetostoretheinformationofamapreducedeploymentdefinition.

AlloftheattributesarestoredasStrings.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/mapreduce:

get:

tags:

-mapreduce

summary:Returnsthedataidentifiedbythemapreduceresource

description:Returnsthedataidentifiedbythemapreduceresource

operationId:cloudmesh.mapreduce.list

responses:

'200':

description:mapreducenames

content:

application/json:

schema:

type:array

items:

type:string

'401':

description:Notauthorized

/mapreduce/{name}:

get:

tags:

-mapreduce

summary:Returnsthedataidentifiedbythemapandfunction

description:Returnsthedataidentifiedbythereducefunction.

operationId:cloudmesh.mapreduce.get

parameters:

-name:name

in:path

required:true

schema:

type:string

Page 152: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Thenameofthefunction

responses:

'200':

description:Thedataidentifiedbyreduce

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Data'

'401':

description:Notauthorized

'404':

description:Theresourcecouldnotbefound

/mapreduce/map/{name}:

put:

tags:

-mapreduce

summary:Createorupdatethemapfunction

description:Createorupdatethemapfunction

operationId:cloudmesh.mapreduce.map.put

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefunction

requestBody:

description:Thenewdefaulttocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Map'

responses:

'200':

description:Themapfunctionwascreatedorupdated

'401':

description:Notauthorized

'404':

description:Theresourcecouldnotbefound

get:

tags:

-mapreduce

summary:Returnsthedataidentifiedbythemapfunction

description:Returnsthedataidentifiedbythemapfunction

operationId:cloudmesh.mapreduce.map.get

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefunction

responses:

'200':

description:Thedataidentifiedbymap

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Data'

'401':

description:Notauthorized

'404':

description:Theresourcecouldnotbefound

/mapreduce/reduce/{name}:

put:

tags:

-mapreduce

summary:Createorupdatethereducefunction

description:Createorupdatethereducefunction

operationId:cloudmesh.mapreduce.reduce.put

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefunction

requestBody:

description:Thenewdefaulttocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Reduce'

Page 153: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

responses:

'200':

description:Thereducefunctionwascreatedorupdated

'401':

description:Notauthorized

'404':

description:Theresourcecouldnotbefound

get:

tags:

-mapreduce

summary:Returnsthedataidentifiedbythereducefunction

description:Returnsthedataidentifiedbythereducefunction

operationId:cloudmesh.mapreduce.reduce.get

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefunction

responses:

'200':

description:Thedataidentifiedbyreduce

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Data'

'401':

description:Notauthorized

'404':

description:Theresourcecouldnotbefound

components:

schemas:

Map:

type:object

description:Thespecificationofthemapfunction

properties:

name:

type:string

description:Thenameofthemapfunction

kind:

type:string

description:Thekindinwhichthespecificationisprovided

content:

type:string

description:Thekindinwhichthespecificationisprovided

Reduce:

type:object

description:Thespecificationofthereducefunction

properties:

name:

type:string

description:Thenameofthereducefunction

kind:

type:string

description:Thekindinwhichthespecificationisprovided

content:

type:string

description:Thekindinwhichthespecificationisprovided

Data:

type:object

description:Thespecificationofthefunction

properties:

name:

type:string

description:Thenameofthedata

content:

type:string

description:Thecontentoftehedata

4.12COMPUTEMANAGEMENT-FUNCTIONS

4.12.1Microservice

Aspartofmicroservices,afunctionwithparametersthatcanbeinvokedhasbeendefined.

4.12.1.1SchemaMicroservice

Page 154: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Reference:☁�

Property Type Descriptionname string Nameofthemicroserviceendpoint string Theendpointofthemicroservicefunction string Thefunctionthemicroservicerepresentsdescription Thedescriptionofthemicroservice

4.12.1.2Paths

HTTP Path Summaryget /microservice Returnsalistofmicroserviceesget /microservice/{name} Returnsthenamedmicroserviceput /microservice/{name} Setanmicroservicedelete /microservice/{name} Deletesthenamedmicroservice

4.12.1.2.1/microservice

4.12.1.2.1.1GET/microservice

Returnsalistofallmicroservicees

Responses

Code Description Schema200 Thelistofmicroserviceses array[Microservice]401 Notauthorized String

4.12.1.2.2/microservice/{name}

4.12.1.2.2.1GET/microservice/{name}

Returnsthenamedmicroservice

Responses

Code Description Schema200 Returnsthemicroservice Microservice

Page 155: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

401 Notauthorized String404 Thenamedmicroservicecouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthemicroservice True String

4.12.1.2.2.2PUT/microservice/{name}

Setsthenamedmicroservice

Responses

Code Description Schema200 Microserviceupdated String400 Errorupdatingmicroservice String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewmicroservicetocreate True Microservice

4.12.1.2.2.3DELETE/microservice/{name}

Deletesanmicroservicebyname

Responses

Code Description Schema200 Deletionsuccessful String400 Errordeletingmicroservice String401 Notauthorized String404 Thenamedmicroservicecouldnotbefound String

Parameters

Page 156: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Name Locatedin Description Required Schemaname path Thenameofthemicroservice True String

4.12.1.3microservice.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Microservice

description:|-

Aspartofmicroservices,afunctionwithparametersthatcanbe

invokedhasbeendefined.

termsOfService:"https://github.com/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/microservice:

get:

tags:

-Microservice

summary:Returnsalistofmicroservicees

description:Returnsalistofallmicroservicees

operationId:cloudmesh.microservice.list

responses:

'200':

description:Thelistofmicroserviceses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Microservice'

'401':

description:Notauthorized

/microservice/{name}:

get:

tags:

-Microservice

summary:Returnsthenamedmicroservice

description:Returnsthenamedmicroservice

operationId:cloudmesh.microservice.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthemicroservice

responses:

'200':

description:Returnsthemicroservice

content:

application/json:

schema:

$ref:'#/components/schemas/Microservice'

'401':

description:Notauthorized

'404':

description:Thenamedmicroservicecouldnotbefound

put:

tags:

-Microservice

summary:Setanmicroservice

description:Setsthenamedmicroservice

operationId:cloudmesh.microservice.add

requestBody:

description:Thenewmicroservicetocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Microservice'

responses:

Page 157: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

'200':

description:Microserviceupdated

'400':

description:Errorupdatingmicroservice

'401':

description:Notauthorized

delete:

tags:

-Microservice

summary:Deletesthenamedmicroservice

description:Deletesanmicroservicebyname

operationId:cloudmesh.microservice.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthemicroservice

responses:

'200':

description:Deletionsuccessful

'400':

description:Errordeletingmicroservice

'401':

description:Notauthorized

'404':

description:Thenamedmicroservicecouldnotbefound

components:

schemas:

Microservice:

type:object

description:Themicroservice

properties:

name:

type:string

description:Nameofthemicroservice

endpoint:

type:string

description:Theendpointofthemicroservice

function:

type:string

description:Thefunctionthemicroservicerepresents

description:

typestring:

description:Thedescriptionofthemicroservice

4.13RESERVATION

4.13.1Reservation

Someservicesmayconsumeaconsiderableamountofresources,necessitatingthereservationofresources.

4.13.1.1SchemaReservation

Reference:☁�

Property Type Descriptionname string Nameofthereservation

service string Thenameoftheserviceforwhichthereservationisapplied

description string Thedescriptionofthereservationstart string(date) Thestarttimeanddateend string(date) Theendtimeanddate

Page 158: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.13.1.2Paths

HTTP Path Summaryget /reservation Returnsalistofreservationsget /reservation/{name} Returnsthenamedreservationput /reservation/{name} Uploadsareservationtothelistofreservationsdelete /reservation/{name} Deletesthenamedreservation

4.13.1.2.1/reservation

4.13.1.2.1.1GET/reservation

Returnsalistofallreservations

Responses

Code Description Schema200 Thelistofreservations array[Reservation]400 NoReservationsfound String

4.13.1.2.2/reservation/{name}

4.13.1.2.2.1GET/reservation/{name}

Returnsanreservationbyname

Responses

Code Description Schema200 Returningtheinformationofthereservation Reservation400 Noreservationfound String401 Notauthorized String404 Thenamedreservationcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthereservation True String

Page 159: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.13.1.2.2.2PUT/reservation/{name}

Uploadsareservationtothelistofreservations

Responses

Code Description Schema200 Reservationupdated String400 Errorupdatingreservation String

RequestBody

Locatedin Description Required SchemaBody Thereservationtobeuploaded True Reservation

4.13.1.2.2.3DELETE/reservation/{name}

Deletesanreservationbyname

Responses

Code Description Schema200 Deletionsuccessful String400 Noreservationfound String401 Notauthorized String404 Thenamedreservationcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthereservation True String

4.13.1.3reservation.yaml

openapi:'3.0.2'

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Reservation

description:|-

Someservicesmayconsumeaconsiderableamountofresources,

necessitatingthereservationofresources.

Page 160: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

termsOfService:'https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt'

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/reservation:

get:

tags:

-Reservation

summary:Returnsalistofreservations

description:Returnsalistofallreservations

operationId:cloudmesh.reservation.list

responses:

'200':

description:Thelistofreservations

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Reservation'

'400':

description:NoReservationsfound

/reservation/{name}:

get:

tags:

-Reservation

summary:Returnsthenamedreservation

description:Returnsanreservationbyname

operationId:cloudmesh.reservation.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthereservation

responses:

'200':

description:Returningtheinformationofthereservation

content:

application/json:

schema:

$ref:'#/components/schemas/Reservation'

'400':

description:Noreservationfound

'401':

description:Notauthorized

'404':

description:Thenamedreservationcouldnotbefound

put:

tags:

-Reservation

summary:Uploadsareservationtothelistofreservations

description:Uploadsareservationtothelistofreservations

operationId:cloudmesh.reservation.add

requestBody:

description:Thereservationtobeuploaded

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Reservation'

responses:

'200':

description:Reservationupdated

'400':

description:Errorupdatingreservation

delete:

tags:

-Reservation

summary:Deletesthenamedreservation

description:Deletesanreservationbyname

operationId:cloudmesh.reservation.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthereservation

responses:

'200':

description:Deletionsuccessful

'400':

Page 161: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

description:Noreservationfound

'401':

description:Notauthorized

'404':

description:Thenamedreservationcouldnotbefound

components:

schemas:

Reservation:

type:object

description:Thereservation

properties:

name:

type:string

description:Nameofthereservation

service:

type:string

description:Thenameoftheserviceforwhichthereservationis

applied

description:

type:string

description:Thedescriptionofthereservation

start:

type:string

format:date

description:Thestarttimeanddate

end:

type:string

format:date

description:Theendtimeanddate

4.14DATASTREAMS

4.14.1Stream

Thestreamobjectdescribesadataflow,providinginformationabout therateandnumberofitems exchangedwhile issuing requests to the stream.A streammay return data items in aspecificformatthatisdefinedbythestream.

4.14.1.1SchemaStream

Reference:☁�

Property Type Descriptionname string Nameofthestreamformat string Formatofthestreamrate integer Therateofmessageslimit integer Thelimitofitemssendendpoint string Theendpointofthestreamprotocol string DThedefinitionoftheprotocolused

4.14.1.2Paths

HTTP Path Summaryget /stream Returnsalistofstreams

Page 162: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

get /stream/{name} Returnsthenamedstreamput /stream/{name} Setanstreamdelete /stream/{name} Deletesthenamedstream

4.14.1.2.1/stream

4.14.1.2.1.1GET/stream

Returnsalistofallstreams

Responses

Code Description Schema200 Thelistofstreamses array[Stream]400 NoStreamfound String401 Notauthorized String

4.14.1.2.2/stream/{name}

4.14.1.2.2.1GET/stream/{name}

Returnsanstreambyname

Responses

Code Description Schema200 Returningtheinformationofthestream Stream401 Notauthorized String404 Thenamedstreamcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthestream True String

4.14.1.2.2.2PUT/stream/{name}

Setsthenamedstream

Page 163: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Responses

Code Description Schema200 Streamupdated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewstreamtocreate True Stream

4.14.1.2.2.3DELETE/stream/{name}

Deletesanstreambyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedstreamcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthestream True String

4.14.1.3stream.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Stream

description:|-

Thestreamobjectdescribesadataflow,providinginformation

abouttherateandnumberofitemsexchangedwhileissuingrequests

tothestream.Astreammayreturndataitemsinaspecificformat

thatisdefinedbythestream.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

Page 164: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

paths:

/stream:

get:

tags:

-Stream

summary:Returnsalistofstreams

description:Returnsalistofallstreams

operationId:cloudmesh.stream.list

responses:

'200':

description:Thelistofstreamses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Stream'

'400':

description:NoStreamfound

'401':

description:Notauthorized

/stream/{name}:

get:

tags:

-Stream

summary:Returnsthenamedstream

description:Returnsanstreambyname

operationId:cloudmesh.stream.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthestream

responses:

'200':

description:Returningtheinformationofthestream

content:

application/json:

schema:

$ref:'#/components/schemas/Stream'

'401':

description:Notauthorized

'404':

description:Thenamedstreamcouldnotbefound

put:

tags:

-Stream

summary:Setanstream

description:Setsthenamedstream

operationId:cloudmesh.stream.add

requestBody:

description:Thenewstreamtocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Stream'

responses:

'200':

description:Streamupdated

'401':

description:Notauthorized

delete:

tags:

-Stream

summary:Deletesthenamedstream

description:Deletesanstreambyname

operationId:cloudmesh.stream.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthestream

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedstreamcouldnotbefound

components:

schemas:

Stream:

type:object

description:Thestream

properties:

Page 165: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

name:

type:string

description:Nameofthestream

format:

type:string

description:Formatofthestream

rate:

type:integer

description:Therateofmessages

limit:

type:integer

description:Thelimitofitemssend

endpoint:

type:string

description:Theendpointofthestream

protocol:

type:string

description:DThedefinitionoftheprotocolused

4.14.2Filter

Filters can operate on a variety of objects and reduce the information received based on asearchcriterion.

4.14.2.1SchemaFilter

Reference:☁�

Property Type Descriptionname string Nameofthefilterfunction string Thefunctionusedtofilterthedatainthestreamkind string Thefilterkindortype

4.14.2.2Paths

HTTP Path Summaryget /filter Returnsalistoffilteresget /filter/{name} Returnsthenamedfilterput /filter/{name} Setanfilterdelete /filter/{name} Deletesthenamedfilter

4.14.2.2.1/filter

4.14.2.2.1.1GET/filter

Returnsalistofallfilteres

Responses

Page 166: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Code Description Schema200 Thelistoffilterses array[Filter]401 Notauthorized String

4.14.2.2.2/filter/{name}

4.14.2.2.2.1GET/filter/{name}

Returnsanfilterbyname

Responses

Code Description Schema200 Returningtheinformationofthefilter Filter401 Notauthorized String404 Thenamedfiltercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefilter True String

4.14.2.2.2.2PUT/filter/{name}

Setsthenamedfilter

Responses

Code Description Schema200 Filterupdated String401 Notauthorized String400 Errorupdatingfilter String

RequestBody

Locatedin Description Required SchemaBody Thenewfiltertocreate True Filter

Page 167: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

4.14.2.2.2.3DELETE/filter/{name}

Deletesanfilterbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenamedfiltercouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthefilter True String

4.14.2.3filter.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Filter

description:|-

Filterscanoperateonavarietyofobjectsandreducethe

informationreceivedbasedonasearchcriterion.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/filter:

get:

tags:

-Filter

summary:Returnsalistoffilteres

description:Returnsalistofallfilteres

operationId:cloudmesh.filter.list

responses:

'200':

description:Thelistoffilterses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Filter'

'401':

description:Notauthorized

/filter/{name}:

get:

tags:

-Filter

summary:Returnsthenamedfilter

description:Returnsanfilterbyname

operationId:cloudmesh.filter.find_by_name

parameters:

-name:name

Page 168: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

in:path

required:true

schema:

type:string

description:Thenameofthefilter

responses:

'200':

description:Returningtheinformationofthefilter

content:

application/json:

schema:

$ref:'#/components/schemas/Filter'

'401':

description:Notauthorized

'404':

description:Thenamedfiltercouldnotbefound

put:

tags:

-Filter

summary:Setanfilter

description:Setsthenamedfilter

operationId:cloudmesh.filter.add

requestBody:

description:Thenewfiltertocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Filter'

responses:

'200':

description:Filterupdated

'401':

description:Notauthorized

'400':

description:Errorupdatingfilter

delete:

tags:

-Filter

summary:Deletesthenamedfilter

description:Deletesanfilterbyname

operationId:cloudmesh.filter.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthefilter

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenamedfiltercouldnotbefound

components:

schemas:

Filter:

type:object

description:Thefilter

properties:

name:

type:string

description:Nameofthefilter

function:

type:string

description:Thefunctionusedtofilterthedata

inthestream

kind:

type:string

description:Thefilterkindortype

4.15DEPLOYMENT

4.15.1Deployment

Aresourcetostoresoftwaredeployments.Thedeploymentisformulatedinaspecificationfile.Todistinguishtheformatofthespecificationfileastringisusedthatdefinesthekindofthedeployment.Incasethespecificationusesanexternalserviceanendpointtotheservicecanbe

Page 169: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

usedandthenameofthespecificationisusedtoidentifythedeployment.

4.15.1.1SchemaDeployment

Reference:☁�

Property Type Descriptionname string Thenameofthedeploymentkind string Thekindofthedeploymentspecification string Thespecificationofthedeploymentendpoint string Thelocationofthedeploymentservice

endpointname stringincaseanendpointisused,theendpointnameisusedtouniqulyidentifythedeploymentwithintheendpointdefinedservice

4.15.1.2Paths

HTTP Path Summaryget /deployment Returnsalistofdeploymentesget /deployment/{name} Returnsthenameddeploymentput /deployment/{name} Setandeploymentdelete /deployment/{name} Deletesthenameddeployment

4.15.1.2.1/deployment

4.15.1.2.1.1GET/deployment

Returnsalistofalldeploymentes

Responses

Code Description Schema200 Thelistofdeploymentses array[Deployment]401 Notauthorized String

4.15.1.2.2/deployment/{name}

4.15.1.2.2.1GET/deployment/{name}

Page 170: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Returnsandeploymentbyname

Responses

Code Description Schema200 Returningtheinformationofthedeployment Deployment401 Notauthorized String404 Thenameddeploymentcouldnotbefound String

Parameters

Name Locatedin Description Required Schemaname path Thenameofthedeployment True String

4.15.1.2.2.2PUT/deployment/{name}

Setsthenameddeployment

Responses

Code Description Schema200 Deploymentupdated String401 Notauthorized String

RequestBody

Locatedin Description Required SchemaBody Thenewdeploymenttocreate True Deployment

4.15.1.2.2.3DELETE/deployment/{name}

Deletesandeploymentbyname

Responses

Code Description Schema200 Deletionsuccessful String401 Notauthorized String404 Thenameddeploymentcouldnotbefound String

Page 171: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

Parameters

Name Locatedin Description Required Schemaname path Thenameofthedeployment True String

4.15.1.3deployment.yaml

openapi:"3.0.2"

info:

version:3.2.0

x-date:17-06-2019

x-status:defined

title:Deployment

description:|-

Aresourcetostoresoftwaredeployments.Thedeploymentisformulatedin

aspecificationfile.Todistinguishtheformatofthespecificationfile

astringisusedthatdefinesthekindofthedeployment.Incasethe

specificationusesanexternalserviceanendpointtotheservicecan

beusedandthenameofthespecificationisusedtoidentifythe

deployment.

termsOfService:"https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt"

contact:

name:NISTBDRAInterfaceSubgroup

url:https://cloudmesh-community.github.io/nist

license:

name:Apache2.0

url:https://github.com/cloudmesh/cloudmesh-nist/blob/master/LICENSE.txt

servers:

-url:/cloudmesh/v3

paths:

/deployment:

get:

tags:

-Deployment

summary:Returnsalistofdeploymentes

description:Returnsalistofalldeploymentes

operationId:cloudmesh.deployment.list

responses:

'200':

description:Thelistofdeploymentses

content:

application/json:

schema:

type:array

items:

$ref:'#/components/schemas/Deployment'

'401':

description:Notauthorized

/deployment/{name}:

get:

tags:

-Deployment

summary:Returnsthenameddeployment

description:Returnsandeploymentbyname

operationId:cloudmesh.deployment.find_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthedeployment

responses:

'200':

description:Returningtheinformationofthedeployment

content:

application/json:

schema:

$ref:'#/components/schemas/Deployment'

'401':

description:Notauthorized

'404':

description:Thenameddeploymentcouldnotbefound

put:

tags:

-Deployment

Page 172: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

summary:Setandeployment

description:Setsthenameddeployment

operationId:cloudmesh.deployment.add

requestBody:

description:Thenewdeploymenttocreate

required:true

content:

application/json:

schema:

$ref:'#/components/schemas/Deployment'

responses:

'200':

description:Deploymentupdated

'401':

description:Notauthorized

delete:

tags:

-Deployment

summary:Deletesthenameddeployment

description:Deletesandeploymentbyname

operationId:cloudmesh.deployment.delete_by_name

parameters:

-name:name

in:path

required:true

schema:

type:string

description:Thenameofthedeployment

responses:

'200':

description:Deletionsuccessful

'401':

description:Notauthorized

'404':

description:Thenameddeploymentcouldnotbefound

components:

schemas:

Deployment:

type:object

description:thedeployment

properties:

name:

type:string

description:Thenameofthedeployment

kind:

type:string

description:Thekindofthedeployment

specification:

type:string

description:Thespecificationofthedeployment

endpoint:

type:string

description:Thelocationofthedeploymentservice

endpointname:

type:string

description:incaseanendpointisused,theendpointnameisused

touniqulyidentifythedeploymentwithinthe

endpointdefinedservice

Page 173: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

5ACRONYMSANDTERMS

Thefollowingacronymsandtermsareusedinthisvolume.

ACID

Atomicity,Consistency,Isolation,Durability

API

ApplicationProgrammingInterface

ASCII

AmericanStandardCodeforInformationInterchange

BASE

BasicallyAvailable,Softstate,Eventualconsistency

Container

Seehttp://csrc.nist.gov/publications/drafts/800-180/sp800-180_draft.pdf

CloudComputing

ThepracticeofusinganetworkofremoteservershostedontheInternettostore,manage,and process data, rather than a local server or a personal computer. Seehttp://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf.

DevOps

AclippedcompoundofsoftwareDEVelopmentandinformationtechnologyOPerationS

Deployment

Theactionofinstallingsoftwareonresources

HTTP

HyperTextTransferProtocolHTTPSHTTPSecure

Hybrid

CloudSeehttp://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf.

Page 174: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

IaaS

InfrastructureasaServiceSaaSSoftwareasaService

ITL

InformationTechnologyLaboratory

MicroserviceArchitecture

Is an approach to build applications based on many smaller modular services. Eachmodulesupportsaspecificgoalandusesasimple,well-definedinterfacetocommunicatewithothersetsofservices.

NBD-PWG

NISTBigDataPublicWorkingGroup

NBDRA

NISTBigDataReferenceArchitecture

NBDRAI

NISTBigDataReferenceArchitectureInterface

NIST

NationalInstituteofStandardsandTechnology

OS

OperatingSystem

REST

REpresentationalStateTransfer

Replica

Aduplicateofafileonanotherresourcetoavoidcostlytransfercostsincaseoffrequentaccess.

ServerlessComputing

Serverlesscomputingspecifiestheparadigmoffunctionasaservice(FaaS).Itisacloudcomputing code execution model in which a cloud provider manages the functiondeploymentandutilizationwhileclientscanutilizethem.Thechargemodelisbasedon

Page 175: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

executionofthefunctionratherthanthecosttomanageandhosttheVMorcontainer.

SoftwareStack

Asetofprogramsandservicesthatareinstalledonaresourcetosupportapplications.

VirtualFileSystem

Anabstractionlayerontopofadistributedphysicalfilesystemtoalloweasyaccesstothefilesbytheuserorapplication.

VirtualMachine

AVMisa softwarecomputer that, likeaphysicalcomputer, runsanoperatingsystemandapplications.TheVMiscomposedofasetofspecificationandconfiguration filesandisbackedbythephysicalresourcesofahost.

VirtualCluster

Avirtualcluster isasoftwarecluster that integrateeitherVMs,containers,orphysicalresourcesintoanagglomerationofcomputeresources.Avirtualclusterallowsuserstoauthenticateandauthorizetothevirtualcomputenodestoutilizethemforcalculations.Optional high-level services that can be deployed on a virtual cluster may simplifyinteractionwiththevirtualclusterorprovidehigher-levelservices.

Workflow

Thesequenceofprocessesortasks

WWW

WorldWideWeb

Page 176: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

BIBLIOGRAPHY

[1] NIST, “Big Data Public Working Group (NBD-PWG).” [Online]. Available:https://bigdatawg.nist.gov/

[2] W. L. Chang (Co-Chair), N. Grady (Subgroup Co-chair), and NIST Big Data PublicWorkingGroup,“NISTBigDataInteroperabilityFramework:Volume1,BigDataDefinitions(NIST SP 1500-1 VERSION 3),” National Institute of Standards; Technology (NIST),Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[3] W. L. Chang (Co-Chair), N. Grady (Subgroup Co-chair), and NIST Big Data PublicWorking Group, “NIST Big Data Interoperability Framework: Volume 2, Big DataTaxonomies (NIST SP 1500-2 VERSION 3),” National Institute of Standards; Technology(NIST), Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[4]W.L.Chang(Co-Chair),G.Fox(SubgroupCo-chair),andNISTBigDataPublicWorkingGroup, “NIST Big Data Interoperability Framework: Volume 3, Big Data Use Cases andGeneral Requirements (NIST SP 1500-3 VERSION 3),” National Institute of Standards;Technology (NIST), Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[5]W. L. Chang (Co-Chair), A. Roy (Subgroup Co-chair), M. Underwood (Subgroup Co-chair), and NIST Big Data Public Working Group, “NIST Big Data InteroperabilityFramework: Volume 4, Big Data Security and Privacy (NIST SP 1500-4 VERSION 3),”National Institute of Standards; Technology (NIST), Gaithersburg,MD, Jul. 2019 [Online].Available:https://bigdatawg.nist.gov/show_InputDoc.php

[6]W.L.Chang(Co-Chair),S.Mishra(Editor),andNISTBigDataPublicWorkingGroup,“NISTBigDataInteroperabilityFramework:Volume5,BigDataArchitecturesWhitePaperSurvey(NISTSP1500-5VERSION1),”NationalInstituteofStandards;Technology(NIST),Gaithersburg, MD, Sep. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[7]W.L.Chang(Co-Chair),D.Boyd(SubgroupCo-chair),O.Levin(Version1SubgroupCo-Chair), and NIST Big Data Public Working Group, “NIST Big Data InteroperabilityFramework: Volume 6, Big Data Reference Architecture (NIST SP 1500-6 VERSION 3),”National Institute of Standards; Technology (NIST), Gaithersburg,MD, Jul. 2019 [Online].Available:https://bigdatawg.nist.gov/show_InputDoc.php

[8]W.L.Chang(Co-Chair),R.Reinsch(SubgroupCo-chair),D.Boyd(Version1SubgroupCo-chair),C.Buffington(Version1SubgroupCo-chair),andNISTBigDataPublicWorkingGroup,“NISTBigDataInteroperabilityFramework:Volume7,BigDataStandardsRoadmap

Page 177: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

(NIST SP 1500-7 VERSION 3),” National Institute of Standards; Technology (NIST),Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[9]W.L.Chang(Co-Chair),G.vonLaszewski(Editor),andNISTBigDataPublicWorkingGroup, “NIST Big Data Interoperability Framework: Volume 8, Big Data ReferenceArchitecture Interfaces (NIST SP 1500-9 VERSION 2),” National Institute of Standards;Technology (NIST), Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[10]W.L.Chang(Co-Chair),R.Reinsch(SubgroupCo-chair),C.Austin(Editor),andNISTBig Data Public Working Group, “NIST Big Data Interoperability Framework: Volume 9,Adoption and Modernization (NIST SP 1500-10 VERSION 2),” National Institute ofStandards; Technology (NIST), Gaithersburg, MD, Jul. 2019 [Online]. Available:https://bigdatawg.nist.gov/show_InputDoc.php

[11]NIST,“V1.0finalversionpageofthenbd-pwgwebsite.”WebPage,Sep-2015[Online].Available:https://bigdatawg.nist.gov/V1_output_docs.php

[12]NIST,“21.0finalversionpageofthenbd-pwgwebsite.”WebPage,Sep-2015[Online].Available:https://bigdatawg.nist.gov/V2_output_docs.php

[13]TheWhiteHouseOfficeofScience andTechnologyPolicy, “BigData is aBigDeal.”OSTP Blog, Feb-2014 [Online]. Available: http://www.whitehouse.gov/blog/2012/03/29/big-data-big-deal

[14]DepartmentofDefense,“Thedodafarchitectureframeworkversion2.02,”DepartmentofDefense,Report 2.02,Apr. 2010 [Online].Available: https://dodcio.defense.gov/library/dod-architecture-framework/

[15] G. von Laszewski, “NIST bdra vol 8. GitHub issues.” GitHub, Jun-2019 [Online].Available:https://github.com/cloudmesh/cloudmesh-nist/issues

[16] G. von Laszewski, “Nist bdra volume 8 github history.” GitHub, Jun-2019 [Online].Available:https://github.com/cloudmesh/cloudmesh-nist/commits/master/docs/nistvol8-2.epub

[17]G.vonLaszewskietal.,“NISTbigdatainteroperabilityframework:Volume8,referencearchitecture interfaces,” National Institute of Standards; Technology (NIST), Gaithersburg,MD, Special Publication (NIST SP), Oct. 2015 [Online]. Available:https://github.com/cloudmesh/cloudmesh-nist/raw/master/history/NIST.SP.1500-9.pdf

[18]G.vonLaszewski,F.Wang,B.Abdul-Wahid,H.Lee,G.C.Fox, andandWoChang,“Cloudmeshinsupportofthenistbigdataarchitectureframework,”IndianaUniversity,2017[Online].Available:https://github.com/cyberaide/nist/blob/master/vonLaszewski-nist.pdf

[19] G. von Laszewski, “NIST bdra vol 8. OpenAPI specifications.” GitHub, Jun-2019

Page 178: NIST Big Data Interoperability Framework: Volume 8 ... · Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts

[Online].Available:https://github.com/cloudmesh-community/nist/tree/master/spec

[20] G. von Laszewski, “Configuration file example: Coudmesh4.yaml.” GitHub, Jun-2019[Online]. Available: https://github.com/cloudmesh/cloudmesh-cloud/blob/master/cloudmesh/etc/cloudmesh4.yaml

[21] Internet2 Middleware Architecture Committee for Education, “EduPerson object classspecification,” Internet2, 2016, 201602, Mar. 2016 [Online]. Available:http://software.internet2.edu/eduperson/internet2-mace-dir-eduperson-201602.html