IEEE: Robotics and automation Society (RAS)

Ontologies for Robotics and Automation

Study group

Craig SchlenoffNational Institute of Standards and Technology

Need

A robot can only achieve tasks and perform missions based on what it knows, which is primarily captured within the robot’s internal knowledge representation. This representation is usually very specialized to the individual robot and often very loosely defined. With the growing complexity of behaviors that robots are expected to perform as well as the need for multi-robot and human-robot collaboration, the need for a standard and well-defined knowledge representation is becoming more evident.

IEEE Ontologies for Robotics and Automation Goal

To develop a methodology for knowledge representation and reasoning in robotics and automation, together with the representation of concepts in an initial set of application domains, to allow for unambiguous knowledge transfer among any group of humans, robots, and other artificial systems.

Study Group Participants (78 subscribed to mailing list)

GroundAirUnderwater/SurfaceSpace

Scope (Robots)

Different Kinds of Robotic Knowledge

Cost-based Models

LADAR and Color Camera Images

Layered Terrain Maps

S3 TentativePlans_Done RoadMarchOrganizationInPlace

S4 sp3_ReadyToConductRouteRecon

S5 qp_ReadyToOrganizeAA

NewCommand

S1 MarchOrganizationDetermined

S2

CONDUCT TACTICAL ROAD MARCH TO ASSEMBLY AREA

S6 qp_ClearOfStartPoint

S7 mb_tp_ReconToStartPoint_Done

1

2

14

13

12

11

10

9

8

7

6

5

4

3

15

16

S8 sp3_RouteRecon_Done

S9 qp_AtReleasePoint

S10 qp_AreaReconOfAADone

17

S11 qp_Status_AssemblyAreaSuitable

S14 mb_Unit_AtReleasePoint qp_AssemblyAreaReady

S16 mb_Unit_AtReleasePoint

S12 mb_Unit_AtStartPoint

S12 mb_LastUnit_AtStartPoint

S13 tp_TrailPartyAtStartPoint

S14 HaltCriteriaMet

NewSituationReport

NewHigherLevelInformation

S15 HaltEnded_ReadyToContinueMarch

DisabledVehicleUnscheduledHalt

S16 tp_AtReleasePoint

S17 mb_tp_OccupyingAssemblyArea

S18 AllUnitsSecureInAssemblyArea

S4 sp3_PrepareForRouteReconnaissance qp_PrepareToOrganizeAssemblyArea

S5 sp3_ConductRouteReconnaissance

S6 qp_FollowReconPlatoonToAssemblyArea

S1 DetermineMarchColumnOrganization:

S2 all_FormTacticalRoadMarchOrganization

S3 MakeTentativePlan: Determine_Route, _FireSupport, _MovementFactors, _AA

S7 mb_tp_PrepareForRoadMarch

S8 PrepareDetailedMovementPlans:

S9 sp3_EstablishAssemblyAreaSecurity

S10 qp_ConductAreaReconnaissanceOfAA

S11 qp_OrganizeAssemblyArea

S12 1st_mb_Unit_MoveIntoRoadMarchFormation

S16 1st_mb_Unit_OccupyAssemblyArea

S16 mb_Unit_OccupyAssemblyArea

S12 mb_Unit_ExecuteTacticalRoadMarch next_mb_unit_MoveIntoRoadMarchFormation

S13 mb_Unit_ExecuteTacticalRoadMarchtp_MoveIntoRoadMarchFormation

S14 tp_SupportMarchColumnMovement

S15 mb_tp_ExecuteScheduledHalt

UpdateDetailedMovementPlans

UpdateDetailedMovementPlans

S14 mb_tp_ResumeExecutionOfTacticalRoadMarch

S15 mb_tp_ExecuteUnscheduledHalt

S17 tp_OccupyAssemblyArea

S18 all_FormStandardTroopOrganization

S0 TacticalRoadMarchToAssemblyArea_Done

25

24

23

22

21

20

19

18

State Tables

Ground TruthPrediction Equations

Databases

Per

cept

ion

Use Cases

Moving

Machine Ontology

Autonomous Vehicle Ontologies

Simulatio

n

State

Planning

Cost Based

Planning

Ontology

Judgment

Value Prediction

Object

Terms

General Logic

Thesauri

formalTaxonomies

Frames(OKBC)

Data Models(UML, STEP)

Description Logics

(DAML+OIL)

Principled, informal

hierarchies

ad hoc Hierarchies

(Yahoo!)structured Glossaries

XML DTDs

Data Dictionaries

(EDI)

‘ordinary’Glossaries

XML Schema

DB Schema

Glossaries & Data Dictionaries

MetaData,XML Schemas, & Data Models

Formal Ontologies & Inference

Thesauri, Taxonomies

Formalities of Knowledge Representations

What is an Ontology?

“a specification of a conceptualization”*Ontologies explicitly represent key concepts, their properties, their relationships, and their rules and constraints.Ontologies often focus more heavily on the meaning of concepts as opposed to terms that are used to represent them

Vocabulary + Structure = TaxonomyTaxonomy + (Relationships and Constraints) = Ontology

*Tom Gruber, Stanford Univ.

Ontology Application Scenarios

Common Access to InformationInformation required by multiple agents

Provides a single source of information for multiple applications

Ontology used as agreed standard

Benefits: knowledge reuse, maintainability, long term knowledge retention

Ontology

Application 1 Application 2 Application 3

referencesreferences

OA

references

• Ontology as Specification – build ontology for required domain– produce software consistent with ontology – manual or partially automated– Benefits: documentation, maintenance,

reliability, knowledge (re)use

Ontology Application Scenarios (cont.)

Ontology as an Exchange LanguageProvides an interlingua among disparate applications

Ensures semantic-based interoperability

Solves the point-to-point integration problem

Benefits: systems integration, semantic interoperability

• Ontologies for Reasoning– Allows one to run queries through

a reasoning engine– Helps to identify information that is

not explicitly represented– Benefits: knowledge inference Application

Ontology

Reasoning Engine

AD

OAaccesses

Interfaces with

Ontologies for Robotics and Automation Approach

Top downDevelop/identify an upper ontology to serve as the overarching structure that information can “hang from”

Develop a methodology to add new information to the ontology

Bottom UpDevelop detailed ontologies for a small set of domains

● Service Robots

● Autonomous Robots

Domains are intentionally broad to allow for overlapping concepts

Tying it all togetherIncorporate the domain ontologies into the upper ontology using the defined methodology

Reconcile any discrepancies that exist among concepts

Define a framework for building ontologies that allows robot designers to build domain-specific ontologies in a controllable wayDefine linguistic framework so that expressions using the ontologies can be communicated and so that it is possible to translate between different ontological realms (e.g. human and robot)Define top-level categories as a foundation for further extension

Upper Ontology and Framework SubgroupGoals

Service Robot Ontology SubgroupTypes of Service Robots

“A Service Robot is a robot which operates semi- or fully autonomously to perform services useful to the well-being of humans and equipment, excluding manufacturing operations” - International Federation of Robotics (IFR)

Industrial robots (e.g., radiographic inspection of welds)

Defense robots (e.g., autonomous scout vehicles)

Healthcare robots (e.g., surgical manipulation, wheel chairs)

Prosthetic robots (e.g., prosthetic arms, legs, etc.)

Scientific robots (e.g., gene sequencers)

Domestic robots (e.g., floor cleaners, lawn mowers)

Diffused robots (e.g., parallel park assist systems)

Military and law enforcement robots (e.g., drones, UAVs)

Service Robot Ontology SubgroupApproach

Phase 1: Identify and summarize the different definitions and glossaries used in the different Service Robots - bibliographical research

Phase 2: Requirement analysis to extend /modify the glossaries to meet the needs of industries and scientific communities

Phase 3: Aims at describing the robots in terms of well-defined ‘taxonomy’ by its components and operating environment

Phase 4: Defines and elaborates on the different knowledge layers - for high level Human-Robot/Robot-Robot Interaction

Autonomous robots are robots that can perform desired tasks in unstructured environments without continuous human guidance.

aerial photography using flying robots

customs and border security

electricity companies, who can inspect power lines, nuclear power plants, wind turbines, and other facilities using robots

gas and oil supply companies, who can use robots to inspect, maintain, and guard pipelines

local civic authorities

meteorological services, who can use UAVs to carry weather stations

river authorities and water boards

landmine detection and destruction.

Autonomous Robots Ontology SubgroupOverview

Description of robot hardware, softwareDescription of the activities that need to be performedDescription of the environment in which the robot needs to workUnderstand of cause and effect of performing actionsRelationship among other robots and/or people…

Autonomous Robots Ontology SubgroupKnowledge Representation

Investigate standards from other SDOs (ISO, ALFUS)Investigate existing efforts and formalisms and list the advantages/disadvantagesCollect practice in academia/industry (interview business management)Collect academia/industry requirements for standardizing autonomous robots ontologyHow to represent the terminology? Dictionaries, Glossaries, XML schemas, Databases, Logic-Based approaches, Others?Define ontology/glossary for autonomous robots

Define glossaries specific to autonomous robots

Define ontology for autonomous robots (air, ground, water, space)

Define ontology for sub-domains, sensor, perception, fusion, actuator, control, guidance, navigation, motion planning, mission planning, decision making (or software, hardware, electrical, mechanical, performance, …)

Define ontology for human machine interactions

Use of ontology to *describe* autonomous robots.

Set up a long-term roadmap for autonomous robots ontology

Autonomous Robots Ontology SubgroupApproach

PAR Procedures

PAR (Project Authorization Request) is the documentation necessary to start the process of turning a Study Group into a IEEE-Recognized Working GroupSeptember 16th - PAR drafted and submitted through IEEE Robotics and Automation Society/Standing Committee for Standards Activities (RAS-SCSA) mid-October - NesCOM (New Standards Committee) will review the PAR and make recommendations to the IEEE-SA Standards Board regarding approval.

Responsible for ensuring that proposed standards projects are within the scope and purpose of IEEE, assigned to the proper Society or other organizational body, and interested parties are appropriately represented in the development of IEEE standards.

How To Get Involved

Come to our meeting tomorrow (Sept 26th)!Union Square Rooms 23&24 (4th floor), 9am-12:30pm

Contact me:Email: craig.schlenoff@nist.gov

Phone: 301-975-3456

Join our Google group

IEEE RAS Ontologies for Robotics and Automationhttps://groups.google.com/forum/#!forum/ieeeraswgVisit the Web site set up for Autonomous Robots Group (http://aro.svn.sourceforge.net/)