1 Don Brutzman and Jeff Weekley Naval Postgraduate School (NPS) Center for Autonomous Underwater Vehicle (AUV) Research Modeling, Virtual Environments.

  • Published on

  • View

  • Download


  • *Don Brutzman and Jeff Weekley

    Naval Postgraduate School (NPS)Center for Autonomous Underwater Vehicle (AUV) ResearchModeling, Virtual Environments & Simulation (MOVES) Institute23 August 2009NPS AUV Workbench:Rehearsal, Reality, Replay forUnmanned Vehicle Operations

  • *TopicsWhy modeling & simulation?AUV Workbench ComponentsSonar VisualizationTechnologies: X3D, XML, XMSFLooking aheadDemonstrations

  • theory = conceptual description of reality

    experiment = test theory in physical worldScientific method, 15th-20th centuries


    Scientific method, 15th-20th centuries




  • model = formal representation of reality

    simulation = behavior of model over timeScientific method, 1950-presentScientific method, 1950-present


    Scientific method, 1950-present




    implements a

    corresponds to an



  • model = formal representation of reality

    simulation = behavior of model over timeScientific method, 1950-presentScientific method, 1950-present


    Scientific method, 1950-present

    Modeling &Simulation




  • Virtual environments can connect all models and simulations togetherScientific method, emerging 21st centuryScientific method, emerging 21st century


    Scientific method, emerging 21st century




    Modeling &Simulation

    Virtual Environments linked together via networking interfaces, shared semantics

  • *AUV Workbench Project DescriptionOpen source, Java, XML, X3D graphicsMission planningRobot mission executionHydrodynamics responseSonar modeling3D visualizationCompressed radio frequency (RF) and acoustic communications

  • AUV Workbench poster

  • *Our 3 Rs: rehearsal, reality, replaySame needs and capabilities for each: mission, visualization, data support, etc.AUV workbench supports eachongoing work, starting to mainstream15 years of accumulated effortintegrating great variety of successful worknew work projects occurring regularlyCollaboration is welcome

  • *RehearsalMission planning and preparation

  • *Visualization, mission planning

  • Multiple vehicles supported

  • *RehearsalPrepare missions, either manually or automatically via other software toolsTest robot softwares ability to perform commandsTest again with physics in the loopHydrodynamics and control are critical, difficultSonar, environmental modelingRepeat until robust, with cautious respectSimulation is doomed to success G. Bekey

  • *Mission views: iconic, tree, XML, dialog box

    Each view is consistent with GIS, 2D, 3D views

  • *Supporting views: mission metadata, state

  • *2D planner: script missions

  • *2D planner: script missionsCan edit missions by adding or removing script commands

  • *2D planner: agenda missions

  • *OpenMap GIS displayOpenMap http://www.openmap.org Geographic Information System (GIS)Open source Java, bundledBuilding layers for areas of interestGeographic coordinates throughoutWill synchronize with mission definitions, X3D

  • *

  • *Secure sftp download of large GIS datasets

  • *Reality: real-time mission supportMonitor mission progressTask-level control using same mission vocabularyVisualize and supervise operationscaveat, again: work in progressIntegrate acoustic and RF communicationsChat for distributed collaboration among participants, both human and robotic

  • *Real-time mission data import/exportExportMission commands that are already rehearsedConvert to specific dialect particular to that robotImportMission telemetry recording detailed track dataData products: imagery, video, mission log, etc.

  • *Record mission metadata for archivesSupport operator keeping detailed notes, kept in context when conducting missionPrompt for full details as appropriateArchive notes for later review and followup

    Future workAutomatic tests to confirm configuration, controlAutomate pre-underway checklists

  • *Serial port communicationsConfigurable to different devices, ports

  • *JavaHelp support

  • *Thesis list

  • *Snapshot support

  • *Replay: post-mission supportAutomatic archiving of mission to serverBeing built into workbench simplify user tasksIntegration and compression of all relevant data into single compressed XML fileMetadata for missionMany pieces: ordered mission, commands, telemetry, coefficients, contacts, etc. etc.Autonomous Vehicle Control Language (AVCL) is Ph.D. work by CDR Duane Davis

  • Telemetry data replay

  • *Geographic track plot

  • *x y z versus t plot

  • *phi theta psi versus t plot

  • *Mission plots using gnuplot

  • *ELumens dome display supportwww.elumens.com

  • *Physical modelingControl algorithms and 6 degree-of-freedom (6DOF) hydrodynamics responseSonar propagation, attenuationCollision detectionDirect vehicle contact and sensor contactSeparate use of same X3D graphics modelsVisualization greatly aids understandingprovides good forcing function for integration

  • *Control algorithm coefficients

  • *6DOF dynamics coefficients

  • *Wave modelingTriple sinusoid Pierson Moskowitz equations provide good emulation of variable sea stateWell understood example modelMany other variations exist, could substituteReal-time modeling of underwater vehicle response when broachedSplit hull into sectionsCompute each one as linear approximationBuoyancy components add to overall response

  • *REMUS mission search, from above

  • *REMUS mission search, from behind

  • *Group development supportOpen standards throughoutOpen source Java for softwareAll data structured as XMLWebsite https://savage.nps.edu/AuvWorkbench XMSF BugtrackerEmail list with hypermail archiveOnline autoinstallers

  • *Website

  • *XMSF Bugtracker

  • *Mailing list support

  • *Environmental data inputsConstant vectors for ocean current, windNetCDF environmental data developed by NAVO/NRL Stennis supercomputer modelsFNMOC web-services query to live/projected meteorological sources using Joint Metoc Brokering LanguageWorked briefly but was a moving targetOther inputs welcome

  • *Goal Outcomes: sonar-vis projectSonar and battlespace visualizationLink aircraft, ship and HPCC computingSimilar interfaces, with/without connectivityTactical Supercomputing refrigerator boxSupport shared situational awarenessCandidate technologies for spiral developmentContinue to lead usw-xml working group for USN NAVSEA

  • Sonobuoy field visualization

  • *Integrating 2D/3D interfaces with Web Services

  • *sonar-vis Project DescriptionVisualize multipath 3D sonar propagationSituational awareness, sensitivity analysisMultiple models: path, transmission loss, PD ... Operator familiarization, training, experienceEnhance TDAs for at-sea operatorsReachback using Web services messaging, accessing both computational and data assetsOpen source open standards: Java, X3D, XML

  • *Sonar Visualization poster, I/ITSEC 2003

  • *XML web services for METOC data 1Query panel and plotted response

  • *XML web services for METOC data 2Monitoring initial query/response sequence

  • *XML web services for METOC data 3Server-side supercomputer response

  • *XML web services for METOC data 4

  • *Collision detection architecturePrinciple: single X3D scene database for 3D rendering and collision detectionBasic functionality implemented in Xj3D for U.S. Army Deformable Surfaces project Potential extension to X3D SpecificationOpen-source implementation & evaluationLarge-scene physics feasible in real time

  • *Collision detection architectureUnder development

  • *TechnologiesExtensible Markup Language (XML)Validatable data, binary compressionWeb Services for message exchangeEnhance current sonar-model enginesRecursive Ray Acoustics (RRA) sonar computationPC-IMAT/STAPLE/STDA(ASPECT) primary targetsEnvironmental data from FNMOC via Web servicesExtensible 3D (X3D) GraphicsOpen-standard open-source interactive visualization

  • *XML in 10 Points http://www.w3.org/XML/1999/XML-in-10-pointsXML is for structuring dataXML looks a bit like HTMLXML is text, but isn't meant to be readXML is verbose by designXML is a family of technologiesXML is new, but not that newXML leads HTML to XHTMLXML is modularXML is basis for RDF and the Semantic WebXML is license-free, platform-independent and well-supported400+ member companies & institutions in World Wide Web Consortium (W3C)already understand the business case

  • *Extensible Modeling & Simulation Framework (XMSF)Web services for all manner of M&SA composable set of standards, profiles, and recommended practices for web-based M&SFoundational precepts: Internet network technologies, Extensible Markup Language (XML)-based languages, and service-oriented architectures for simple messaging Enable a new generation of distributed M&S applications to emerge, develop, interoperate with tactical systemsMany easily repeatable exemplars using Web Services


  • *What is 3D?2D works for chart-oriented displays3D gives fly-thru freedom of viewpointView physically based propagation pathsView depth separationView bottom, surface interactionsView multiple overlapping sensorsAugment (not replace) existing displays

  • *What is X3D?Extensible 3D (X3D) GraphicsVirtual Reality Modeling Language (VRML) updatedThird-generation ISO specification Compatible XML .x3d and Classic VRML .wrl encodingsDeliverablesSpecification updates, with compatible XML tagset Multiple implementations, including open-sourceScene Access Interface (SAI) strongly typed APIConformance suite and examplesAuthoring capability: X3D-Edit, using XML for XML

  • *Further X3D motivationsAuthoring is hard, Content is KingX3D is not competing with specialty formats, instead provide common interoperability/interchangeStrong validation checks eliminate most authoring errors before content escapesPlays well with next-generation Web languages3D hardware problem is already solved

  • X3D Specifications honeycomb diagramX3D Specification itself iscomponentizedand extensible

  • X3D-Edit complete interface

  • SAVAGE 3D Model Archive

  • *Numerous underwater vehicle models

  • *Forward error correction (FEC)Added redundancy allows receiver-side detection & correction of message errorsMany military channels are noisy RF linksAvoids retry until you die on acoustic linksBig help on long-latency, low-bandwidth links!Hamming FEC is one technique of severalRe-exploring Stephen Reimers 1995 thesis Towards Internet Protocol (IP) over Seawater

  • *Tactical supercomputingLinux clusters can create new resources10 new off-the-shelf PCs + GPUs + disk storage$10K = 20-30 processors = maybe 10 Gigaflop Refrigerator-rack footprints easily fit shipboard Exploring intermediate-level resources for previously supercomputer-level problemsConsistent data access via grid, web services Supercomputer reachback provides even higher capabilities (emerging as cloud computing)

  • XML Tactical Chat (XTC), I/ITSEC 2003 poster

  • Chat log window

  • Chat log debug mode shows underlying XML

  • *Event monitoring via instant messaging chatbot listens and reacts tofree-form messages of interestby plotting mine onto chartlet

  • *Java regular expression parser on chat:Any characters or words before "MINE"Case-insensitive word MINEs or "S" occurs once or none at allAny characterComma followed by 0 to 2 character "space"Group 1 (digit)One or no character "space", fullstop or ")"1 to 2 characters "space" or "("Comma followed by 0 to 2 character "space"Group 2 digit)Group 3 digit)Breakdown of regular expression pattern:

    Meaningful messages can be extracted from chat text,thus enabling automatic structure for user support

    Regular Expression^.*(?i)MINE[s|S]?.*[ (]{1,2}(\d*),[ ]{0,2}(\d*),[ ]{0,2}(\d*)[ ).]?+

  • *Related work: usw-xml The usw-xml working group is improving Undersea Warfare (USW) interoperability using Extensible Markup Language (XML) tagsets for system data interchange.

    Cooperative collaboration between many stakeholders is needed to achieve good interoperability.

  • *MotivationMany XML technical capabilities enable significantly improved capabilities for USW system interoperabilityConnecting legacy systems, diverse partnersUSW Decision Support System, other projects

    We expect this work to broadly benefit the Navy, industry and scientific community.

  • *Recent work: AUV Fest 2008Naval Undersea Warfare Center (NUWC) Newport RI, 12-24 May 2008Sponsors: NOAA and ONR17 underwater robots and various surface support craft exploring archaeological sitesAUV Workbench supported data collection, reporting, visualization

  • AUV Fest sediments data

  • AUV Fest 2008

  • AUV Fest

  • AUV Fest

  • AUV Fest NUWC Waterfront

  • *Current and Future WorkMultiple related projects

  • *Major projectsAutomatic Report GenerationViskit Discrete Event Simulation (DES)Efficient XML Interchange (EXI)Binary compression of XML, smaller + fasterOpen-DIS codebaseIEEE Distributed Interactive Simulation ProtocolX3D EarthX3D graphics models for terrain, bathymetry

  • *SunspotSun Programmable Object Technology (SPOT) devices are small wireless Java microcontrollers that include accelerometers, USB port, LEDs and other connectionsWe are looking ahead towards using SunSpot devices as UUV, USV and UAV mobile robot controllershttp://www.sunspotworld.com https://robotics.dev.java.net

  • *Physics modeling for the Web: Dissertation Work by Loren Peitso lepeitso@nps.edu Current physically based simulations do not scale well and do not take advantage of parallelization, distributed processing or composability in an extendable, scalable manner.Most simulations also restrict the domains of physics they evaluate, substituting constructive estimations for physical modeling. This precludes accurately simulating world-scale scenarios such as a theatre level battle space and simulations which are run yield results which are only as good as their radically simplifying assumptions allow.It is proposed that a hybrid high-delity, distributed, asynchronously evaluated discrete event multi-body dynamics simulation architecture will signicantly increase scalability compared to current synchronously evaluated methods.This architecture also supports non-classical mechanics physics domains in the simulation, adding capability not presently found in state-of-the-art synchronously evaluated physically based simulations.We also propose that this architecture lends itself to increased component and scenario composability as well as re-use through standards-based approaches and published extendable interfaces. Performance comparisons are to be made between existing synchronous physics simulations and an implementation of this new architecture.

  • *New scenarios under way

  • *SeaDiver search

  • *Automating server-side supportWork is resuming to automate upload and report generation

    Technical metric: make it easy!

  • *SummarySignificant collected AUV capabilitiesSupport rehearsal, reality, replayIntegrated as tactical applicationOpen standards: XMSF, X3D, chat, etc.Open source + commercial compatibilityImproved messaging, net-centric exemplarWe hope to add all possible vehicles!Collaboration and questions welcome

  • *Acronyms13D: Three dimensional6DOF: Six degrees of freedom (x y z, roll pitch yaw)AUV: Autonomous Underwater VehicleAVCL: Autonomous Vehicle Control LanguageCD: Compact DiskCUP: Common Undersea PictureFEC: Forward Error CorrectionFNMOC: U.S. Navy Fleet Numerical Meteorological & Oceanographic CenterHPCC: High-Performance Computing CenterJava: programming languageMETOC: meteorological and oceanographic (data)NAVAIR: U.S. Naval Air Systems Command

  • *Acronyms2NPS: Naval Postgraduate School, Monterey CaliforniaPD: Probability of detectionRF: radio frequencyRRA: Recursive Ray Acoustics Sonar PropagationSBIR: Small Business Innovative ResearchTDA: Tactical Decision AidUSW: Undersea Warfare

    X3D: Extensible 3D Graphics SpecificationXML: Extensible Markup Language XMSF: Extensible Modeling and Simulation FrameworkXSBC: XML Schema-based Binary CompressionXTC: XML Tactical Chat

  • *Contact

    Don Brutzman

    brutzman@nps.edu http://web.nps.navy.mil/~brutzman

    Code USW/Br, Naval Postgraduate SchoolMonterey California 93943-5000 USA1.831.656.2149 voice1.831.656.7599 fax

  • Data-driven visualizationDefine visualization features of interestX3D examples collected in archivePrototypes for parameterized visualizationArbitrary data sources can be used as inputConvert into relevant form of XMLXSLT stylesheet for X3D prototype instanceIntegrate new model into master sceneVisualize, annotate, publish/archive, repeat*

  • TopicsAUV Workbench overviewinstallation, operation, mission examplesAdding a vehicleX3D GraphicsX3D Showcase DVD, learning resourcesSavage X3D model archives, X3D EarthHands onBuilding and testing a mission, add metadata


  • AUV Workbench projected work 1Mission report generation from metadata, plots, related data sourcesMission project archivingMore robot missionsKML export: GoogleEarth, NASA WorldWindCollected assets pushed to serverVirtual environments archive of mission resultsVirtual environments resource repository?*

  • AUV Workbench projected work 2Efficient XML Interchange (EXI) compressionOther media assetsCreation of 3D storyboards, playbackRender 3D visualizations to video filesProvide data to other Navy systemsTrack Data Conversion Hub provides translations between various XML-based, track-related data languages


  • AUV Workbench wish listWeekly teleconference?If sufficient demand existsSimplify addition of new vehicle typesClear up large backlog of bug reportsAdding GIS files for other test sitesAUV 2010, autumn next year, MontereyIEEE OES sponsor, MBARI + NPS hostCandidate theme: Mapping the Ocean


    *A model is a physical, mathematical or logical representation of a system, entity, phenomenon, or process. http://www.education.dmso.mil/ms_primer.asp?a=s4&b=view&c1=272

    Verification = did we build the model correctly, validation = did we build the correct model.Accreditation = blessing by a review board or standards body that process was correctly followed.

    *A simulation must have a model to run these operate together when running.**Quick-summary XML reference: XML in Ten Points by Bert Bos et al., http://www.w3.org/XML/1999/XML-in-10-points*Historically, for the past 30 years, the two gating problems to broad 3D graphics deployment have been graphics hardware performance and software interoperability for 3D models. One down, one to go.*X3D-Edit is the exemplar authoring tool that demonstrates the X3D tagset. It is built using the Xeena GUI tool, which configures itself for X3D by reading XML files.*


View more >