Embed Size (px)
Citation preview
Eric IronsideBrett Phillips
ArcGIS GeoEvent Server: Creating Connectors and ProcessorsUsing the GeoEvent SDK
Apps
DesktopAPIs
analytics storage
visualization
ingestion
actuation
ArcGIS EnterpriseWith real-time Capabilities
AgendaCreating & Configuring GeoEvent Connectors
Custom Development with the GeoEvent SDK
Developing Custom Transports
Developing Custom Adapters
Developing Custom Processors
Development Odds & Ends
Summary
1
2
3
4
5
6
7
Creating & Configuring GeoEvent Connectors1
Inbound & Outbound ConnectorsWhat does a connector do?
• By choosing a Connector, the user implicitly selects components from the
GeoEvent Server that knows:
HOW to move data (Transport)
WHAT the data looks like (Adapter)
Example Input
TCP Transport Text Adapter
Input (Receive text from TCP)
Start Byte[ ]
GeoEvent
Connector defines an Input or Output
Port
Example Output
JSON Adapter HTTP Transport
Output (Post JSON to HTTP)
JSON
PostGeoEventGeoEvent Services
StopStartStop
Creating Inbound & Outbound ConnectorsHow does a connector help?
• Hides the technical details from the end user- HOW to move data (Transport)- WHAT the data looks like (Adapter)
• Completely hide properties that the user should not see- Hardcoded values- Properties that don’t apply
• Move properties under an “advanced” area to discourage modification
• Provide default values• Re-label properties to be appropriate to the context
Polling Rate (s):
GeoEvent Server
Inpu
ts
Out
puts
GeoEvent Services
Connectors Transport Adapter
Feature-JSON
Adapters
GeoJSON
JSON
RSS
Text
XML
CAP
Cursor-on-Target
GeoMessage
NMEA
Sierra Wireless (RAP)
Trimble (TAIP)
VMF
Esri
Gal
lery
Waze
Feature Service
Transports
File
HTTP
HTTP+BasicAuth
HTTP+OAuth
TCP
UDP
Waze
WebSocket
Esri
Gal
lery Amazon IoT
Azure IoT
IRC
MQTT
Feature-JSONReceive Feature-JSON from Kafka Kafka
GeoJSONReceive GeoJSON on a REST endpoint HTTP
RabbitMQ
Connectors for InputConfigure a New Input Connector
Pair an Inbound Transport and an Inbound Adaptertogether
Input Connector = Adapter + Transport
Kafka
GeoEvent Server
Inpu
ts
Out
puts
GeoEvent Services
Message Formatter SMTP
Connectors TransportAdapter
Feature-JSON Kafka
Send an Email
Publish Feature-JSON to Kafka
Big Data Store
Transports
Feature Service
File
HTTP
HTTP+BasicAuth
HTTP+OAuth
SMTP
Stream Service
TCP
UDP
WebSocket
XMPP
Cache
Adapters
Feature-JSON
GeoJSON
JSON
Message Formatter
RSS
Text
WebSocket
ActiveMQ
Amazon IoT
Azure IoT
Hadoop
IRC
Kafka
Esri
Gal
lery
MongoDB
MQTT
RabbitMQSMS
TCP-Squirt
Connectors for OutputConfigure a New Output Connector
Output Connector = Adapter + Transport
Pair an Outbound Adapterand an Outbound Transport together
Deploy Custom ComponentsAdding custom items to GeoEvent
• Get the .jar file for the Adapter, Transport, or Processor- Download the .zip file from ArcGIS Online “ArcGIS GeoEvent Server Gallery”- Compile your custom project
• Deploy it- Put the .jar file into the Deploy directory*
- Linux: ~/arcgis/server/GeoEvent/deploy/- Windows: C:\Program Files\ArcGIS\Server\GeoEvent\deploy\
- Upload it in ArcGIS GeoEvent Manager**- Site > Components > Add Local ** Doesn't always work
* Delete the file to remove the component from GeoEvent
Copy an existing connector and add a new one
ConnectorConfiguration Demo
Custom Development with the GeoEvent SDK2
GeoEvent Server
Inpu
ts
Out
puts
GeoEvent Services
Extending GeoEventSoftware Development Kit (SDK)
• You can create your own custom connectors and processors using the GeoEventSoftware Development Kit (SDK)
Your custom processors
Yourcustom output
connectors
Yourcustom inputconnectors
• Outbound Adapter• Outbound Transport• Outbound Connector
• Processor
Inbound Adapter •Inbound Transport •
Inbound Connector •
GeoEvent Server SDKInstalled with GeoEvent Server
• Linux: ~/arcgis/server/GeoEvent/sdk• Windows: C:\Program Files\ArcGIS\Server\GeoEvent\sdk
- API: JavaDoc (open in browser)- Repository: Local maven repository- Samples: Sample connectors and processors- GeoEvent Developer Guide (PDF)
file:///C:/Program%20Files/ArcGIS/Server/GeoEvent/sdk/api/index.html
Implementation PatternBundle Deployment Using OSGI
Service Class • Uses the definition• Instantiates a new implementation
Implementation Class • Implements application logic
Definition Class • Defines metadata and properties
Blueprint Configuration • Recipe for service creation
Blueprint ConfigurationConfiguring Your Service
• Declare service implementing GeoEvent Service interface• Declare bean with bundle context
- Optional: Data folders- Optional: References to other GeoEvent capabilities
Service ClassCreates Adapters, Transports, and Processors
Definition ClassStep 1: Metadata
• Java class or XML file• Metadata*:
- Identification- Name- Domain- Version
- Description- Label- Description- Contact Information
*Localized strings can be usedNOTE: Don’t use com.esri.ges !
• Property definition- Name, label, description, type, default value, read only, mandatory, …- Allowed values are represented by a List of label value pairs. - Localized strings can be used.- Properties can be dependent on others: “Compression Algorithm” depends on “Compressed = True”
• Add property definitions to the property definitions list
Definition Class (cont.)Step 2: Properties
Definition Class (Cont.)Property Examples
• Properties can be Mandatory or Optional.- Example: “Name” is required, but “URL” is optional
• Some properties are dependent on others- Example: “Compression Algorithm” depends on “Compressed = True”
• Some properties only accept values from a list of Allowed Values
My Sample Input
Definition Class (Cont.)As XML
• Definition class optionally defined as an XML file
Developing CustomTransports3
TransportLife Cycle
STOPPED
STOPPING
STARTED
STARTING
ERROR
STOPPED
Server calls start() Transport reads properties and starts
Server calls stop()Transport closes resources
Internal error occurs
STARTED
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
TransportLife Cycle Detail
1. Create2. Initialize3. Validate4. Start5. Receive (I/O)6. Stop
Developing CustomAdapters4
AdapterCreation & Initialization
AdapterParsing Data (Inbound)
AdapterParsing Data (Outbound)
Developing Custom Processors5
ProcessorLife Cycle
GeoEvent Server
GeoEvent Service
Processor
Validation
Shutdown Initialization
Creation
Processing…
ProcessorLife Cycle Detail
1. Create2. Initialize3. Validate4. Process5. Shutdown6. Start/Stop
1. Create2. Initialize3. Validate4. Process5. Shutdown6. Start/Stop
ProcessorLife Cycle Detail
ProcessorLife Cycle Detail
1. Create2. Initialize3. Validate4. Process5. Shutdown6. Start/Stop
ProcessorLife Cycle Detail
1. Create2. Initialize3. Validate4. Process5. Shutdown6. Start/Stop
ProcessorLife Cycle Detail
1. Create2. Initialize3. Validate4. Process5. Shutdown6. Start/Stop
Import, deploy, and validate an existing processor
Processor Demo
Development Odds & Ends6
Maven ProjectConfiguring and building your project
• Create/Import in IDE as a Maven Project• Maven POM.xml file(s)
- One POM (processors), Parent/Child POMs (connectors)
• Project Structure- Java Source- Resources Source
- config.xml, I18N Strings, definition xmls
File path for string property files must match your bundle group ID + artifact ID
LoggingUsing the logging interface
• Static Bundle logger• Log methods
- Parameterization- Stack Trace- Localized strings
• Suggested practices- error: kill your code- warn: cause data loss- info: user level details- debug: stack trace and
logic milestones- trace: code tracing
DebuggingPart 1: Configure GeoEvent
• Open file C:\Program Files\ArcGIS\Server\GeoEvent\etc\ArcGISGeoEvent.cfg- uncomment lines (as Administrator!!)- Continue numbering scheme (actual number will depend on version of GeoEvent)
• Debug Port = 5005
Debugging (Cont.)Part 2: Configure your project
• Configure Debugging for Remote Java Application
- IP of GeoEvent Server- Port from config file
Summary7
Real-Time GIS: Using the GeoEvent SDKSummary
• Connectors – Recipe for creating inputs/outputs- Transport – Moves raw data in/out of the GeoEvent Server- Adapter – Converts raw data to GeoEvents and back- Properties – Allows the end user to configure component for a specific use case.
• Processors – Real-time analytics- Modify, accumulate, combine, or filter events in real-time- Can take advantage of Esri Java Geometry libraries
• GeoEvent SDK – Tools to extend the GeoEvent capabilities for custom applications- GeoEvent Developer Guide- Samples- Develop and deploy custom processors, adapters, transports, and connectors
Real-Time GIS: Using the GeoEvent SDKResources
• Connector Examples- GeoEvent SDK Samples: Adapter w/Connector- Esri GitHub: https://github.com/Esri?q=geoevent+connector
• Transport Examples- GeoEvent SDK Samples: Sample Transport, GeoEvent Aware Transport
• Adapter Examples- GeoEvent SDK Samples: Sample Adapter, Sample Blueprint, Custom GeoEvent Definition,
Outbound Adapter, Adapter Properties, Adapter w/Connector
• Processor Examples- GeoEvent SDK Samples: Sample Processor, Asynchronous Processor, Slope Processor,
Volume Control Processor- Esri GitHub: https://github.com/Esri?q=geoevent+processor- Demo Processor: https://github.com/mrasante/GeoEventDemoProcessor
https://community.esri.com/community/gis/enterprise-gis/geoevent
Real-Time & Big Data Technical Workshops
• ArcGIS Analytics for IoT: An Introduction• ArcGIS GeoEvent Server: Working with Real-Time Services• ArcGIS GeoEvent Server: Applying Real-Time Analytics• The Road Ahead: Real-Time & Big Data GIS• Real-Time & Big Data GIS: Best Practices• ArcGIS GeoEvent Server: Creating Connectors & Processors
using the GeoEvent Server SDK
Comments?Questions?
Eric IronsideArcGIS GeoEvent Server
Product [email protected]
Brett PhillipsArcGIS GeoEvent Server
Software [email protected]
https://community.esri.com/community/gis/enterprise-gis/geoevent
