Upload
tibco-spotfire
View
1.442
Download
0
Embed Size (px)
DESCRIPTION
TIBCO StreamBase: Real-Time Analytics: The Last Mile in Big Data Richard Tibbetts, CTO, TIBCO StreamBase ANALYTICS AND VISUALIZATION FOR THE FINANCIAL ENTERPRISE CONFERENCE June 25, 2013 The Langham Hotel Boston, MA
Citation preview
1
Enabling Real Time Analytics in the Capital Markets
Richard Tibbetts
CTO, TIBCO StreamBase
Real Time Analytics: The Last Mile in
Big Data
2
Real Time Visibility: The Last Mile in Big Data
The Last Mile:
Delivery to
usage site
• Expensive
• Purpose Built
• Non-Optional
3
Big Data: Everyone Talks about Volume
In the past 10 years, the amount of
data to be processed has exploded.
Mobile devices, sensors, market
data, video, audio, and social media
are among the sources of this
explosion of data.
8
Closing the loop – creating value with big data
Accumulate – Capture and store the data
• Use the right technology to capture data as it happens (e.g. Flume)
• Load it into appropriate price/performance storage (e.g. HDFS)
Analyze – Form and test hypotheses
• Discover mathematical relations and patterns (e.g. R)
• Crunch enormous volumes of data to validate and tune (e.g. Hadoop)
Access – Use the knowledge at the point of decision
• Get the data to the people who need it, when they need it
• Present it in the way that they can consume it
Automate – Optimize even small decisions with data
• Control systems with a closed feedback loop
• Monitor and focus on the exceptions, automate the common actions
10
StreamBase EventFlow - Integration, Analytics, Reaction
Visual programming integrates components developed by team members with different skills
• Developers, Quants, Data Scientists, Traders, Business Analysts
Connectivity
Data Infrastructure
Quantitative Analytics
Audit Logging
Business Strategy
Off the Shelf Components
11
Applications Integration
Normalization Aggregation Correlation
Rules Alerts
Automation
Adapters and
Handlers
Adapters and
Handlers
StreamBase Server(s)
StreamBase Studio for Developing EventFlow Applications
Data Management
Persistence Stores
Logs
Market Data
Trade Data
Static Data
Systems Data
Performance Benchmarks
Automation
Desktop
Alerts
StreamBase Platform for Complex Event Processing
Inputs Outputs
12
StreamBase Studio: Eclipse Visual Development
Visual operators
Algorithm library
Schemas, threading, etc.
Event Application Code
13
StreamBase Studio: Agile Development, Mixed Teams
Feed Simulation
Refactoring
Seamless Java & visual EventFlow Debugger
Visual Diff/Merge
Unit Testing
Java Operators & Function Wizards
14
API’s
C++, Java, .NET, Python
Messaging
TIBCO RV & EMS, JMS, MQ Series, RMDS, Solace, Tervela, Wombat, 29West
Additional General Purpose Adapters
FIX, STAMP, JDBC, POP3, IRC, HTTP, SMTP, IM, XML, RSS, Twitter
StreamBase Connectivity and Visualization Options
Market Data/Venue Samples
Visualization
Eclipse RCP
Adobe Flex
Java Swing
.NET
MS Excel
SL RTView
Databases
TR Velocity
15
Kosmos: StreamBase for High Frequency Trading
Who is Kosmos?
• One of Brazil’s leading hedge funds which employs both quantitative and qualitative investment strategies
StreamBase for HFT
• Connectivity to wide selection of equities and FX data sources
• Building next generation trading system
• Equities, FX and derivatives
• Selected StreamBase over other CEP providers
Why StreamBase?
• Rapid time-to-market
• Flexibility
• Developer empowerment
16
SUNGARD: Pre-Trade Risk Solution
Business Driver:
• Fast Response to Regulatory Change
Project:
• Enhancement of Assent Liquidity Solutions and Brass Order Management System
Asset classes:
• Equities
• Derivatives
• Fixed income
Why StreamBase?
• Maturity of StreamBase’s platform and success in the CEP market
• Graphical event-flow language and visual environment that enable developers and business users to work more efficiently
• Superior performance to reduce latency issues
“CEP... is very good for rapid development of applications which are deployed in high-volume, low-latency type environments.
The products offered by Assent are constantly being challenged with lots of market data, orders and execution updates. We want to use StreamBase to augment and enhance some of the products that we have.”
Chris Lees, VP Liquidity Services, SunGard
17
BNY ConvergEx: StreamBase for Monitoring
BNY ConvergEx: The Convergence of Brokerage and Technology
• Top 10 NYSE liquidity provider
• Vortex, an electronic ATS / dark pool
• A leader in algorithmic trading and OMS technology
• Deployed StreamBase for monitoring of U.S. trading operations
BNY ConvergEx Deploys StreamBase for U.S. Trading Operations
• Intelligent monitoring of trading operations, including its dark pool Vortex
• Real-time market interaction monitoring
• Real-time credit risk correlation
• System health monitoring
• Real-time analysis of market feeds for latency and response time
“The implementation of StreamBase allows us to monitor our risk… and assess the health of our entire U.S. trading operations in real-time.” Joseph Weisbord, Managing Director, BNY ConvergEx Group
18
CityIndex: StreamBase for Options Pricing and Monitoring
Who is CityIndex?
• Leading global provider of retail trading services, including CFDs, margined FX and spread betting
StreamBase for Pricing and Monitoring
• Real-time access to prices
• Rapid application development and deployment
Why StreamBase?
• Rapid time-to-market
• High performance
• Ease-of-use
Watch what Toby Young has to say about StreamBase
(Requires 2D barcode scanner or click to view)
19
Internet
Hadoop
MR
MR
MR
MR
MR
MR
MR
MR
MR
MR
MR
MR
Map Reduce
StreamBase CEP
Monitoring
Automation
Filtering
Alerting
Real-Time Applications
Data Warehouse
Hadoop Flume MySQL
HDFS
Analytic
Analytic
Analytic
Analytic
Operational Monitoring
LiveView
Historical Analytics
Analysts
Operational Staff
Fraud Analysts
Fraud detection
Fraud, Security
Management Control
Site Metrics
Bu
sin
ess I
nte
llig
ence
Real Time Big Data: Virtual Goods Phishing and Fraud
21
Example End User Experience
Real-Time Graphs
Active Tables
Ad-hoc query
Notifications
End-user configurability
22
StreamBase LV Use Case - Real-Time Risk Management
Real-time exposure by asset class
Real-time transaction detail
Real-time exposure by desk
23
EventFlow and R: Pre-Deal Real Time VAR Checks
Normalize
Integrate with R Simulations
Loop each trade through all scenarios
Calculate per-trade, per-scenario Risk
Calculate Risk: 95%,99%,X%
Accept/Reject proposed trade, minimize risk to
counterparty
Aggregate
Configure parameters for back testing
Connect
24
StreamBase LiveView Use Case - Real-Time TCA
Real-Time TCA Alerts
Live Orders, TCA, FIX
Which orders are slipping?
25
LiveView Server – Data Source and Query Clients
Message Bus
Market Data
News
Orders &
Executions
Data
LiveView Server
Snapshot Results
Continuous Query Processor
Query
Continuous Alerting
Active Tables Active Tables
Continuous Updates
Sources Clients
Maintain Data Image
Respond to Continuous Queries
Materialize analysis
Pre-aggregate data
Manage alerts
27
GAIN Capital: FX Pricing, Risk, Surveillance and Compliance
GAIN Capital
• Improved risk management, surveillance and compliance system
• Rapid applications design, development and deployment
• Faster execution
GAIN’s developers
• Intuitive development environment
• Library of application examples and help files
• Responsive support team
GAIN’s Forex.com clients
• Tighter spreads for customers through improved pricing and market making
StreamBase/GAIN Institutional Clients
• Rapid FX applications design, development and deployment
• Allow developers to focus on strategies and not the plumbing
• High frequency trading connectivity and order execution to all FX liquidity providers including tier 1 and tier 2 banks
“StreamBase Event Flow programming language is 50% faster than traditional programming techniques.” Andrew Haines, CIO, GAIN Capital
Watch what Andrew Haines has to say about StreamBase on TABB Forum
(Requires 2D barcode scanner or click)
28
Summary and Next Steps
Once you have accumulated, analyzed, and accessed big data you will have insight
Operationalizing insight on real time data is Event Processing
StreamBase’s Event Processing Platform is best of breed
• Natural and powerful EventFlow authoring environment
• Extensible integrations with analytics
• High performance deployment server
• Rich portfolio of connectivity with focus on capital markets
• LiveView for operational monitoring and alerting
Download and try out http://streambase.tibco.com
Questions?