View
7
Download
0
Category
Preview:
Citation preview
© 2013 Azul Systems 1 1
Why Java Works (Well) for
Low-Latency Trading
Systems
Howard Green
Azul Systems Inc.
© 2013 Azul Systems 2 2
Who we are
© 2013 Azul Systems 3 3 3 3
Azul Systems: Deep Java DNA
• Award-Winning Leader in Java Technology
• Founded in 2002, privately held; based in Sunnyvale, CA
• Trusted partner to the financial sector for over ten years
• Azul CTO Gil Tene on the JCP Executive Committee
• Unique technology and IP in Java runtimes and performance ─ Over 40 patents issued, 15 additional filed
• Deep low-latency Java expertise
• Strong financial partner & ISV ecosystem
© 2013 Azul Systems 4 4
Azul: Proven in Capital Markets
Java use cases:
• HFT and Algo Trading
Platforms
• FX, Fixed Income
• Real-time Risk
• Hedge Funds (CEP)
• DMA, Matching
Engines, FIX engines,
Messaging, BI, & more
• Low-latency Java is
here – and improving
© 2013 Azul Systems 5 5 5 5
So…
Why do people use Java for
low latency apps?
Java in the low latency world?
© 2013 Azul Systems 6 6 6 6
Are they crazy?
© 2013 Azul Systems 7 7 7 7
Are they crazy?
• No.
• There are good, easy to articulate reasons:
• Projected lifetime cost
• Developer productivity
• Time-to-product, Time-to-market, ...
• Leverage, ecosystem, ability to hire
© 2013 Azul Systems 8 8 8 8
Why use Java in Algo Trading?
• One customer’s perspective:
“Our strategies have a shelf life”
We have to keep developing and deploying new ones
Only one out of N is actually productive
Profitability therefore depends on ability to successfully deploy new strategies, and on the cost of doing so
Our developers seem to be able to produce 2x-3x as much when using a Java environment as they would with C/C++ ...
© 2013 Azul Systems 9 9
Some Low-Latency
Java Examples
© 2013 Azul Systems 10 10 10 10
FX Example: LMAX Exchange
• London-based FX exchange
• Primarily Open Source infrastructure
• 3 ms average latency; 100% Uptime, 500 usec internal latency
• LMAX Disruptor: 6 million TPS in 2011
“LMAX Exchange are already the fastest venue for FX and
our deep technological collaboration with Azul will help to
ensure we continue to deliver even faster execution for our
clients in the future."
-- Edward McDaid, CIO, LMAX Exchange
© 2013 Azul Systems 11 11 11 11
Algo Example: OptionsCity
• Java-Based (Some Scala) ─ Quoting and execution
─ Options pricing and modeling
─ Algo trading
─ Integrated risk management
• Co-location next to matching engines at key exchanges
• Under 100 µsec round trip
© 2013 Azul Systems 12 12
What’s Driving Java LL
Performance?
© 2013 Azul Systems 13 13 13 13
5 Things People Think They Know about Java
1. Java is inherently slower
2. My team of C and C++ developers can write faster code
3. Java isn't for low-latency applications
4. Garbage Collection makes Java impractical for Trading
5. Java heap sizes over 10 GB are asking for trouble
© 2013 Azul Systems 14 14 14 14
Is Java Slow?
No • A good programmer will get roughly the same speed
from both Java and C++
• A bad programmer won’t get you fast code on either
• The 50%‘ile and 90%‘ile are typically excellent...
© 2013 Azul Systems 15 15 15 15
Tiered Compilation
• Today’s C1 and C2 compilers generate high-quality code
• C1: “Fast and Dumb” – 5X-10X interpreter speed ─ Smaller code
─ Lower-quality
• C2: Slower and Smarter – 30-50% faster than C1 ─ Bigger (aggressive/ more inlining)
─ Much higher peak throughput
• Tiered (default) behavior – Both are used ─ C1 for fast startup
─ C2 for peak performance
• Ensure sufficient iterations for optimization
© 2013 Azul Systems 16 16 16 16
Memory Management
• Garbage Collectors are getting better
• CMS can be very good in tuned applications
• Some Jrockit features being added to Oracle HotSpot
• Traditional GC target peaks in low milliseconds
• Object allocation much faster than malloc
• Can drive peak latencies far lower
• With aggressive JVM tuning and optimized application
• Tens of microseconds peak latency is the rough current state of the industry
© 2013 Azul Systems 17 17 17 17
Thread handling
• Concurrency has been key to Java since its inception
• Moore’s law / human capabilities
• Accelerates developer productivity
• Asynchronous development simplified
• Very fast, low overhead
• Originally depended upon operating system
• Major advantage of Java – the JVM does more
© 2013 Azul Systems 18 18 18 18
“Mechanical Sympathy”
• cf: Martin Thompson’s blog
• Modern Intel-architecture servers have lots of cores
• …And very good cache designs
• Cache miss – “lost opportunity for 500 instructions”
• Linux optimizations can help tremendously
• Power management can generate more jitter than GC does
• Designers that understand how to keep key data in cache…
• Locking threads to particular sockets (or cores)
• Designing for few (1) writers and many readers
© 2013 Azul Systems 19 19 19 19
Better Open Source/ More Profiling Tools
• Use of Open Source throughout the financial sector has become widespread
• Driven by replatforming onto Linux
• Good frameworks and other OSS initiatives help jumpstart development
• Java profiling tools are extremely robust ─ AppDynamics
─ New Relic
─ Jprofiler
─ VisualVM
─ Many, many more
© 2013 Azul Systems 20 20 20 20
Human Capital is Key
• There’s a worldwide war for talent
• “The New Kingmakers” – James Governor of RedMonk
• You need to provide the best tools and most interesting challenges for your developers
• And you’d rather use your best and most creative people to build new functionality and enable new offerings, instead of tuning glitchy legacy systems
• Java helps keep your best people engaged in building your business
© 2013 Azul Systems 21 21
Any issues for Java in
Trading Systems?
© 2013 Azul Systems 22 22 22 22
• Example: some Java-based systems look like this:
Traditional JVM: Typical case is fine; worst case more than 20x too high
Java in Today’s Electronic Markets
© 2013 Azul Systems 23 23 23 23
Is “jitter” even the right word for this?
99%‘ile is ~60 µsec
Max is ~30,000%
higher than “typical”
© 2013 Azul Systems 24 24 24 24
What do actual L-L developers do?
• Today…
• They use “Java” instead of Java
• They write “in the Java syntax”
• They avoid allocation as much as possible
• E.g. They build their own object pools for everything
• They write all the code they use (no 3rd party libs)
• They train developers for their local discipline
• In short: They revert to many of the practices that hurt productivity. They lose out on much of Java.
© 2013 Azul Systems 25 25
<And a very brief commercial>
Summing Up…
© 2013 Azul Systems 26 26 26 26 Low latency trading application
Real-world Java Performance
© 2013 Azul Systems 27 27 27 27 Side-by-side – Unmodified Low latency App - Drawn to scale…
Standard JVM
(pure newgen) Zing JVM
© 2013 Azul Systems 28 28 28 28
The Zing JVM eliminates the SLA issues and performance glitches people spend weeks (or even months) trying to solve
Zing delivers lower averages, eliminates latency spikes
30-50 sec peaks in highly tuned applications
•
Java for Today’s Trading Systems
© 2013 Azul Systems 29 29 29 29
In one slide…
Real Java is finally viable for low-latency applications
Garbage Collection is no longer a dominant issue
…Even for outliers
No need to code in special ways any more
You can finally use “real” Java for everything
You can finally 3rd party libraries without worries
You can finally use as much memory as you want
You can finally use regular (good) programmers
©2013 Azul Systems, Inc. 30
Azulsystems.com/solutions/onWallSt
Azulsystems.com/solutions/low-latency
Azulsystems.com/solutions/big-data
www.azulsystems.com/resources
LinkedIn: /company/azul-systems
Twitter: @azulsystems
FaceBook: AzulSystemsInc
Email: hgreen@azulsystems.com
Phone: +1 650 230 6616
Got Questions? Just Ask Us
For more information:
© 2013 Azul Systems 31 31 31 31
Everybody knows Java tuning can be tough
java -Xmx12g -XX:MaxPermSize=64M -XX:PermSize=32M
-XX:MaxNewSize=2g -XX:NewSize=1g -XX:SurvivorRatio=128
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:MaxTenuringThreshold=0 -XX:CMSInitiatingOccupancyFraction=60
-XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly
-XX:ParallelGCThreads=12 -XX:LargePageSizeInBytes=256m
java –Xms8g –Xmx8g –Xmn2g -XX:PermSize=64M -XX:MaxPermSize=256M
-XX:-OmitStackTraceInFastThrow -XX:SurvivorRatio=2
-XX:-UseAdaptiveSizePolicy -XX:+UseConcMarkSweepGC
-XX:+CMSConcurrentMTEnabled -XX:+CMSParallelRemarkEnabled
-XX:+CMSParallelSurvivorRemarkEnabled
-XX:CMSMaxAbortablePrecleanTime=10000
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=63 -XX:+UseParNewGC
–Xnoclassgc
Two real-world Java tuning settings. And they contradict each other.
© 2013 Azul Systems 32 32 32 32
…Except with Zing
java –Xmx40g
Just pick a good starting point and let Zing tune itself
Now you can focus more on new capabilities
And keep your best talent developing instead of tuning
Recommended