Upload
anatole-tresch
View
56
Download
0
Embed Size (px)
DESCRIPTION
Speak done at GeeCon and other Conferences in 2014 showing an experience report of some of the complexities to be faced when changing commonly used frameworks and runtime components in Credit Suisse and how issues were solved finally.
Citation preview
Anatole Tresch
Renewal of Legacy Frameworks
An Experience Report
GeeCon 2014
May 2014
2Renewal of Central Frameworks – An Experience Report May 2014
Bio
Anatole Tresch Consultant, Coach Credit Suisse Core Framework Engineer,
Technical Architect Specification Lead JSR 354 Driving Java EE Config
Twitter/Google+: @atsticks [email protected] [email protected] Java Config Discussion https://groups.google.com/forum/#!forum/java-config Java Config Blog: http://javaeeconfig.blogspot.ch Zurich Java Community (
https://plus.google.com/u/0/communities/106363890913137321158) Zurich Hackergarten (
https://plus.google.com/u/0/103936743237262644768/posts)
3Renewal of Central Frameworks – An Experience Report May 2014
Credit Suisse
Global Bank with Headquarter in Zurich, Clients in Private Banking, Investment Banking and Asset Management.
Shares as Credit Suisse Group AG (CSGN) in Switzerland(SIX) and as American Depositary Shares (CS) in New York (NYSE).
47,400 Employees. Ratings: Moody’s A2, Standard & Poor’s A, Fitch
Ratings A.
4Renewal of Central Frameworks – An Experience Report May 2014
Renewal of Central FrameworksAgenda
Where we started
Objectives and Constraints
Challenges
CS Java Application Platform
Implementation
Summary
5Renewal of Central Frameworks – An Experience Report May 2014
Where we started
6Renewal of Central Frameworks – An Experience Report May 2014
The Credit Suisse ITFrom a Bird’s View
On-, Offshore ca. 20’000+ Employees in IT
Multiple global distributed Operation Centers
Protocol-Mix: Corba, WS, Restful, RMI, …
Technology-Mix: Java, DotNet, PL/1, Oracle, Sybase,
MS SQL Server, DB2, IMS, Smalltalk, Python,
MS Access
In-House and COTS Solutions
Total > 1000 applications
7Renewal of Central Frameworks – An Experience Report May 2014
Java Core FrameworksDeployment
Global Deployment
Part of the Java Application Platform
Infrastructure (JAP)
2-3 Major Releases active in parallel
Delivery: 3 Jars + Scripts
Developer-Desktop: Windows
Production: Solaris/Linux
8Renewal of Central Frameworks – An Experience Report May 2014
Java Core Frameworks
Java Core FrameworksPositioning within the Software and Hardware Stack
Infrastructure-Code!
9Renewal of Central Frameworks – An Experience Report May 2014
1.5 Mio LOC
2 Mio LOC
1 Mio LOC
500k LOC
Java in Credit SuisseA typical Application-Stack
Clients
Entry
Host
Web
Business
CSIB (synchron)MQ Series
(asynchron)
WebWeb
Business
Employees
Web
BusinessBusiness
External Interfaces
COTS
BatchBatch
TA Monitoring
Web
EntryEntry
Web WebWeb
BusinessBusiness
Batch
Web
BusinessBusiness
Business
Core FrameworksColors = versions
10Renewal of Central Frameworks – An Experience Report May 2014
Java Core FrameworksThe Java Core Frameworks’s Functionality
Go for the money - JSR 354 - http://java.net/projects/javamoney 23th October 2013
Context Management (Tracing, Accounting)
Exception Propagation (incl. Trace-Context)
High Performance Logging, Audit Logging
Platform-Abstraction Layer
Configuration Management (Deployment, Application)
JMX Registration und Naming Services
i18n
Common Utilities
11Renewal of Central Frameworks – An Experience Report May 2014
Java Core FrameworksFunctionality of the Java Core Frameworks – a Simulation
Entry
Web
Business Business2 HostExternal
Service Bus
ContextContext
ExceptionContext
Error
Audit
Audit
Audit
Audit
Audit
Log
Log
Log
Log
LogLog
Log
MessageBus
Log
Audit
12Renewal of Central Frameworks – An Experience Report May 2014
Java Core FrameworksFrom Daily Business
Increasing Support Efforts
Non solvable Problems
Complicated, invasive APIs
Missing or complicated SPIs
Implementation- and Runtime Dependencies
Lack of Flexibility and Extendability
Slow, uncomprehensive Code
Missing, outdated or uncomprehensive documentation
…
13Renewal of Central Frameworks – An Experience Report May 2014
Quality IssuesAPI?
public interface Manageable {
/** * Invokes different methods in all classes that implement this * interface. What method actually is performed is determined by the * argument <B>cmd</B>. The Stringarray options contains the * arguments for invoking the method. * @param command * The method that should be invoked. * @param options * The command line arguments for the command. * @throws Throwable * If an error occurs. * @return The return value depends on the method that is invoked. */ public Serializable invoke(String command, String[] options) throws Throwable;}
14Renewal of Central Frameworks – An Experience Report May 2014
Quality IssuesSPI?
/** * Manages the storage of a {@link CoreContextHolder} object. Extending this * class, allows to support different storage behaviours. A CoreContextHolder * Store is used in {@link CoreContext}. Hint: Abstract class * is used to be visible package local only. */abstract class CoreContextHolderStore {
/** Returns the previous stored CoreContextHolder. */ abstract CoreContextHolder getCoreContextHolder(); /** Stores a CoreContextHolder. */ abstract void setCoreContextHolder(CoreContextHolder holder);
/** Factory to get CoreContextHolderStore instance. */ final static class Factory { enum Type { DEFAULT, SINGLETON, THREADLOCAL, WORKCONTEXT }; private final static Factory SINGLETON = new Factory();
/** type of factory. */ private Type instanceType = Type.DEFAULT; /** store created last time */ private CoreContextHolderStore storeCreated = null; static Factory getSingleton() { return SINGLETON; } …
15Renewal of Central Frameworks – An Experience Report May 2014
Quality IssuesComprehensivness/Complexity
public void loadDefaultFromRescource(String[] resources) throws ConfigException{ String environment = System.getProperty(PROPERTY_ENVIRONMENT); environment = ENVIRONMENT_MAP.get(environment); String rescourcePath = null; for (int i = 0; i < resources.length; i++) { rescourcePath = CONFIG_IN_JAR_FOLDERS_START + environment + CONFIG_IN_JAR_FOLDERS_END; try { loadDefaultFromRescource(rescourcePath + resources[i]); } catch (ConfigException e) { if("BASE0130".equals(e.getExceptionContext().getMessageID())) { rescourcePath = CONFIG_IN_JAR_FOLDERS_START + FOLDER_ENV_GLOBAL + CONFIG_IN_JAR_FOLDERS_END; loadDefaultFromRescource(rescourcePath + resources[i]); } } } }
16Renewal of Central Frameworks – An Experience Report May 2014
Quality IssuesOther Metrics
17’000 LOC / 8721 Statements
204 Typen, 1400 Methoden
23 Packet Cycles !
Strong Coupling
Big Artifacts (> 1500 LOC)
Only a Few Interfaces
No Annotations
Missing Information Hiding
…
17Renewal of Central Frameworks – An Experience Report May 2014
Where we startedConclusion
Signific. Issues
Widely Used
RISK!
18Renewal of Central Frameworks – An Experience Report May 2014
Objectives and Constraints
19Renewal of Central Frameworks – An Experience Report May 2014
Objectives and ConstraintsTechnical Objectives
Improved Stability, Maintainability and Extendibility
Improved Performance and Scalability
Improved Usability
Convention over Configuration
Easy Testing (independent tests!)
Reduction of API Footprint (decoupling)
Technical Innovation (IoC)
Maximal Backward Compatibility, smooth Migration Path
20Renewal of Central Frameworks – An Experience Report May 2014
Objectives and ConstraintsCompatibility
Backward Compatibility:
Context Propagation
Serialization (Context, Exceptions)
Build and behavioral Compatibility (API)
Reduced Compatibility Requirements on
Implementation Artifacts
Forward Compatibility:
Context Propagation
Serialization (Context, Exceptions)
21Renewal of Central Frameworks – An Experience Report May 2014
Objectives and ConstraintsOrganizational Objectives
Reduce Support Efforts
Simplify Framework Development
Continuous Build
(Semi-) Automatic Deployment
Building up Know How
22Renewal of Central Frameworks – An Experience Report May 2014
Objectives and ConstraintsChallenges
Quality Culture
Non-Standard Tool-Chain
Stakeholder Management
Time & Budget
Skills
23Renewal of Central Frameworks – An Experience Report May 2014
CS Java Application PlatformThe Environment
24Renewal of Central Frameworks – An Experience Report May 2014
Java Application Platform JAPIdea and Motivation
Applikation LogicGUI, Business Logic, DB Schemas, Configuration, etc.Infrastructure Design/ConfigurationHW, OS, Middleware, NetworkSystems ManagementOperating ManualsDevelopment Tools and ProcessesSecurity Services and ProcessesIntegration Services and Processes
Applikation Platform (AP)Optimized for similar applications,Based on Hosting PlatformsHosting PlatformsProvide Common Services• Computation CHP (Hardware, OS)• Persistence DHP (Databases)
25Renewal of Central Frameworks – An Experience Report May 2014
JAP Lifecycle ManagementWhy Lifecycle Management?
• Applications and Components without managed Lifecycle
• Uncoordinated technical Upgrades
• Incompatibilities
• Managed Lifecycle• Coordinated and tested
technical Upgrades (HW+SW!)
• Constant Decommissioning of old Platform Releases
• New Releases wont impact existing applications
26Renewal of Central Frameworks – An Experience Report May 2014
JAP Lifecycle ManagementRelease Planning
27Renewal of Central Frameworks – An Experience Report May 2014
Implementation
28Renewal of Central Frameworks – An Experience Report May 2014
ImplementationRelease Planning
Major Releases synchronous with JAP Platform
(periodically all 2-3 Years)
Patch Releases 2-3/Year
Emergency Releases as necessary
1 Major Framework Release =
Proposals, Requirements, Feature Selection beforehand
6 months Implementation and Component Tests
6 months Integration Testing and Documentation
29Renewal of Central Frameworks – An Experience Report May 2014
ImplementationSpecification/Design
High Level Design (Make or “Buy”)
Existing Functionality in Java EE 5/6, Weblogic?
Third Party / OSS Libraries? Functional Gap?
Low Level Design Minimize APIs, go for Effective Java Immutable Value Objects Thread-Save with Low Latency Multi-Tenancy, Multi-Language, Multi-Timezone etc. Decide on Adaption of existing APIs vs. Recreation and
Coexistence Consider Operational Aspects
30Renewal of Central Frameworks – An Experience Report May 2014
ImplementationTeam and Organization
Agile Process (no Offshoring)
2 Full-Time Programmers
Pair Programming as Needed
Hire the Best you get!
31Renewal of Central Frameworks – An Experience Report May 2014
ImplementationFurther Details
Extend and Improve Existing Tests
Improve/write Component Tests instead of Smoke Tests
Create Compatibility and Interop Tests
Ensure Independent Test Execution (Test Isolation)
Ensure Integration Tests (across multiple releases!)
Integrate early!
API
Deprecate old APIs (for at least(!) one Release)
Check Backward Compatibility with Sonar/clirr
continously
32Renewal of Central Frameworks – An Experience Report May 2014
ImplementationQuality Management
Static Code Analysis: Continuous and integrated, on each code change,
that compiled Based on Sonar, with customized Profile and Custom
Rules Coverage Analysis of Component Tests Basic Integrationstests Regular Check of Metrics, Alarms!
Peer-Reviews of all Code Changes Integration Tests with big Pilots
33Renewal of Central Frameworks – An Experience Report May 2014
ImplementationFramework Release Cycles
Similar to Platform Releases Playground 1 (first drop) Playground 2 (functional) Alpha (API stable) Beta (only patches) Release Candidate Final Release Patch Releases (regular updated) Emergency Releases (as needed)
34Renewal of Central Frameworks – An Experience Report May 2014
ImplementationFist Run 2010 - 5.0.0 -> 6.0.0
Partial API Renewal and API Reduction
Defined SPIs
Implementation Modularization and Encapsulation
(Spring/Lenz)
Focus on
Critical Functionalities
Stabilize Existing Code Base
Focus on Stability, reduce Complexity
3 Jars + Scripts-> Only Partial Renewal
-> significant Compatibility Breaks
-> Massive Performance Gains
35Renewal of Central Frameworks – An Experience Report May 2014
ImplementationSecond Run 2012 - 6.0.0 -> 7.0.0
HW/OS Change from Solaris to Linux Several small Improvements Fixing of Known Issues (sometimes with behavioral
changes!) Low Latency Requirements Introduction of a Runtime Platform Abstraction Layer Replacement of Spring with a leaner Container Introduction of Continuous Build based on Maven
7 Jars + Scripts QC with Sonar -> Renewal still ongoing
-> only a few compatibility breaks
-> New, efficient tool chain
36Renewal of Central Frameworks – An Experience Report May 2014
ImplementationThird Run 2013/2014 - v7.0.0 -> 8.0.0 (today)
Introduction of Java EE 6 (formerly EE 5)
Complex Integration of Low Level Features / CDI
New Annotation based API
Long Lasting Deprecations (New IOC Paradigma)
Renewal of Further Areas Untouched so far
Full CDI-Support -> Renewal Finished
-> only few Compatibility Issues
-> reduced API Footprint
(Annotations)
-> modern API (CDI)
37Renewal of Central Frameworks – An Experience Report May 2014
Conclusion
5.0 6.0 7.0 8.0
……..?
38Renewal of Central Frameworks – An Experience Report May 2014
ConclusionKey Factors
Functional and Effective Tooling
Continuous Build with Integrated Quality Measurement
Agil and Effective Collaboration Culture
Focus on task: minimize Maintenance for former Releases
Ensure enough Budget and Time
Define a realistic Scope
Managed Environment (Lifecycle and Platform
Management)
Use complex Test-Pilots early!
39Renewal of Central Frameworks – An Experience Report May 2014
ConclusionDid we met our Objectives?
Significant Reduction of Support Efforts
Establishment of Maintainability and
Extendibility
Improved Runtime Behavior (Performance,
Memory)
Reduction (Optimization) of API Footprint
Simplify Testing
Enable Technical Progress
Improved Quality
Backward Compatibility, if possible
Yes, but moreartifacts & LOC!
-50%
Yes.
100 x faster
Yes.
Yes EE6/7supported
-50%
mostly
SUCCESS!
40Renewal of Central Frameworks – An Experience Report May 2014
ConclusionActual Metrics
41Renewal of Central Frameworks – An Experience Report May 2014
ConclusionAPI Evolution
Configuration appCfg = ConfigHelper.getConfigManager() .getApplicationConfig();Map<String,String> config = null;if(appCfg.containsArea("a.b.c")){ config = appCfg.getArea("a.b.c", false);}
@Configured@ConfigRoot(value="a.b.c", recursive=false)@Optionalprivate Map<String,String> config;
OLD:
NEW:
42Renewal of Central Frameworks – An Experience Report May 2014
ConclusionAPI Evolution 2
private static final TraceLogger tl = TraceLoggerHelper.getTracLogger( getClass(), TraceType.SERVICE);
public void bookSettlement(Settlement s){ tl.entering(getClass(), "bookSettlement", s); // perform… tl.exiting(getClass(), "bookSettlement", s);}
OLD:
NEW:@TraceLogged(TraceType.SERVICE)public void bookSettlement(Settlement s){ // perform…}
43Renewal of Central Frameworks – An Experience Report May 2014
Q & A
???
44Renewal of Central Frameworks – An Experience Report May 2014
The End
THANK YOU!