The new FTS proposalFTS status
EMI INFSO-RI-261611
30/05/2010*FTS 2.2.6
10/05/2010*30/05/2010*Bugs fixedSupport an SE publishing more
than one SRM endpointFTS kills transfer while dCache is computing
checksumMore
https://savannah.cern.ch/patch/?func=detailitem&item_id=4503New
featuresRestart / resume failed transfersRestart: GridFTP client
build-in plug-inResume: range markers are kept in memory, use them
if transfer fails to resumePrecondition: range markers MUST be sent
back from the GridFTP server
EMI INFSO-RI-261611
10/05/2010*FTS 2.2.730/05/2010*FTS 2.2.7New overwrite files
logicIf -o flag is usedCheck first if file exists If it exists,
delete itFinally, initiate the transferboth srm-copy and url-copy
supportedWorks with srm and gsiftp endpoints
EMI INFSO-RI-261611
30/05/2010*FTS 2.2.8New featureTransfer monitoringInstrument FTS
instances to report monitoring informationActiveMQ will be used2
types of messagesTransfer events (for every transfer)Queue status
reports (sent at regular time
intervals)https://twiki.cern.ch/twiki/bin/view/LCG/WLCGTransferMonitoring
EMI INFSO-RI-261611
Why new FTS?
Architectural limits reachedDifficult to maintain and
improveBased on static channel modelUses Oracle backendOverlaps
with other projects Requires complex configurationCould handle
resources better
EMI INFSO-RI-261611
Schedule + new features
Step 1: end of 2011Remove channel model, more transfer protocols
(HTTP), replace Java WS, new config, support MySQL?Prototype!Step
2: end of 2012Messaging, transfer optimizationLHC shutdown
startsStep 3: middle of 2013Merge FTS and LCG_UtilWe need to set up
priorities
EMI INFSO-RI-261611
Channel modelReflects hierarchical Tier modelNot scalable in
Tier2 domainStatic properties set by adminSE pairs, mutual
agreementChannel groups, star channel: still not optimalSE and
network load not taken into accountDropping channel model dropping
FTS architectureUsers welcome the idea
EMI INFSO-RI-261611
Channel model - solutionTake SE into accountNeed to discuss it
with other SE providers(CASTOR done)Take network load into
accountCollaboration with OSGStep 1: no channelsSE-s are configured
on FTS sideNetwork utilization measured by FTSStep 2: using the new
interfacesResource handling: heuristics based on performance of
last transfersUltimately, no configuration needed?Open questions.
Fair share: how? Control throughput?
EMI INFSO-RI-261611
Transfer protocols, WS
Currently, SRM + gridFTPLCG_Util supports moreStep 1: Transfer
pluginsSRM, gridFTPStep 2: HTTP, XRDCPC++ client Java web service
Duplicates, knowledge of Tomcat, etc. neededSolution: everything in
C/C++Step 1: FTS in C++, based on new generic componentsStep 2: No
web service (decision pending, still balancing
drawbacks/benefits)
EMI INFSO-RI-261611
Persistency, configuration
Using Oracle onlyProblem for Tier2 sites: no license, etc.FTS
deployed in T2-s is not a hot use case now - later?Step 1: FTS on
top of generic DB accessDatabase access pluginsMySQL, ProgreSQL
implementation if neededClear architectureSimpler schema (no
channels) Configuration goes to databaseSimple CLI, query and
(re)configure remotelyStep 2: Job queue based on messagingGlobal
FTS queue? Transparent FTS-es? Monitoring?Problem with
delegation
EMI INFSO-RI-261611
File Transfer Service now
EMI INFSO-RI-261611
New FTS
EMI INFSO-RI-261611
Resources
https://svnweb.cern.ch/trac/glitefts
Jean-Philippe Baud Oliver Keeble Zsolt Molnar Michail Salichos +
our students and short term visitors
