Embed Size (px)
Citation preview
1HeMoLab - http://hemolab.lncc.br
Porting HeMoLab's SolverGP
to EELA glite Grid Environment
FINAL REPORT
Ramon Gomes Costa - [email protected]
Paulo Ziemer - [email protected]
2HeMoLab - http://hemolab.lncc.br
Just Remembering ...
● HeMoLab is a software used to generate simulation model files which are used as input by SolverGP (Fortran application) that computes the simulation and produces its result.
HeMoLab ↔ SolverGP
3HeMoLab - http://hemolab.lncc.br
Achievements
● SolverGP was executed successfully on processes distributed among different machines - MPICH2:
– 10 processes in 4 different machines in the site from
Venezuela (~6 hours of running time and DataOut.txt
returned as OutuputSandBox with 223MB).
● SolverGP's input files are now being sent to SE (clear copy) and after this to the WN.
● MyProxy is being delegated in order to execute jobs that could last more than 12 hours.
4HeMoLab - http://hemolab.lncc.br
Achievements
InputFiles.tar.gz (scr)SolverGP.xifortlibs.tar.gz
UI SE WNSolverGP.sh
InputFiles.tar.gz (scr)SolverGP.xifortlibs.tar.gz
DataOut.txt.tar.gzDataOut.txt.tar.gz
WatchDogSecureStorage-Client
1 3
2
45
5HeMoLab - http://hemolab.lncc.br
Achievements
● UI's local script (localScript.sh) is used to send the required files to SE
– localScript.sh jobid <filename>
● The script submits the job and monitors its execution until “done
status”. The script returns the job execution output files.
6HeMoLab - http://hemolab.lncc.br
Using SecureStorage
● Secure send the files from UI to SE (Encryption is performed locally at UI)
● Secure storage client was installed at UI
● Secure storage can be also installed only in user area from UI
tar -zxvf securestorage-client.tar.gz
su
cp securestorage-client/* /opt/lgc/
7HeMoLab - http://hemolab.lncc.br
Using SecureStorage
● local script at UI: Secure Sending input files to SE
lcg-scr -v --vo $PROD -d $STORAGE_ELEMENT --vo_permission /C=BR/O=ICPEDU/O=UFF\ BrGrid\ CA/O=LNCC/OU=CCC/CN=Ramon\ Gomes\ Costa -l lfn:/grid/$PROD/hemolab/InputFiles2.tar.gz file://$PWD/InputFiles.tar.gz
8HeMoLab - http://hemolab.lncc.br
Using SecureStorage
● If lcg-del lfc-rm commands are used with secure stored files a consistency problem between SE, File Catalog and “Secure Catalog” is generated.
● If a new secure copy is made with the same name of deleted file (lcg-del and lfc-rm) a error is generated and the copy operation is aborted:
– Keystore: A key for this LFN just exists
● To erase “secured files” lcg-sdel
9HeMoLab - http://hemolab.lncc.br
Using SecureStorage
● SE to WN
echo "*******downloading input files: " >> ./log.txtlcg-scp -v --vo $PROD lfn:/grid/$PROD/hemolab/InputFiles2.tar.gz file://$PWD/InputFiles.tar.gz >> ./log.txt
# criar .tar.gz do Dataout.txt que esta no WNtar cvzf DataOut.tar.gz DataOut.txt
#copiar DataOut para SElcg-cr --vo $PROD -l lfn:/grid/$PROD/hemolab/DataOut.tar.gz -d $STORAGE_ELEMENT file://$PWD/DataOut.tar.gz
10HeMoLab - http://hemolab.lncc.br
Using SecureStorage
● Secure storage client is not installed in WN from Venezuela
● So we have to send the secure client as inputSandBox and install it in $USER area at the WN.
echo "***instaling securestorage-client***" >> ./log.txttar -xvzf securestorage-client.tar.gz >> ./log.txtexport PATH=$PWD/bin/:$PATH >> ./log.txtexport LD_LIBRARY_PATH=$PWD/lib/:$LD_LIBRARY_PATH ./log.txt
11HeMoLab - http://hemolab.lncc.br
Using WatchDog
● WatchDog was successfully integrated with SolverGP script:
watchdog.conf
# Put here below the list of file you want to checkFILES2CHECK=( \log.txt \run.parallel.log \)
# Absolute path into the catalogWD_CATALOGPATH="/grid/${WD_VO}/hemolab"
# SE to use (leave it empty for default SE)WD_SE=grid008.cecalc.ula.ve
12HeMoLab - http://hemolab.lncc.br
Using WatchDog
# iniciando o watchdog# prepare and start the watchdogchmod +x watchdog.* >> ./log.txt./watchdog.ctrl start >> ./log.txt#uuencode/uudecodeexport PATH=$PATH:.
# mpr ring establishment
mpirun -np ....
#stop and wait the watchdog completes./watchdog.ctrl stop >> ./log.txt
13HeMoLab - http://hemolab.lncc.br
Using WatchDog
● solver.jdl[Type = "Job";JobType = "Normal";CpuNumber = 2;Executable = "/bin/bash";Arguments = "SolverGP.sh";StdError = "log.txt";InputSandbox = {"SolverGP.sh","wd_E2GRIS2/watchdog.sh","wd_E2GRIS2/watchdog.ctrl","wd_E2GRIS2/watchdog.conf","wd_E2GRIS2/uuencode","wd_E2GRIS2/uudecode"};OutputSandbox = {"run.parallel.log", "log.txt"};Requirements = Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment) && other.GlueCEInfoHostName != "gridgate.cs.tcd.ie";MyProxyServer = "px.eela.ufrj.br";]
14HeMoLab - http://hemolab.lncc.br
Using WatchDog
20091008122953 wd> exec tail run.parallel.logCommand 'tail run.parallel.log' successfully sentoutput file: '091008130403_wdcli_cmd3.out'136 KSP Residual norm 7.191886228430e-04137 KSP Residual norm 6.924502918218e-04138 KSP Residual norm 6.707716766324e-04139 KSP Residual norm 6.487745907036e-04140 KSP Residual norm 6.311729108481e-04141 KSP Residual norm 6.166899267342e-04142 KSP Residual norm 6.023739729675e-04143 KSP Residual norm 5.869267747671e-04144 KSP Residual norm 5.732280310883e-04145 KSP Residual norm 5.609710310899e-04error file: '091008130403_wdcli_cmd3.err'20091008123609 wd>
15HeMoLab - http://hemolab.lncc.br
Todo
● To use glite C++ API● To generate full static version of SolverGP (no
extra libs would be needed)● To compile SolverGP with mpich1● To run some larger cases (that take more than 12
hours of computer running time)● To explore DIRAC capabilities
16HeMoLab - http://hemolab.lncc.br
Acknowledgments
Tutor: Eduardo Murrieta León (UNAM, Mexico)
Rafael Silva (UFCG, Brazil)
Elisa Ingrà (INFN, Italy)
Vanessa Hamar (IN2P3, France)
Leandro N. Ciuffo (INFN, Italy)
17HeMoLab - http://hemolab.lncc.br
Thanks!!!
Ramon Gomes Costa
Paulo Gustavo Portella Ziemer
