Contents
Objective
This page is aimed to explain how site services should be configured to support the main and backup VOMS and LFC IBERGRID services.
Introduction
VOMS and LFC grid services are single point of failure objects in the infrastructure. However, redundancy is obtained through a Master-Slave arquitecture. Masters are hosted at LIP, and Slaves are hosted at IFCA. Slaves keeps a read-only copy of the Master MySQL databases, are notified once there is a change on the Master, and synchronize automatically on the fly.
The main advantage of this setup is that it guarantees that users can continue to use the infrastructure even if the Master servers are down. The drawback is that the Slaves do not allow adding new information to the databases.
Users can only take advantage of this setup if it is properly configured at the different sites.
VOMS Redundancy
Main VOMS: voms01.ncg.ingrid.pt
Backup VOMS: ibergrid-voms.ifca.es
Yaim setup
In the example bellow we use the ict.vo.ibergrid.eu example. However, the same procedure should be applied for all IBERGRID Macro VOs, taking care that the port number is different for each VO.
Include the following definitions under your site-info.def (or alternatively under the vo.d/ict.vo.ibergrid.eu). The YAIM variables of interest are VOMSES and VOMS_CA_DN.
- VOMSES="'ict.vo.ibergrid.eu voms01.ncg.ingrid.pt 40008 /C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=voms01.ncg.ingrid.pt ict.vo.ibergrid.eu' 'ict.vo.ibergrid.eu ibergrid-voms.ifca.es 40008 /DC=es/DC=irisgrid/O=ifca/CN=host/ibergrid-voms.ifca.es ict.vo.ibergrid.eu'"
- VOMS_CA_DN="'/C=PT/O=LIPCA/CN=LIP Certification Authority' '/DC=es/DC=irisgrid/CN=IRISGridCA'"
- Run YAIM on your services. You should apply these changes on all site services requiring or enabling authentication (UI, CE and SE)
#/opt/glite/yaim/bin/yaim -c -s site-info.def -n gLite-NODETYPE
Cross Checking
Login on your service, and check that the proper lsc files were properly created. Case you do not want to execute YAIM, you can implement these changes by hand:
# ll /etc/grid-security/vomsdir/ict.vo.ibergrid.eu/ total 8 -rw-r--r-- 1 root root 89 Apr 28 15:39 ibergrid-voms.ifca.es.lsc -rw-r--r-- 1 root root 99 Apr 28 15:39 voms01.ncg.ingrid.pt.lsc # cat /etc/grid-security/vomsdir/ops.vo.ibergrid.eu/ibergrid-voms.ifca.es.lsc /DC=es/DC=irisgrid/O=ifca/CN=host/ibergrid-voms.ifca.es /DC=es/DC=irisgrid/CN=IRISGridCA # cat /etc/grid-security/vomsdir/ops.vo.ibergrid.eu/voms01.ncg.ingrid.pt.lsc /C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=voms01.ncg.ingrid.pt /C=PT/O=LIPCA/CN=LIP Certification Authority
Special Notes
gLite-WMS and gLite-FTS still do not support the .lsc mechanism. For those services, the .pem certificates of the two IBERGRID VOMS servers have to be installed under /etc/grid-security/vomsdir
gLite-UI Testing
Check that during several attempts to generate proxies from your UI, both different VOMS servers are tried.
- Proxy generated using main VOMS server
$ voms-proxy-init --voms ict.vo.ibergrid.eu Enter GRID pass phrase: Your identity: /C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=Goncalo Borges Creating temporary proxy .................................. Done Contacting voms01.ncg.ingrid.pt:40008 "ict.vo.ibergrid.eu" Done Creating proxy .............................................. Done Your proxy is valid until Fri Jul 22 00:17:26 2011
- Proxy generated using backup VOMS server
$ voms-proxy-init --voms ict.vo.ibergrid.eu Enter GRID pass phrase: Your identity: /C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=Goncalo Borges Creating temporary proxy ................................... Done Contacting ibergrid-voms.ifca.es:40008 "ict.vo.ibergrid.eu" Done Creating proxy ........................ Done Your proxy is valid until Fri Jul 22 00:17:31 2011
gLite CreamCE testing
The following examples demonstrate how to probe CreamCE authentication using proxies from both VOMS servers.
CreamCE is accepting proxies issued by the main VOMS server
$ voms-proxy-info -acissuer /C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=voms01.ncg.ingrid.pt $ glite-ce-allowed-submission ngiescream.i3m.upv.es:8443 Job Submission to this CREAM CE is enabled
The same CreamCE is NOT accepting proxies issued by the backup VOMS server
$ voms-proxy-info -acissuer /DC=es/DC=irisgrid/O=ifca/CN=host/ibergrid-voms.ifca.es $ glite-ce-allowed-submission ngiescream.i3m.upv.es:8443 2011-07-21 17:31:50,901 ERROR - = = Description=[User C=PT,O=LIPCA,O=LIP,OU=Lisboa,CN=Goncalo Borges not authorized for operation getServiceInfo] =[User C=PT,O=LIPCA,O=LIP,OU=Lisboa,CN=Goncalo Borges not authorized for operation getServiceInfo] Timestamp=
SE Testing
The following examples demonstrate how to probe SRM authentication using proxies from both VOMS servers.
- The SE does allow to store files from users arriving with proxies issued by the main VOMS server
$ voms-proxy-info -acissuer
/C=PT/O=LIPCA/O=LIP/OU=Lisboa/CN=voms01.ncg.ingrid.pt
$ export LFC_HOST=lfc01.ncg.ingrid.pt
$ lcg-cr -v --vo ict.vo.ibergrid.eu -d ngiesse.i3m.upv.es -l lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges
/teste-`date +%s`.txt file:/home/ingrid/csys/goncalo/test_proxy.jdl
Using grid catalog type: lfc
Using grid catalog : lfc01.ncg.ingrid.pt
Checksum type: None
SE type: SRMv2
Destination SURL : srm://ngiesse.i3m.upv.es/dpm/i3m.upv.es/home/ict.vo.ibergrid.eu/generated/2011-07-21
/fileccb89a7d-362a-4695-ba09-90ce7373a78d
Source SRM Request Token: bfaffc51-e4f4-4703-ab80-67676a348aa8
Source URL: file:/home/ingrid/csys/goncalo/test_proxy.jdl
File size: 259
VO name: ict.vo.ibergrid.eu
Destination specified: ngiesse.i3m.upv.es
Destination URL for copy: gsiftp://ngiesse.i3m.upv.es/ngiesse.i3m.upv.es:/storage/ict.vo.ibergrid.eu
/2011-07-21/fileccb89a7d-362a-4695-ba09-90ce7373a78d.1435741.0
# streams: 1
259 bytes 0.43 KB/sec avg 0.43 KB/sec inst
Transfer took 1040 ms
Using grid catalog type: lfc
Using grid catalog : lfc01.ncg.ingrid.pt
Site URL to be registered: srm://ngiesse.i3m.upv.es/dpm/i3m.upv.es/home/ict.vo.ibergrid.eu/generated
/2011-07-21/fileccb89a7d-362a-4695-ba09-90ce7373a78d
File size: 259
Using LFN: lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311271148.txt
Using GUID: guid:9bbf75e5-52f8-41b8-a218-7e1e0cfd0fe6
Registering LFN: /grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311271148.txt (9bbf75e5-52f8-41b8-
a218-7e1e0cfd0fe6)
Registering SURL: srm://ngiesse.i3m.upv.es/dpm/i3m.upv.es/home/ict.vo.ibergrid.eu/generated/2011-07-21
/fileccb89a7d-362a-4695-ba09-90ce7373a78d (9bbf75e5-52f8-41b8-a218-7e1e0cfd0fe6)
guid:9bbf75e5-52f8-41b8-a218-7e1e0cfd0fe6The same SE does NOT allow to store files from users arriving with proxies issued by the backup VOMS server
$ voms-proxy-info -acissuer /DC=es/DC=irisgrid/O=ifca/CN=host/ibergrid-voms.ifca.es $ export LFC_HOST=lfc01.ncg.ingrid.pt $ lcg-cr -v --vo ict.vo.ibergrid.eu -d ngiesse.i3m.upv.es -l lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges /teste-`date +%s`.txt file:/home/ingrid/csys/goncalo/test_proxy.jdl Using grid catalog type: lfc Using grid catalog : lfc01.ncg.ingrid.pt Checksum type: None SE type: SRMv2 Destination SURL : srm://ngiesse.i3m.upv.es/dpm/i3m.upv.es/home/ict.vo.ibergrid.eu/generated/2011-07-21 /file874b2650-5f4c-4d71-a52e-81feac743389 httpg://ngiesse.i3m.upv.es:8446/srm/managerv2: CGSI-gSOAP running on ui01.ncg.ingrid.pt reports Error reading token data header: Connection closed Source SRM Request Token: Internal error lcg_cr: Communication error on send
LFC Redundancy
Unfortunately, the switch between main and backup LFC has to be done manually by the user.
Main LFC: lfc01.ncg.ingrid.pt
Backup LFC: ibergrid-lfc.ifca.es
Let us consider the following use case to demonstrate how to use the backup LFC service
- Copy and register a file in the master LFC
$ export LFC_HOST=lfc01.ncg.ingrid.pt
$ lfc-ls /grid/ict.vo.ibergrid.eu/goncalo_borges
NG-5252_S-143.tar
out-NG-5252_S-143-run_kmer23_cofcutoff7_expcov13_inslength1-15072011-201925.tar
out-NG-5252_S-143-run_kmer24_cofcutoff7_expcov13_inslength1-15072011-201052.tar
$ lcg-cr -v --vo ict.vo.ibergrid.eu -d srm01.ncg.ingrid.pt -l lfn:/grid/ict.vo.ibergrid.eu
/goncalo_borges/teste-`date +%s`.txt file:/home/ingrid/csys/goncalo/test_proxy.jdl
Using grid catalog type: lfc
Using grid catalog : lfc01.ncg.ingrid.pt
Checksum type: None
SE type: SRMv2
Destination SURL : srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated/2011-07-21/file0ec1c54b-29ed-444f-
b728-97658b9f016b
Source SRM Request Token: d27db3bd-1960-45a5-9ced-19e82037b26b
Source URL: file:/home/ingrid/csys/goncalo/test_proxy.jdl
File size: 259
VO name: ict.vo.ibergrid.eu
Destination specified: srm01.ncg.ingrid.pt
Destination URL for copy: gsiftp://gftp01.ncg.ingrid.pt:2811//lustre/ncg.ingrid.pt/data3/ibergrid
/ict/generated/2011-07-21/file0ec1c54b-29ed-444f-b728-97658b9f016b
# streams: 1
259 bytes 1.61 KB/sec avg 1.61 KB/sec inst
Transfer took 1030 ms
Using grid catalog type: lfc
Using grid catalog : lfc01.ncg.ingrid.pt
Site URL to be registered: srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated/2011-07-21/file0ec1c54b-
29ed-444f-b728-97658b9f016b
File size: 259
Using LFN: lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt
Using GUID: guid:ca993392-7a08-4d85-81fa-681d15f8a66f
Registering LFN: /grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt (ca993392-7a08-4d85-81fa-
681d15f8a66f)
Registering SURL: srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated/2011-07-21/file0ec1c54b-29ed-444f-
b728-97658b9f016b (ca993392-7a08-4d85-81fa-681d15f8a66f)
guid:ca993392-7a08-4d85-81fa-681d15f8a66f- Check it if it is available in the backup LFC.
$ export LFC_HOST=ibergrid-lfc.ifca.es $ lfc-ls /grid/ict.vo.ibergrid.eu/goncalo_borges NG-5252_S-143.tar out-NG-5252_S-143-run_kmer23_cofcutoff7_expcov13_inslength1-15072011-201925.tar out-NG-5252_S-143-run_kmer24_cofcutoff7_expcov13_inslength1-15072011-201052.tar teste-1311267719.txt
- Try to retrieve same the file using the backup LFC
$ lcg-cp -v --vo ict.vo.ibergrid.eu lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt
file://`pwd`/teste2
Using grid catalog type: LFC
Using grid catalog : ibergrid-lfc.ifca.es
VO name: ict.vo.ibergrid.eu
Checksum type: None
Trying SURL srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated/2011-07-21/file0ec1c54b-29ed-444f-
b728-97658b9f016b ...
Source SE type: SRMv2
Source SRM Request Token: 25fd05dc-90fd-41c4-9e7e-0e165081c371
Source URL: /grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt
File size: 259
Source URL for copy: gsiftp://gftp01.ncg.ingrid.pt:2811//lustre/ncg.ingrid.pt/data3/ibergrid
/ict/generated/2011-07-21/file0ec1c54b-29ed-444f-b728-97658b9f016b
Destination URL: file:/home/ingrid/csys/goncalo/teste2
# streams: 1
0 bytes 0.00 KB/sec avg 0.00 KB/sec inst
Transfer took 1020 ms- Remember that you can delete the file from the SRM but you can not unregister any LFC entries using the backup LFC.
$ lcg-del -v -a lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt VO name: ict.vo.ibergrid.eu SE type: SRMv2 srm://srm01.ncg.ingrid.pt/ibergrid /ict/generated/2011-07-21/file0ec1c54b-29ed-444f-b728-97658b9f016b - DELETED $ lfc-ls /grid/ict.vo.ibergrid.eu/goncalo_borges NG-5252_S-143.tar out-NG-5252_S-143-run_kmer23_cofcutoff7_expcov13_inslength1-15072011-201925.tar out-NG-5252_S-143-run_kmer24_cofcutoff7_expcov13_inslength1-15072011-201052.tar teste-1311267719.txt $ lcg-lg lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt guid:ca993392-7a08-4d85-81fa-681d15f8a66f $ lcg-uf guid:ca993392-7a08-4d85-81fa-681d15f8a66f srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated /2011-07-21/file0ec1c54b-29ed-444f-b728-97658b9f016b srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated/2011-07-21/file0ec1c54b- 29ed-444f-b728-97658b9f016b: ibergrid-lfc.ifca.es: Read-only file system lcg_uf: Read-only file system
- Unregister files is only possible using the main LFC
$ export LFC_HOST=lfc01.ncg.ingrid.pt $ lcg-lg lfn:/grid/ict.vo.ibergrid.eu/goncalo_borges/teste-1311267719.txt guid:ca993392-7a08-4d85-81fa-681d15f8a66f $ lcg-uf guid:ca993392-7a08-4d85-81fa-681d15f8a66f srm://srm01.ncg.ingrid.pt/ibergrid/ict/generated /2011-07-21/file0ec1c54b-29ed-444f-b728-97658b9f016b
Index
No Pages to show