DFN Grid certificates and membership to VO bwGRiD

The Baden-Wuerttemberg Grid (bwGRiD) as part of the D-Grid  initiative provides  more  than 11.000 cores installed at 7 universities and institutions throughout Baden-Wuerttemberg. The main objective of the Grid is to provide extensive computational power for scientific research and industry. For a detailed introduction to the hardware provided at the site at Ulm and a listing of contributors involved in the bwGRiD project have a look at our bwGRiD page.

In order to access the Grid system you have to be a member of a university situated in Baden-Wuerttemberg. If so you may apply for a so called "DFN Grid user certificate" and for a membership of the virtual organization (VO) bwGRID, both of which are required to proof your authorization when accessing the Grid system. The following sections cover the rather sophisticated steps needed to be taken in order to attain the mentioned certificate and VO membership. The description is for the Firefox browser and only valid for members of Ulm University.

It is important to follow the instructions step by step due to interdependencies between sections.

Section I: Application for an DFN Grid user certificate

A certificate is kind of a digital passport signed by an authority (your certificate authority) that authenticates the owner of this particular passport/certificate. So if you show your certificate to somebody else and that person trusts the signing authority, this person knows for sure who you are. Such a certificate always consists of a private and a public key. The private key has similarities to a real world key, while you can imagine the public key to be some kind of a corresponding lock. Everybody can access the public key without any restrictions. By signing something with your private key, everybody else can verify your identity by checking your sign with the public key. Hence it is of utmost importance to keep your private key secret. Firefox utilizes a master keystore password to protect your private keys. It will ask for your keystore password when you try to access the private keys in the store. We strongly suggest to use a strong password to ensure a high level of protection for your private keys.

1. First of all you should enter the Preferences of Firefox (Edit => Preferences, Firefox 3.5: Tools => Options) and activate the "[v] Use a master password" checkbox under the "Security" tab. In addition some features require activation of the "[v] Java" and "[v] JavaScript" checkboxes (Mac/Firefox3.6: Only JavaScript available) below "Content" tab of menu "Edit => Preferences, Firefox 3.5: Tools => Options".

2. Use Firefox to visit the DFN Grid user certificate page and accept the unsigned certificate of that page: 

   • Firefox 2.x: Just click "OK".
   • Firefox 3.x: Click the "Or you can add an exception" link in the lower part of the white box, then the "Add Exception" button followed by the "Get Certificate" and "Confirm Security Exception" buttons.
   • Firefox 3.5+: Since the Grid root certificate is not part of Firefox, one always has to add a security exception as described at "Firefox 3.x".

3. When you are at the DFN page "Willkomen zur DFN-PKI", first select tab "CA-Zertifikate" and there the sub-tab "Grid Root CA Zertifikate". This should install the Grid Root certificate in your browser. Thereafter select tab "Zertifikate" and sub-tab "Nutzerzertifikat" from the menu. The displayed page should look similar to the figure shown below (please click the image to enlarge - for details see text below image):

7. Print the form and fill it with your gender (1), where "Herr" means male and "Frau" female. Put in your passport identification data at (2): In case of a passport you have to write "Reisepass" and in case of an ID card "Personalausweis" and the corresponding number as stated in your passport / ID card (if your are not sure just leave the line blank and fill it out when you visit your RA). Append the name of your institute after "Universitaet Ulm" in line "Abteilung / Institut" (3) - students add the word "student" in the line. Please add your work address at (4) - students use their private address instead. Finally put in the town and date at (5) and sign the document at (6).

8. Bring the application, along with your valid ID card to your local Grid registration authority (RA). Please arrange for an appointment with a member of staff from your Grid RA to avoid inconveniences. Grid RA contact information can be found in the gray box on the right hand side of this page's upper part. If you own an old Grid certificate and your new Grid Certificate has a different DN (see line "Eindeutiger Name"), please inform the operator about that change.

Section II: Certificate import (Firefox)

After your identity has been verified at your RA you will receive an email containing information about your signed public key and how to fetch it. This email is created automatically by the DFN. It is not sent by us.

The Firefox Browser offers a convenient method to securely store private keys with the help of a so called "Certificate Manager". A "Master Password" ensures that your certificates are safe from illegitimate access. If you have not already set a strong "Master Password" in your Firefox browser we strongly suggest, that you do so now. (The steps involved into setting such a password are explained in previous section).

1. Before you proceed please select
   Firefox-Menu: "Edit",
   Submenu: "Preferences" ("Firefox-Preferences" window opens),
   Tab: "Advanced",
   Sub-Tab: "Encryption"
   and activate the check box "(x) Ask me every time". This will help to prevent some problems we faced, while testing this procedure.

2. If the root certificate is still missing in your Firefox browser, you can install it by copying the first link of the email "1. Für die CA-Zertifikate wählen Sie bitte die Seite ..." into your Firefox browser and clicking on the Sub-Tab "Grid Root CA Zertifikat".

3. To import your new certificate into Firefox's certificate manager use the second link situated after "2. Ihr eigenes Zertifikat erhalten Sie direkt über folgenden Link:" in the email from your RA. At the page the link refers to, click on "Zertifikat importieren" and -- depending on your Firefox settings and version -- the following dialog will either appear or remain absent.

   "[x] Dieser CA vertrauen, um Webseiten zu identifizieren."
   "[x] Dieser CA vertrauen, um E-Mail-Nutzer zu identifizieren."
   "[x] Dieser CA vertrauen, um Software-Entwickler zu identifizieren."

   If the dialog appears make sure that all boxes are checked and proceed. Typically when the former dialogbox does not appear, a warning message will be displayed telling you that the downloaded private key has been installed and you should consider to make a backup.

   "Warnung: Ihr persönliches Zertifikat wurde installiert. Sie sollten eine Sicherungskopie dieses Zertifikats aufheben."

   Though this dialog is a warning everything is ok.
   

4. As you will need a backup copy of your certificate in section IV you should follow Firefox's advice and create one now. This can be done in the "Certificate Manager" which can be found via
   Firefox-Menu: "Edit",
   Submenu: "Preferences" ("Firefox-Preferences" window opens),
   Tab: "Advanced",
   Sub-Tab: "Encryption"
   Button: "View Certificates" ("Certificate Manager" window opens),
   Tab: "Your Certificates".
   If everything went well so far your name should be listed here. Simply select the line containing your name and click on Button "Backup" or "Export" (depending on Firefox version). Choose the location where the backup file shall be created and enter your Master Password into the respective dialog box. The last dialog asks you for a password for the backup. All these passwords are a nuisance but if you choose a bad password for this backup your private key is at stake. So you should once more think of a strong password and remember it. The password quality meter in the lower part of the dialog box is a hint on the strength of the password you made up. Finally click on "OK" and the backup file (PKCS12 format) will be created at the previously chosen location.

If your original private key is lost either by a hardware failure or due to faulty operation you will have to repeat the previous and following steps to apply for a new one. Hence it is sensible to save the backup at a secure place. Consider for example burning a CD containing your key.

Section III: Enrollment into virtual organization bwGRID

Please register your VO-membership only once. For renewal after 12 month please read section VIII.

1. If you prefer not to be asked to manually select a certificate about ten times during the following steps you should reactivate the "Select one automatically" check box, you deactivated in section II paragraph i. Nevertheless the automatic function sometimes raises trouble -- especially when the certificate manager is responsible for more than one certificate -- so it may be unavoidable to use the "Ask me every time" alternative and manually select your DFN-certificate.

2. Follow the link to the bwGRiD Virtual Organisation registration page at the Forschungszentrum Jülich. Follow this link and accept the server certificate of the remote server. Furthermore Firefox will ask you for your user grid certificate installed in the Firefox keystore. Usually it is sufficient to confirm each of the questions by simply hitting "OK".

7. Fill that form completely (you realy have to fill all fields and have to take care that your answers are the same as those you gave in the DFN certificate application from Section I).

   Your email address has to be entered into (1) and in (2) you are supposed to select your representative, in case of Ulm University this is usually Christian Mosch. At the "Personal Information" paragraph you have to enter your givenname (3) and surname (4) as well as your phone number your nationality (6, e. g. Spanish), followed by the street where your workplace is situated (7, e. g. Alber-Einstein-Allee 11), your faculty (8, students specify "student"), the respective zip code (9) and corresponding placename (10) and finally the country (11) of your workplace. Pressing the "Submit" button (12) will submit the application to a server.

8. After a while you will again receive an email containing a confirmation link.

9. Simply follow the provided link to a page allowing you to apply to VO group memberships. Here you may only select the boxes labeled "/bwgrid" and "/bwgrid/uniulm". Make sure that all other boxes are unchecked! Now you should take a glance at the terms of use available under the link "the Grid and VO AUPs", approve those terms by checking the corresponding box and finally submit the data.

10. It may take some days before you receive two emails, one confirming your VO membership to "/bwgrid" and the second your enrollment to sub-VO "/bwgrid/uniulm".

11. Even after you have received these emails it may still take a while till the news of your membership is spread among the Grid and hence till you are actually able to use your certificate in the following section.

Section IV: Certificate import for usage with Globus at the KIZ

1. In order to proceed you have to copy the key backup file you generated in the previous section into the home directory of your kiz-Unix-Account. If you lack such an account you can apply for one here. When you used an official kiz linux pool computer to perform the preceeding steps everything should be fine and there is no need for further actions.
   

   In case the file is stored on a Unix flavored (Linux, Mac OS X, SunOS) computer, open a terminal and enter (where DFN-user-certificate.pkcs12.p12 has to be replaced with the actual name of your backup file and username with your unix account username):

   scp DFN-user-certificate.pkcs12.p12 username@zeus.rz.uni-ulm.de~/

   After hitting enter you will be asked for the password of your account and finally the file will be copied. (Zeus is just one of a great number of interactive login servers, which can interchangeably be used as the target of your copy.)

   When you are on a Windows computer you have to install extra software (for example WinSCP or pscp), in order to copy the files.

2. Now you have to convert your private key into the so called pem-format and copy it into the .globus directory. Therefore see the following steps: (Line breaks in the following grey boxes containing the commands are due to automatic word-wrapping and you should enter the commands on a single line.)

   1. Log into a computer at our kiz linux pools and open a terminal window (press "Alt" + "F2" on your keyboard and enter "konsole" followed by the "enter" key into the pop up window appearing) or remotely connect to a public kiz unix computer with:

      ssh -l username zeus.rz.uni-ulm.de

      (The ssh command is only available under Unix flavored OS. In case of Windows you have to install another third party tool, e. g. putty.)

   2. Convert your private key into a format suitable for further processing by entering into your terminal window:

      openssl pkcs12 -in DFN-user-certificate.pkcs12.p12 -out userkey.pem -nocerts

      You will now be asked for two passwords. The first one you have to supply is that utilized to protect the backup and the second one is needed to secure the key stored in the so called "pem"-file that will be generated.

   3. Similar to the private key of the previous step the public key is also converted into the pem-format with:

      openssl pkcs12 -in DFN-user-certificate.pkcs12.p12 -out usercert.pem -clcerts -nokeys

      Where you only have to supply the backup password since the public key does not need any protection.

      This approach acts on the assumption, that private as well as public key are stored within the backup file. If you had problems storing both keys in one file (e. g. the file only contains the private key part) it is possible to use the pem-file that is available at the link given in the DFN confirmation mail as public key (usercert.pem) directly.

   4. Create a directory ".globus" in your home directory and move both pem-files into this directory issuing the commands:

      mkdir -vp ~/.globus;
      mv -v userkey.pem usercert.pem ~/.globus/

   5. Though the private key is secured with a password you should nevertheless adjust the access rights of public and private key with:

      chmod -c 700 ~/.globus;
      chmod -c 600 ~/.globus/user*

   6. You can delete your backup file now by command:

      rm -vf DFN-user-certificate.pkcs12.p12

Section V: Certificate testing

• When your VO registration has been completed and has been spread among the Grid (after receiving the conformation mail this will last about one more day) you can verify if the certificate is installed correctly with the following procedure. When you face problems during the execution of the following steps please have a look at the Problems and solutions section.

• Log into a Linux computer at a kiz Linux pool or remotely log into a server using ssh. To enable the globus system (required to work with bwGRiD) you have to activate the Globus commands by loading the corresponding software module:

  module load system/globus

• Now you may generate a temporary certificate (commonly refered to as proxy certificate) with:

  grid-proxy-init -debug -verify

  This proxy certificate is required to authenticate you with the Grid system. Its validity period is confined to 12 hours. It should also be mentioned that each temporary certificate will only work on the same computer it was created on. E.g. if you use the generic login.rz.uni-ulm.de address to log into your kiz account via ssh, you may want to use a fixed server address (e. g. corona.rz.uni-ulm.de) instead.

• Useful information about your current proxy certificate is provided by command:

  grid-proxy-info

• Finally by entering command

  gsissh -x -p 2222 koios.rz.uni-ulm.de

  you can access koios without any username or password. If you are asked for a username or password, please check the "Problems and solutions section" for help.

  Since the grid system in Ulm is up to now quite experimental, the koios server my be down from time to time. If you can not establish a connection to koios in Ulm please try to use a server in Stuttgart with:

  gsissh -x -p 2222 gridway.dgrid.hlrs.de

  If this does not work either please have a look at the "Problems and solutions section".

• To test if the globus-system actually works you may now start a job issuing the following command (when you are loged into a globus server):

  globusrun-ws -submit -s -Ft PBS -c /bin/hostname

  For details about the queueing system in Ulm please read the message automatically displayed after log in to koios or use command "man bw-grid" later (only available on koios).

• You can upload files to our Globus server via command:

  gsiscp -P 2222 my-local-file koios.rz.uni-ulm.de:

  To download a file from a remote Globus server to your local disk, please use a command like:

  gsiscp -P 2222 koios.rz.uni-ulm.de:my-remote-file ./

  Please note that the port specification -P uses a capital P. In contrast to this, the gsissh command uses a lowercase p.

• Since 2010 there is a global 128TB+256TB storage area in Karlsruhe. Only certificate based access is possible (after calling "grid-proxy-init") and only members of VO bwGRiD can access the storage. The 128TB home area (~/) is part of the Backup in Karlsruhe. The 256TB work area (~/work/) should be used to store scratch files. You can upload a file to the work area in Karlsruhe via command:

  gsiscp -P 2222 my-file bwgrid-se.scc.kit.edu:~/work/

  You can download a file from the work area in Karlsruhe by command:

  gsiscp -P 2222 bwgrid-se.scc.kit.edu:~/work/my-file ./

  Please note that the hostname "bwgrid-se.scc.kit.edu" is redirected automatically to one of 6 Globus frontend server. Therefore the central storage area should be highly available. Additional access possibilities are "gsissh", "globus-url-copy" and "uberftp". See also the output of command

  module help system/globus

  and the central storage documentation at the bwGRiD website for more details.

Section VI: Problems and Solutions

If the certificate based gsissh access to the Globus servers in Ulm or Stuttgart does not work without providing a password several reasons may apply:

• The server you are trying to reach is down (e. g. due to maintainance or malfunction). You can try to reach the servers with the ping command from your command line

  ping koios.rz.uni-ulm.de

  if you want to check for the server in Ulm. To check the server in Stuttgart use:

  ping gridway.dgrid.hlrs.de

  If the servers do not respond you can be assured that failing to establish a connection is caused by the Grid and not your fault.

• Did you execute "grid-proxy-init" command before "gsissh"?

• Did you execute "grid-proxy-init" on the same machine?

• Is there an entry in the "~/.ssh/config" file defining a fixed username? Such an entry would look similar to:

  Host *
  User dagobert

  saying that the username for all hosts shall be "Dagobert". Remove those entries from the "~/.ssh/config" file.

• Your VO registration may not yet be completed. Please complete this registration first.

• It takes at least one day till your VO data is spread among the Grid. So you can log in at the earliest one day after you have received the two confirmation mails concerning your VO membership.

• Please verify at https://vomrs.zam.kfa-juelich.de:8443/vo/bwgrid/vomrs that your VO membership to "/bwgrid" and "/bwgrid/uniulm" have an "Approved" status. This information can be found by clicking on the "[+]" sign of "[+] Member Info" followed by a click on "Select Groups & Group Roles". If your status does not change to "Approved" within some days you should contact your local Grid-RA. (Contact information at the right upper part of this page)

• You may also have a deprecated Chain-of-Trust or it may not be found on your system. If calling "grid-proxy-init" results in an output similar to:

  grid_proxy_init.c:1079:globus_credential: Error verifying credential: Failed to verify credential
  globus_gsi_callback_module: Could not verify credential
  globus_gsi_callback_module: Could not verify credential: unable to get issuer certificate
  globus_gsi_callback_module: Can't get the local trusted CA certificate: Cannot find trusted CA certificate with hash 34f8e29c

  contact your local RA. (Contact information at the right upper part of this page)

Abschnitt VII: Your own Globus/GSI-SSH system

1. If you prefer to use your own hardware with Globus you have to download, compile and install Globus (version 4.0.8) on your own. You further have to install the openssl package. While openssl is included into most Linux distributions it sometimes has to be installed manually. We are not able to offer you support for the operation of your own Globus system.

   Windows as well as Linux users may also use the Java based GSISSH-Term. In the case of GSISSH-Term openssl is also needed in order to convert the keys. We also do not offer support for GSISSH-Term.

2. After installing your own Globus system (version 4.0.8) or GSISSH-Term you have to download and install a so called "Chain of Trust". The basics for your "$GLOBUS_LOCATION/share/certificates/" or "~/.globus/certificates/" folder respectively may be downloaded from here:

   http://www.grid.lrz.de/res/globus/certificates.tar

   Since the certificates from this archive are sometimes deprecated you may also need to download some updates from:

   http://dist.eugridpma.info/distribution/igtf/current/accredited/tgz/

3. When everything is set up correctly you should be able proceed analogue to Section IV.

Section VIII: Renew your DFN Grid Certificate and your VO-membership (once each year)

Renewing the VO-membership

Twelve month after your VO bwGRiD registration you will get an automatic email "Your VO membership will expire". If you want to renew your VO-membership, please follow the instructions in that email:

• Therefore click on "re-sign" or "renew" in that email and authenticate yourself with your grid certificate as stored in the Firefox keystore. Hint: If you have not got an email with the link, the steps i to v of section III will guide you to the VO registration area.
• Navigate to page "Re-sign the Grid and VO AUPs" (see picture on the right). Hint: The "Member Info" area can only be expanded by clicking on the corresponding "[+]" sign.
• Please first read the "Grid and VO AUPs" by clicking on the corresponding link.
• Thereafter you can click on the "I have read and agree ..." button.

Renewing the DFN Grid user certificate

Twelve month after creating your DFN Grid certificate you will get an automatic email "Ihr GridGermany Zertifikat wird ablaufen" which points out that your certificate will expire soon. If you want to renew your certificate please follow the instructions found in sections I, II and IV within this text. To renew the certificate it has to be recreated and you have to prove your identity again.

Please keep in mind that the DN (among other things your name and your department) of the new certificate is identical with the DN of your former certificate. Obviously this makes sense only if you are still working in the same department. In this case the VO registration of section III must NOT be repeated. Only re-signing the "Grid and VO AUPs" is required as described above in this section.

On the other hand please report changes of your DN (e.g. you're now a member of a new department and not a student anymore) during your identification appointment. We will then ask the VO-registration center (vomrs-admin(at)fz-juelich.de) to modify your VO-bwGRiD group membership accordingly.

The Grid RA of Ulm University has no immediate influence on the contents of these emails. They are created automatically by the DFN servers.