Skip to main content



GLOBUS is a wide-area distributed computing system.

Versions and Availability

Module Names for globus on philip
Machine Version Module Name
None Available N/A N/A
▶ Module FAQ?

The information here is applicable to LSU HPC and LONI systems.


A user may choose between using /bin/bash and /bin/tcsh. Details about each shell follows.


System resource file: /etc/profile

When one access the shell, the following user files are read in if they exist (in order):

  1. ~/.bash_profile (anything sent to STDOUT or STDERR will cause things like rsync to break)
  2. ~/.bashrc (interactive login only)
  3. ~/.profile

When a user logs out of an interactive session, the file ~/.bash_logout is executed if it exists.

The default value of the environmental variable, PATH, is set automatically using SoftEnv. See below for more information.


The file ~/.cshrc is used to customize the user's environment if his login shell is /bin/tcsh.


Modules is a utility which helps users manage the complex business of setting up their shell environment in the face of potentially conflicting application versions and libraries.

Default Setup

When a user logs in, the system looks for a file named .modules in their home directory. This file contains module commands to set up the initial shell environment.

Viewing Available Modules

The command

$ module avail

displays a list of all the modules available. The list will look something like:

--- some stuff deleted ---

---------------- /usr/local/packages/Modules/modulefiles/admin -----------------
EasyBuild/1.11.1       GCC/4.9.0              INTEL-140-MPICH/3.1.1
EasyBuild/1.13.0       INTEL/14.0.2           INTEL-140-MVAPICH2/2.0
--- some stuff deleted ---

The module names take the form appname/version/compiler, providing the application name, the version, and information about how it was compiled (if needed).

Managing Modules

Besides avail, there are other basic module commands to use for manipulating the environment. These include:

add/load mod1 mod2 ... modn . . . Add modules
rm/unload mod1 mod2 ... modn  . . Remove modules
switch/swap mod . . . . . . . . . Switch or swap one module for another
display/show  . . . . . . . . . . List modules loaded in the environment
avail . . . . . . . . . . . . . . List available module names
whatis mod1 mod2 ... modn . . . . Describe listed modules

The -h option to module will list all available commands.

Module is currently available only on SuperMIC.


▶ Globus Tutorial?
== What do I need? ==

* Access to LONI Account.

* Active LONI Grid Certificate associated with your account.

* Active LONI Allocation for submitting and running compute
  jobs. (optional during testing and development of workflows)

== Where are the Globus services on LONI? ==

They are only available on 1 (i.e. qb1, eric1, etc)

== Do I need to setup Globus Environment on LONI? ==

Your Globus environment on LONI machines is already setup by

== Why Do I need a Globus Certificate? ==

Every user and service on the Grid is identified via a certificate,
which contains information vital to identifying and authenticating the
user or service.

A GSI certificate includes four primary pieces of information:

* A subject name, which identifies the person or object that the
  certificate represents.

* The public key belonging to the subject.

* The identity of a Certificate Authority (CA) that has signed the
  certificate to certify that the public key and the identity both
  belong to the subject.  * The digital signature of the named CA.

GSI certificates are encoded in the X.509 certificate format, a
standard data format for certificates established by the Internet
Engineering Task Force (IETF).

== How do I submit a Job using Globus? ==

Lets first visit the job submission command 'globus-job-run' and know
how it is run:

  $ globus-job-run -help

Read over the different options it presents for our target usage

Run a simple command to be executed on the target grid computing
service node:

 $ globus-job-run /bin/date
 Mon Mar  2 21:32:56 CST 2009

Now, lets create a script using our favorite editor with the following

 $ cat 
 #! /bin/sh
 awk "BEGIN {
   for (i = 0 ; i < $NLINES; i++ ) {
         print int(rand() * 10^$NDIGITS )

first lets change the script so that it can be used as an executable
using the command:

  $ chmod u+x 

now lets test the script using:

 $ ./ 4 7

Now, we are ready to submit this as our executable to be run remotely
on a target grid computing service node.

 $ globus-job-run -s 4 7

the -s switch is used to stage in the executable. Note the arguments
at the end of the command.

Now, lets try staging in the input file for an executable like 'cat'
be used to print out the contents.

 $ globus-job-run -stdin -s /etc/hosts -l /bin/cat       loopback        localhost       # loopback (lo0) name/address
 # Begin LONI Infrastructure

Here we are staging in the input file '/etc/hosts' from the submit
machine to the target machine where the local executable '/bin/cat' is
used to execute upon.

Now, if we want to get the output written into a file and brought back
to us, we can add some more options to the above command.

 $ globus-job-run -stdout -s hellohosts -stdin \
      -s /etc/hosts -l /bin/cat 
 $ ls hellohosts 
 $ rm hellohosts 
 $ globus-job-run -stdout -l hellohosts -stdin \
      -s /etc/hosts -l /bin/cat 
using the '-stdout' we can obtain the output in the form of a written
file. We can further specify whether you want to leave the output file
at the submission site or at the target compute site.

== Can I submit Parallel Jobs too? ==

 $ globus-job-run -np 8 -m 10 \
      -p loni_cybertools -q checkpt -stdout \
      -s helloparallel /bin/hostname

In the above command, we are running a serial executable in parallel
over Queenbee. We are requesting for 8 processors for a wall clock
time of 10 minutes using the allocation of 'loni_cybertools' to be run
in the 'checkpt' queue. We are specifying options to dump the output
in the file to be named as 'helloparallel'.

 $ cat helloworld.c 
 int main(int argc, char *argv[])
         int id, nprocs;
         MPI_Init(&argc, &argv);
         MPI_Comm_rank(MPI_COMM_WORLD, &id);
         MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
         printf("Hello world from %d out of %d\n", id, nprocs);

On compiling the parallel executable:

 $ mpcc helloworld.c -o helloworld

Submitting the parallel job from the command line:

 $ globus-job-run \
    -env "GBLL_NETWORK_MPI=sn_all,not_shared,US" \
    -x "&(jobType=mpi)(count=8)(maxWallTime=10)(queue=checkpt)\
    (stderr=/work/default/sirish/hello.err)(hostCount=1)" \
    -l /work/default/sirish/hello

On submitting the parallel job asynchronously:

 $ globus-job-submit \
    -env "GBLL_NETWORK_MPI=sn_all,not_shared,US" \
    -x "&(jobType=mpi)(count=8)(maxWallTime=10)(queue=checkpt)\
    (stderr=/work/default/sirish/hello.err) (hostCount=1)" \
    -l /work/default/sirish/hello \

On checking for the job status later:

 $ globus-job-status

== How do I move files using Globus? ==

Globus provides ''globus-url-copy'' which is a basic command to do
file transfer from source to destinations by urls.
$ globus-url-copy \
 -vb gsi \
 Source: gsi
 Dest:   gsi
     104857600 bytes        52.63 MB/sec avg        52.63 MB/sec inst

In the above command, we are transferring a file from one machine to
another while using the switch ''-vb'' that will show us more
output. For complete list of available options:

  $ globus-url-copy -help

Lets say  we test  the transfers using  files of different  sizes, say
100mb and 1gb each.

Creating each of the files:

100mb file:

 $ /opt/freeware/bin/dd count=100 bs=1024k if=/dev/zero \
 100+0 records in
 100+0 records out

1gb file:

 $ /opt/freeware/bin/dd count=100 bs=1024k if=/dev/zero \
 100+0 records in
 100+0 records out

 $ ls -lh *
 -rw-r--r--    1 sirish   sys          100M 2010-04-20 09:40 100mbfile
 -rw-r--r--    1 sirish   sys          100M 2010-04-20 09:40 1gbfile

Transferring the files on command line:

 $ globus-url-copy -vb file:///work/default/sirish/1gbfile \
 Source: file:///work/default/sirish/
 Dest:   gsi
     101711872 bytes        13.22 MB/sec avg        16.00 MB/sec inst

Improving performance using parallel streams:

 $ globus-url-copy -p 4 -vb file:///work/default/sirish/1gbfile \
 Source: file:///work/default/sirish/
 Dest:   gsi
     104857600 bytes        24.39 MB/sec avg        24.39 MB/sec inst

You can even batch transfers together. Using a file containing the
list of source and destination files. For example:

On aix machines:

 $ cat Xfertest.urls.1gb.aix 
 gsi \
 gsi \
 gsi \
 gsi \

On linux machines:
 $ cat Xfertest.urls.1gb.linux 
 gsi \
 gsi \
 gsi \
 gsi \
 gsi \

On using without parallel streams:

 $ globus-url-copy -vb -f Xfertest.urls.1gb
 Source: gsi
 Dest:   gsi
     104857600 bytes        18.87 MB/sec avg        10.83 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes        30.30 MB/sec avg        30.30 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes        22.73 MB/sec avg        22.73 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes         6.76 MB/sec avg         6.77 MB/sec inst

On using the parallel streams:

 $ globus-url-copy -p 4 -vb -f Xfertest.urls.1gb
 Source: gsi
 Dest:   gsi
     104857600 bytes        27.03 MB/sec avg        27.03 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes        47.62 MB/sec avg        47.62 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes        40.00 MB/sec avg        40.00 MB/sec inst
 Source: gsi
 Dest:   gsi
     104857600 bytes        62.50 MB/sec avg        62.50 MB/sec inst

See the performance improvement!!

===How do I use globus online to transfer files===

Globus Online(GO) manages file transfer for you, monitoring
performance, retrying failures, auto-tuning and recovering from faults
automatically where possible, and reporting status. And there’s no
custom infrastructure or software to install, so you don’t have to be
an IT wizard to use Globus Online.

As a user you need to follow the following steps to use globus online
to transfer files from LONI clusters

# Obtain a LONI Grid Certificate

# Sign Up for a Globus Online Account

# Install GridFTP or Globus on your server.
  ** LONI clusters already have Globus tools installed, so you can
     skip this step for transferring files between LONI clusters or
     between LONI and other resources which have globus installed

  ** To install GridFTP on your local machine, globus online provide a
     software [ Globus
     Connect] which will install gridftp and create a local endpoint
     for use with GO.

# Login to the head node where globus is installed e.g.,
  and create and store your grid credentials.

# Login to your GO account

# Click on "Start Transfer". You should see two windows side by side

# Select you endpoint for the machines that you want to transfer files
  between e.g. loni#qb on one window and your local machine on another

# Enter the password/passphrase requested # Select files/directories
  that you want to transfer and hit the transfer button (the

# You will receive an email when the transfer is complete.

To learn more about GO, look through the documentation on the GO

▶ LONI Certificates?

The LONI Certification Authority

LONI CA Certificates for the Users/Other Certification Authorities (CAs)

LONI Certification Authority (CA) provides the CA public certificates intended for Browser and Grid related use by the users. For example:

  • Users can have their LONI certificates trusted by other Certification Authorities(CAs).
  • Users can import LONI CA public certificate into their browser to validate the server certificates of the LONI websites to ensure security.
  • Other Certification Authorities can use the LONI CA revocation list to maintain an updated version of active LONI CA issued certificates

LONI CA (Root) Certificate

Following are the different formatted versions of LONI CA (Root) Certificate:
  • Privacy Enhanced Mail (pem)
  • Distinguished Encoding Rules (der)
  • Browser Importable (crt)
  • Hashed File format

Usage Scenarios

Importing LONI CA certificates into your Browser

For Internet Explorer

  • Click on link.
  • Click Open.
  • Click "Install Certificate..."
  • Click Next twice.

For Firefox:

  • Click on the browser importable format link for the LONI CA public/root certificate.
  • Check the boxes depending on your needs.
  • Click Ok.
Importing LONI CA Revocation list into your Browser

For Internet Explorer:

  • Click on link.
  • Click Open.
  • Click "Install Certificate..."
  • Click Next twice.

For Firefox:

  • Click on the browser importable format link for the LONI CA published revocation list.
  • You will see a message that LONI CA certificate revocation list has been successfully imported. You will be asked whether to enable automatic updates. Click Yes.
  • Check the enable automatic updates option for the certificate revocation list. Add the Url of the certificate revocation list and click ok.

Getting your LONI Grid Certificate mapped onto machines with other CAs

You have to supply your Distinguished Name (DN) of the globus certificate to the CAs of the machines where you want your certificate to be mapped. Provide this DN to the CA of the machine. In case the CA of the machine does not already trust LONI CA, you will have to provide the CA with LONI CA (root) certificate in the Hashed file format and the CA signing policy file in addition to your DN.

Adding LONI CA to your list of Trusted Certification Authorities

This means you want to add LONI CA to be trusted by your Globus installation. After configuring the Globus environment on your machine, add LONI CA Root certificate file in hashed file format and LONI CA Signing Policy file to the trusted certificates directory.

On an uncustomized Globus installation, this trusted certificates directory is usually /etc/grid-security/certificates.

Setting up LONI CA Published Certificate Revocation list in your Globus Installation

After adding LONI CA to your list of trusted certificates directory or if you have already done so, place the LONI CA revocation list in hashed file format along with LONI CA Root certificate. For this:

  • Download the file at the Url for the hashed file format.
  • Rename it as named in the Url.
  • move it into globus trusted certificates directory.

On an uncustomized globus installation, this trusted certificates directory is usually /etc/grid-security/certificates.

▶ Request LONI Cert?
== Login to one of the Grid Service Nodes ==

Login to one of the #1 headnodes on a LONI Cluster.

== Generating Globus Certificate Request ==

=== If this is your first request for a LONI Grid Certificate: ===

Then run the command:

 $ grid-cert-request
 A certificate request and private key is being created.
 You will be asked to enter a PEM pass phrase.
 This pass phrase is akin to your account password,
 and is used to protect your key file.
 If you forget your pass phrase, you will need to
 obtain a new certificate.
 Generating a 1024 bit RSA private key
 writing new private key to '/home/sirish/.globus/userkey.pem'
 Enter PEM pass phrase:

As the text above says, you need to choose a pass phrase for
activating the proxy from your certificate. This pass phrase is
used to secure your private key, so DO NOT leave it blank. Also, DO
NOT use your account password as the pass phrase.

Note: Your signed grid certificate is valid for six months. It is the
responsibility of the user to store/manage the pass phrase, and we
trust our users to take that responsibility seriously. Once a signed
certificate is issued to a user for the first time, no new certificate
will be issued to the same user within three months. If the new pass
phrase is ever lost again, the user must wait until the current one
expires. If a user renews an expired certificate and forgets the pass
phrase, then he or she must wait until the expiration as well. For
example, if a grid certificate is signed and issued to a user for the
first time on January 1, the earliest date for the user to receive a
new certificate is April 1. If the pass phrase is lost again within
six months, then the user must wait until October 1. So please choose
a secure pass phrase which is easy for you to remember or store it at
a safe place.
=== If this is not your first request for a LONI Grid Certificate: ===

Sometimes the users want to renew an expired certificate or request
for a new certificate. In such case use the -force switch in the

The -force switch is required on the machine where you already have an
existing .globus folder in your home directory.

 $ grid-cert-request -force

 $ $GLOBUS_LOCATION/bin/grid-cert-request -force

=== What is happening? ===

Once grid-cert-request is issued, the directory .globus will be
created in your home directory with three files: usercert_request.pem,
userkey.pem, and an empty usercert.pem.

==Sending LONI CA your certificate request==

You need to email the usercert_request.pem file to to
aquire a user certificate signed by the CA.

DO NOT email for anything else other than sending the
generated certificate request.

At this point a LONI administrator will take over, issue the
certificate, and normally, this is sent it back to the user by email.

==Installing your certificate==

After receiving the signed certificate, you need to copy it
to ~/.globus/usercert.pem. Then, run the command:

 $ grid-cert-info -s

[[Category:Grid Computing]]
[[Category:Globus Certificates]]

To check your grid certificate information. 
▶ Manage LONI Cert?

We assume that you already have an active globus certificate with the
LONI CA. If you do not have the LONI issued globus certificate yet,
please refer to Obtaining a LONI Grid Certificate to obtain one.

Once you have a valid LONI grid certificate, using the myproxy server
setup at, you can store the globus certificate
credentials to be made available for your applications.

First, we need to make sure you have myproxy tools available in your
path. If not already, login to a number 1 headnode for an already set
up globus environment.

Second, make sure that you have globus certificates directory on the
machine from which you are trying to store the credentials into

 $ ls .globus/
 job                   usercert_request.pem
 usercert.pem          userkey.pem

Optionally, you can verify if the certificate is valid or not using

 $ grid-proxy-init 
 Your identity: /C=US/O=Louisiana Optical Network \
    Initiative/ Tummala
 Enter GRID pass phrase for this identity:
 Creating proxy ...................................................... Done
 Your proxy is valid until: Thu Dec  4 01:33:01 2008

 $ grid-proxy-info 
 subject  : /C=US/O=Louisiana Optical Network \
    Initiative/ Tummala/CN=1150172559
 issuer   : /C=US/O=Louisiana Optical Network \
    Initiative/ Tummala
 identity : /C=US/O=Louisiana Optical Network \
    Initiative/ Tummala
 type     : Proxy draft (pre-RFC) compliant impersonation proxy
 strength : 512 bits
 path     : /tmp/x509up_u26027
 timeleft : 11:59:50

Now that you have setup a valid and working certificate, its time to
store it using myproxy tools in our credential management server at

 sirish@l3f1n03$ myproxy-init -s
 Your identity: /C=US/O=Louisiana Optical Network \
    Initiative/ Tummala
 Enter GRID pass phrase for this identity:
 Creating proxy ..................................... Done
 Proxy Verify OK
 Your proxy is valid until: Wed Dec 10 13:35:50 2008
 Enter MyProxy pass phrase:
 Verifying - Enter MyProxy pass phrase:
 A proxy valid for 168 hours (7.0 days) for user sirish now \
    exists on

The first password you enter during the above command is your LONI
grid certificate password. The second password is the one that 'you
will assign' for your applications to use so that they can retrieve
the proxy from the server.

If you want to verify the retrieval of your proxy from the credential
server, first destroy the generated proxy on the machine using


Then use the following command and enter the second password that you
assigned for the applications to use to retrieve.

 $ myproxy-logon -s
 Enter MyProxy pass phrase:
 A credential has been received for user sirish in /tmp/x509up_u26027.

By default, the proxy will be valid for 7 days on the credential
management server. If you would like to customize this behavior or
know about any more options that you might want to use, it can be
useful to read the immediate help available to the command.

 myproxy-init -help
 myproxy-logon -help

▶ Check LONI Cert Info?
== Certificate Information ==

You can view your grid certificate information using the command:

 $ grid-cert-info

This will display your entire certificate.

=== Distinguished Name (DN) ===

You can find your Distinguished Name (DN/Subject) only using:

 $ grid-cert-info -subject
 /C=US/O=Louisiana Optical Network \
     Initiative/ name

=== Issuer Hash ===

To find the issuer hash of your certificate:

 $ grid-cert-info -issuerhash

=== More Details ===

For more interesting details as how to use the above command to obtain
your certificate information:

 $ grid-cert-info -help

== Generating a valid proxy ==

Initializing your certificate into a working proxy can be done using
the command:

 $ grid-proxy-init
 Your identity: /C=US/O=Louisiana Optical Network \
    Initiative/ name
 Enter GRID pass phrase for this identity:
 Creating proxy .................................... Done
 Your proxy is valid until: Tue Mar  3 09:15:28 2009


Last modified: November 11 2014 16:48:02.