1

I am running a big job on a HPC. But the installed programs for the job are old versions and I have the new versions in my home directory. I can log in to any node, export the path to the new versions using export PATH=/home/bharat/scratch/bin/:$PATH. However I am not sure if I export the path in my login shell and submit a job using bsub, then will the newly exported path be used by all the nodes that are running the job.

I can copy these files in one of the default include directories but my sysadmin is not too eager to do it for me soon.

What is the best way now to export the path to all nodes? Can I write a shell script with the export command + the job command and pass this script to bsub?

WYSIWYG
  • 201

1 Answers1

1

What is the best way now to export the path to all nodes? Can I write a shell script with the export command + the job command and pass this script to bsub?

Yes. Write a script (say its named myscript) that both sets the PATH and then starts your program. Then submit your job with the command line bsub myscript. Your local installation will be used rather than the system installation.

An even simpler option, LSF will set the environment variables of the submission environment into the execution environment, including PATH.

[mclosson@hostA ~]$ echo $PATH
/lsf/9.1/linux2.6-glibc2.3-x86_64/etc:/lsf/9.1/linux2.6-glibc2.3-x86_64/bin:/bin:/usr/bin
[mclosson@hostA ~]$ bsub -m hostB -Is 'echo $PATH'
Job <217> is submitted to default queue <interactive>.
<<Waiting for dispatch ...>>
<<Starting on hostB>>
/lsf/9.1/linux2.6-glibc2.3-x86_64/bin:/lsf/9.1/linux2.6-glibc2.3-x86_64/etc:/bin:/usr/bin

Now update your path so that it includes the local installation of your program.

[mclosson@hostA ~]$ export PATH=/tmp/xxx:$PATH
[mclosson@hostA ~]$ bsub -m hostB -Is 'echo $PATH'
Job <218> is submitted to default queue <interactive>.
<<Waiting for dispatch ...>>
<<Starting on hostB>>
/lsf/9.1/linux2.6-glibc2.3-x86_64/bin:/tmp/xxx:/lsf/9.1/linux2.6-glibc2.3-x86_64/etc:/bin:/usr/bin
Michael Closson
  • 221