==== Passing environment variables to nodes ====

While some batch schedulers (e.g. Slurm) do some tricks to transparently pass some environment variables from the head node shell to all execution nodes given to ''mpirun'', OAR does not (OAR provides no more than what OpenSSH does, be it used directly as ''ssh'' or through the ''oarsh'' wrapper). Therefore, in order to have more than the default environment variables (''OMPI_*'' variables) passed/set on execution nodes, one can use the ''--mca mca_base_env_list "ENV1;ENV2;ENV3"'' option of ''mpirun'' (or ''-x ENV1 -x ENV2 -x ENV3'' depending on the Openmpi version) .

Example: assuming the ''MY_ENV1'' and ''MY_ENV2'' environment variables are defined in the shell calling mpirun, the value of ''MY_ENV1'' and ''MY_ENV2'' will be available on all execution nodes, along with ''MY_ENV3'' set to ''value3'' when calling mpirun as follows:

<code bash>mpirun -machinefile $OAR_NODE_FILE --mca orte_rsh_agent "oarsh" --mca mca_base_env_list "MY_ENV1;MY_ENV2;MY_ENV3=value3" ~/bin/test.sh</code>

Settings:
<code bash>
orte_rsh_agent="oarsh"
mca_base_env_list="MY_ENV1;MY_ENV2;MY_ENV3=value3"
</code> 
can also be set in the ''~/.openmpi/mca-params.conf'' file, so that the command becomes:
<code bash>mpirun -machinefile $OAR_NODE_FILE ~/bin/test.sh</code>


This could especially be useful to pass OpenMP variables, such as ''OMP_NUM_THREADS''.



More info [https://www.open-mpi.org/doc/v4.0/man1/mpirun.1.php#toc22 in OpenMPI manual pages].