If HyperThreading is enabled in BIOS, using threads as the leafs of OAR (implicit) resources tree is quite simple.

The last version of the oar_resource_init script comes with the -H and -T options. It generates the oarnodesetting commands with the thread properties.

Here we consider an existing resource setup to which one would like to had the thread as leaf resources.

First you need to define the thread property:

$ oarproperty -a thread

Assuming your cluster is homogeneous, and that resources were consistently created (i.e. the ids of the properties of the resources are well set), you can add to every resource lines the property: thread = (2 * core - 1) (oarnodesetting -p … ), and then for each of them create a new resources (oarnodesetting -a …) with exactly the same values for all properties, except thread = (2 * core) and cpuset = (original cpuset value + # cores per cpu) (using hwloc in a job afterward to verify the correct cpuset settings could be a good idea).

Of course, any shell scripting language will be your friend.

(Remember that the values for the properties which define the hierarchy of the resources must be unique to the resource)

• HyperThreading is 2 threads per core.
• threads of core N = N and N+#cores

To be completed.

In this case, cores are the leaf resources (no thread resource property in OAR resources table), but we would like to enable users to possibly get the 2 hyper-threads associated to each core it in their job.

It's indeed easy to disable a 2nd thread for a core (see /sys/device/system/cpu), but once this 2nd thread is disabled, it's not so easy to find its number again if we did not keep the information somewhere. Indeed the “linux processor” does not show up in hwloc or /proc/cpuinfo anymore, and /sys does not seem very helpful either. One may argue that the numbering seems to be: threads of core N = N and N+#cores, but can we rely on that on any machine ?

This is an issue if we want to consider disabling the 2nd thread of only some cores of a host, not all. Otherwise indeed, disabling all second threads or re-enabling all of them is straight-forward using a for loop and looking at /sys/devices/system/cpu/possible.

We are discussing a way for handling that in OAR, by maybe changing the value (and type :( ) of the cpuset field in the resource table to for instance “0-8” instead of just 0 (in the case of the number of the 2nd thread of core 0 being 8), and adapting the job resource manager to handle that value accordingly to an HT job type.

Another option is to have the job resource manager look at a configuration file located on every nodes providing the information.

— Pierre Neyron 2014/02/21 12:47