Trace:
[[wiki:using_hyperthreading_on_nodes]]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
wiki:using_hyperthreading_on_nodes [2014/05/15 00:07] – [Adapting an existing resource setup] neyronwiki:using_hyperthreading_on_nodes [2015/07/05 10:19] – [On offlining/onlining hyper-threads with OAR] neyron
Line 1: Line 1:
-====== On using hyper-threads as the leaf level of resources ======  +====== On using (hyper)threads as the leafs of the resources tree======  
-Once HyperThreading is enabled at BIOS level, using threads as the leaf level of OAR resources is actually quite strait forward.+If HyperThreading is enabled in BIOS, using threads as the leafs of OAR (implicit) resources tree is quite simple.
  
 ===== Using the oar_resource_init script ===== ===== Using the oar_resource_init script =====
-The last version of the ''oar_resource_init'' script comes with the ''-H'' and ''-T'' options. This allows for generating the ''oarnodesetting'' commands with the ''thread'' properties.+The last version of the ''oar_resource_init'' script comes with the ''-H'' and ''-T'' options. It generates the ''oarnodesetting'' commands with the ''thread'' properties.
  
 ===== Adapting an existing resource setup ===== ===== Adapting an existing resource setup =====
-Here we consider an existing resource setup to which one would like to had the thread leaf resources.+Here we consider an existing resource setup to which one would like to had the thread as leaf resources.
  
 First you need to define the thread property: First you need to define the thread property:
Line 14: Line 14:
 </code> </code>
  
-Assuming your cluster is homogeneous, and that resources were consistently created (i.e. the ids of the properties of the resources are well set), you can just add to every resource lines the property: ''thread = (2 * core - 1)'' (''oarnodesetting -s''), and then for each of them create a new resources (''oarnodesetting -a'') with exactly the same values for all properties, except ''thread = (2 * core)'' and ''cpuset = (original cpuset value + # cores per cpu)'' (using ''hwloc'' in a job afterward to verify the correct cpuset settings could be a good idea).+Assuming your cluster is homogeneous, and that resources were consistently created (i.e. the ids of the properties of the resources are well set), you can add to every resource lines the property: ''thread = (2 * core - 1)'' (''oarnodesetting -p ... ''), and then for each of them create a new resources (''oarnodesetting -a ...'') with exactly the same values for all properties, except ''thread = (2 * core)'' and ''cpuset = (original cpuset value + # cores per cpu)'' (using ''hwloc'' in a job afterward to verify the correct cpuset settings could be a good idea).
  
 Of course, any shell scripting language will be your friend. Of course, any shell scripting language will be your friend.
Line 29: Line 29:
  
 ====== On offlining/onlining hyper-threads with OAR ====== ====== On offlining/onlining hyper-threads with OAR ======
-In this case, cores are the leaf resources (no hyper-thread resource property in OAR resources table), but we would like to enable users to possibly get the 2 hyper-threads associated to each core it in their job. +In this case, cores are the leaf resources (no thread resource property in OAR resources table), but we would like to enable users to possibly get the 2 hyper-threads associated to each core it in their job. 
  
 It's indeed easy to disable a 2nd thread for a core (see /sys/device/system/cpu), but once this 2nd thread is disabled, it's not so easy to find its number again if we did not keep the information somewhere. It's indeed easy to disable a 2nd thread for a core (see /sys/device/system/cpu), but once this 2nd thread is disabled, it's not so easy to find its number again if we did not keep the information somewhere.
Line 37: Line 37:
 This is an issue if we want to consider disabling the 2nd thread of only some cores of a host, not all. Otherwise indeed, disabling all second threads or re-enabling all of them is straight-forward using a for loop and looking at /sys/devices/system/cpu/possible. This is an issue if we want to consider disabling the 2nd thread of only some cores of a host, not all. Otherwise indeed, disabling all second threads or re-enabling all of them is straight-forward using a for loop and looking at /sys/devices/system/cpu/possible.
  
-We are discussing a way for handling that in OAR, by maybe changing the value (and type :( ) of the cpuset field in the resource table to for instance "0-8" instead of just 0 (in the case of the number of the 2nd thread of core 0 being 8), and adapting the job resource manager to handle that value accordingly to an HT job type.+For that purpose, the cpuset field of OAR was changed to string with OAR 2.5.4+. This allows setting the value of the cpuset field in the resource table to,for instance, "0,8" instead of just 0 (core 0 has threads with logical cpu id 0 and 8). Then the job resource manager can handle that value very easily whenever an HT job type is in use by a job.
  
-Another option is to have the job resource manager look at a configuration file located on every nodes providing the information. + --- //[[pierre.neyron@imag.fr|Pierre Neyron]] 2015/07/05 10:14//
- +
- --- //[[pierre.neyron@imag.fr|Pierre Neyron]] 2014/02/21 12:47//+
wiki/using_hyperthreading_on_nodes.txt · Last modified: 2015/09/04 17:24 by neyron
Recent changes RSS feed GNU Free Documentation License 1.3 Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki