This shows you the differences between two versions of the page.
— | wiki:old:gsoc_proposal_node_resource_isolation [2013/07/10 22:55] (current) – created - external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | The goal of this project is to develop upon OAR a mechanism for task confinement / isolation and task binding upon the resources of a node based upon **cgroups** (control groups) which is a (recently appeared) Linux kernel feature [1,2,3]. | ||
+ | |||
+ | ===== Background | ||
+ | |||
+ | The emergence of multicore architectures that introduced new levels of hierarchies inside the node arose the need for | ||
+ | methods of job confinement upon specific allocated resources in a single node. Indeed, multiple | ||
+ | users can find themselves working on the same computing node, which raised the need of task | ||
+ | placement upon specific cores in order to avoid collisions between different tasks. The problem | ||
+ | of collisions are that cores or socket resources easily can be oversubscribed, | ||
+ | performance, | ||
+ | upon cores and in general specific parts of the resources ( memory, disk I/O, bandwidth) became | ||
+ | an important challenge for the new Resource Managers. OS techniques like Cpusets, sched_affinity are | ||
+ | supported by the Resource and Job Management Systems to provide a fine management of the CPUs (cores) of the | ||
+ | nodes. | ||
+ | |||
+ | In latest version of Linux kernels it is possible to specifically tell to the scheduler which process | ||
+ | can run upon which core. This core affinity, can be influenced, in Linux, via 2 different ways: | ||
+ | |||
+ | * **sched_affinity** [4] which is a system call which takes a bitmask as parameter where each bit reflects one core and if a core is on with bit value 1 (or off with value 0) then the scheduler does (or not) migrate the process to that core | ||
+ | * **Cpusets** [5] which are processor sets defined by a hierarchical pseudo-filesystem upon which only processes bound to this set can be executed. | ||
+ | |||
+ | The mask or set is inherited by all child threads/ | ||
+ | The use of any of this techniques allows the RJMS to have full control of the processes of its jobs and permits the dedicated binding of jobs upon specific cores along with the efficient cleaning of remaining processes after the end of the job. | ||
+ | |||
+ | |||
+ | |||
+ | An extension of cpusets which has recently been added to the Linux kernel (after 2.6.32) are the **cgroups** [1,2,3]. Each control group is a set of tasks on a system that have been grouped together to better manage their interaction with system hardware. They provide a mechanism for aggregating/ | ||
+ | |||
+ | **OAR currently provides tasks confinement and binding based upon the cpusets core affinity mecanism.** | ||
+ | |||
+ | |||
+ | ===== Goals of this GSOC project | ||
+ | - Design and implement the basic framework for the support of cgroups upon OAR. This framework will allow the easy porting of any of the different cgroup subsystems depending the needs of each system. | ||
+ | - Provide the implementation of //cpuset//, //memory// and //devices// subsystems for task confinement/ | ||
+ | - Develop mechanisms for tasks binding upon cores using either //cpuset// or //hwloc// [6] to allow applications to change on their own the proposed task binding internally . | ||
+ | - Experimentation with benchmark or real-life applications. | ||
+ | - Eventually provide the implementation of //freezer// and //blkio// subsystems as well. | ||
+ | |||
+ | ===== References | ||
+ | - http:// | ||
+ | - http:// | ||
+ | - http:// | ||
+ | - http:// | ||
+ | - http:// | ||
+ | - http:// | ||