Experimenting Green OAR upon Grid5000
From WikiOAR
For the experimentation and evaluation of the energy saving feature, I have created an image based on debian lenny distribution, kernel 2.6.26 and OARv2.4.0.
The name of the image is lenny2.6.26-OAR2.4.0-MPI-green-testing and it can be found in grenoble, lyon clusters for the moment.
Here is the procedure to follow for the deployment of a Green OAR cluster upon Grid5000:
Contents |
Connection to a grid5000 frontal server
I)you connect to grenoble frontal server by writing:
ssh USERNAME@digitalis.imag.fr
II)then you connect to grenoble (or lyon) frontal server
ssh grenoble.grid5000.fr
Allocation of nodes
III)then you will allocate some nodes for deployment in Interactive mode
oarsub -l /nodes=4,walltime=3 -p 'cluster="genepi"' -t deploy -I
on the -l you put the number of nodes and the duration of your job on the -p you put the name of the cluster (here you put either "griffon" or "grelon" for nancy OR "paradent" for rennes) on the -t you put deploy so that you can make a deployment and you finally got to add -I so that your job becomes Intereactive
with man oarsub you can see all options
IV) Once the allocation of nodes is granted, you can check out your job
oarstat |grep USERNAME
Environment Deployment
VII)then you can deploy the environment on the 4 nodes you have allocated with the following command:
~/bin/katapult --min-deployed-nodes 4 --max-deploy-runs 30 --deploy-env lenny2.6.26-OAR2.4.0-MPI-green-testing --file $OAR_NODEFILE --deploy-user ygeorgiou
OAR cluster Configuration
VIII) The configuration of Green OAR can be made automatically by executing the following script on the frontal:
sort -u $OAR_FILE_NODES | tail -3 | ssh -l g5k $(sort -u $OAR_NODEFILE | head -1) "xargs ~g5k/launch_OAR.sh"
when it finishes the first node will be the cluster server and the rest 3 nodes will be the computing nodes of your personal cluster...
IX)Then you will connect to the first node of the 4nodes that you deployed (which will be the OAR server):
ssh -l g5k $(sort -u $OAR_FILE_NODES | head -1)
the password is grid5000 and you will be as user g5k the user root has password grid5000 as well.
Green OAR Customization
For customization of OAR Green features you need to play with the following values on /etc/oar/oar.conf
########################################################################### # If you want to manage starting and stopping node feature. OAR gives you # # this API: # ############################################################################### # # When OAR scheduler wants some nodes to wake up then it launches this command # and puts on its STDIN the list of nodes to wake up (one hostname by line). # The scheduler looks at the cm_availability field in the resources table to know # if the node will be started for enough time. #SCHEDULER_NODE_MANAGER_WAKE_UP_CMD="/usr/lib/oar/oardodo/oardodo /usr/local/sbin/wake_up_nodes.sh" SCHEDULER_NODE_MANAGER_WAKE_UP_CMD="/usr/lib/oar/oardodo/oardodo /usr/local/sbin/wake_up_nodes_ipmi.sh" SCHEDULER_LATE_NODE_MANAGER_WAKE_UP_CMD="/usr/lib/oar/oardodo/oardodo /usr/local/sbin/wake_up_nodes_ipmi_ power_cycle.sh"
# When OAR considers that some nodes can be shut down, it launches this command # and puts the node list on its STDIN(one hostname by line). #SCHEDULER_NODE_MANAGER_SLEEP_CMD="/path/to/the/command args" SCHEDULER_NODE_MANAGER_SLEEP_CMD="/usr/bin/taktuk -t 30 -s -c '/usr/bin/ssh -p 6667' -f - broadcast exec [ 'oardodo /sbin/halt -p' ]" #SCHEDULER_NODE_MANAGER_SLEEP_CMD="/usr/local/oar/sentinelle.pl -f - -t 3 -p 'oardodo halt'"
# Parameters for the scheduler to decide when a node is idle. # Number of seconds since the last job was terminated on nodes SCHEDULER_NODE_MANAGER_IDLE_TIME="600"
# Parameters for the scheduler to decide if a node will have enough time to sleep. # Number of seconds before the next job SCHEDULER_NODE_MANAGER_SLEEP_TIME="600"
################################################################################
Green OAR Experimentation
Using the tools xionee we can replay collected workload traces and observe the funtion of Green OAR.
cd xionee #start the server responsible for the submission of jobs ./rctrl_drb_server.rb #submit a workload file (format CIRNE) on the OAR cluster (the trace file must be collected from a cluster with the same number of total processors with our cluster) ./ixnilatis.rb oar:genepi-1.grenoble.grid5000.fr:workloads/fs2-trace70-86.file:9000
After the end of the experiments we can export a trace file from the OAR database so that to observe how the workload trace file was really executed:
./oarDB-2-swf.rb -s localhost -d oar -u root
For the electrical consumption....
