**Student and mentor ;), please read carefully this page...** Student: **Jean-Pierre POUTCHEU** Mentor: Bruno Bzeznik Co-Mentor: Olivier Richard ---- ===== Student: Things to do before starting ===== * Subscribe to oar commits mailing list * Play with [[Kameleon]] (generate a OAR/Debian appliance) * Practice [[RESTfullAPI | OAR RESTfull API]] (using the OAR appliance) ===== Project Specification ===== * First, implement a pagination feature as discussed here: https://gforge.inria.fr/tracker/index.php?func=detail&aid=8982&group_id=125&atid=586 (as a bootstrap coding into the API) as a requirement for the Web portal GSOC * Then, integrate oaradmin as discussed (resources creation, admin rules edition, config file edition, prologue/epilogue) (add details here) * Take a look at Mason+mod_perl to optimize the API ==== Pagination specifications ==== * List of current jobs /jobs.(yaml|json|html) /jobs/details.(yaml|json|html) /jobs/table.(yaml|json|html) * with pagination options /jobs.(yaml|json|html)?start_id=X /jobs/details.(yaml|json|html)?start_id=X /jobs/table.(yaml|json|html)?start_id=X * hasmap data structure items ==> jobs results (hashmap if structure = oar, table if structure = simple) page ==> current page number in the results list, used in the next and previous links generation limit ==> limit value (this value overwrite the value defined in the OAR file config) [[next_url_link|==> the next url link]] e.g. "/jobs.(yaml|json|html)?start_id=X&page=3" [[previous_url_link|==> the previous url link]] e.g. "/jobs.(yaml|json|html)?start_id=X&page=1" 'next_url_link' and 'previous_url_link' may be absent * struct methods : struct_job_list(jobs, structure, previous_url_link, next_url_link) struct_job_list_details(jobs, structure, previous_url_link, next_url_link) 'previous_url_link' and 'next_url_link' will be added as parameters ==== Pagination specifications (Bruno version) ==== === Every data structure returned by the API should be wrapped into this hash table: === * In the case a collection is returned (array or hash): items ==> query results (hashmap if structure = oar, table if structure = simple) limit ==> limit value (this value overwrite the value defined in the OAR file config) offset ==> id at which the listing started total ==> total number of items links ==> array of hash describing useful links: href => , rel => We know that we have a next page if : (# of items) + offset < total or if there's a link->rel=="next" * In the case a simple element is returned: key1=>value1 key2=>value2 ... links ==> array of hash describing useful links: href => , rel => === Specific case of the jobs uri (as an example of an uri that needs pagination): === * Uri GET /jobs[[/details|/table]].(yaml|json|html)?[[from=]]&[to=]&[[state=]]&[offset=]&[[limit=]] * Input parameters: FROM: From unix timestamp TO: To unix timestamp STATE: List of possible states (ex: Running,Waiting,Finishing,Launching) ID: the first id to be listed. Jobs are listed from this id until LIMIT is reached LIMIT: if more than LIMIT jobs satisfy the query, stop when this LIMIT is reach === input/output Examples === * GET /jobs.yaml?from=1274096990&to=1276688990&limit=2 (or GET /jobs.yaml?from=1274096990&to=1276688990 && $API_NUMBER_ITEMS_LIMIT=2 into oar.conf) items: - api_timestamp: 1276689144 id: 11 name: ~ owner: kameleon queue: default state: Running submission: 1276689106 links: - href: /jobs/11 rel: self - href: /jobs/11/resources rel: resources - api_timestamp: 1276689144 id: 12 name: ~ owner: kameleon queue: default state: Running submission: 1276689109 links: - href: /jobs/12 rel: self - href: /jobs/12/resources rel: resources total: 5 offset: 0 links: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2 rel: self # Optional: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=2 rel: next * GET /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=2 items: - api_timestamp: 1276689144 id: 13 name: ~ owner: kameleon queue: default state: Running submission: 1276689106 - api_timestamp: 1276689144 links: - href: /jobs/13 rel: self - href: /jobs/13/resources rel: resources id: 14 name: ~ owner: kameleon queue: default state: Running submission: 1276689109 links: - href: /jobs/14 rel: self - href: /jobs/14/resources rel: resources total: 5 offset: 2 links: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=2 rel: self # Optional: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=4 rel: next - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=0 rel: prev * GET /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=4 items: - api_timestamp: 1276689144 id: 15 name: ~ owner: kameleon queue: default state: Running submission: 1276689106 links: - href: /jobs/15 rel: self - href: /jobs/15/resources rel: resources total: 5 offset: 4 links: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=4 rel: self # Optional: - href: /jobs.yaml?from=1274096990&to=1276688990&limit=2&offset=2 rel: prev === Default uri parameters === * In oar.conf: # Maximum default number of items API_NUMBER_ITEMS_LIMIT # Default parameters for the /jobs uri # if a "&limit=" is given, the $API_NUMBER_ITEMS_LIMIT is ignored for this uri API_JOBS_URI_DEFAULT_PARAMS="Finishing,Running,Resuming,Suspended,Launching,toLaunch,Waiting,toAckReservation,Hold&limit=200 ==== Oaradmin api specifications ==== === Admission rules === ;GET /admission_rules >//list admissions rules// ;GET /admission_rules/ >//list the admission rule #id//: id: 1 rule: | code line 1 code line 2 .... links: - href ====== rel=current ;PUT /admission_rules >//create a new admission rule// #PUTDATA: rule: | code line 1 code line 2 .... ;POST /admission_rules >//create a new admission rule (for browsers compatibility)// #POSTDATA: method: put rule: | code line 1 code line 2 .... ;DELETE /admission_rules/ >//delete an admission rule// ;POST /admission_rules/ >//delete an admission rule (for browsers compatibility)// #POSTDATA: method: delete >//erase an admission rule// #POSTDATA: rule: | code line 1 code line 2 .... === Resources creation === * POST /resources/generate.(yaml|json|html) #POSTDATA: param = '-a /node=node1 -p memnode=1024 -p cpufreq=3.2 -p cputype=xeon' resources: '/node=node{4}/cpu={2}/core={2}' properties: memnode: 1024 cpufreq: '3.2' cputype: 'xeon' * RESULT ressources: - hostname : node1 properties : - memnode : 1024 cpufreq : 3.2 cputype : xeon .... .... items: - hostname : node1 properties : cpu: 1 core: 1 cpuset: 0 memnode : 1024 cpufreq : 3.2 cputype : xeon - hostname : node1 properties : cpu: 1 core: 2 cpuset: 1 memnode : 1024 cpufreq : 3.2 cputype : xeon ... links : - rel : create method : post url : /resources - rel : self method : post url : /resources/generate === Configuration variables (oar.conf) === ; GET /config.(yaml|json|html) > Get all the configuration variables #Example output: DB_TYPE: 'mysql' DB_HOSTNAME: 'localhost' DB_PORT: '3306' ... ; GET /config/.(yaml|json|html) > Get a variable # Example query: GET /config/DB_TYPE.yaml # output: DB_TYPE: 'mysql' api_timestamp: 1278574909 links: - rel: set method: post url: /config/DB_TYPE - rel: self method: get url: /config/DB_TYPE ; POST /config/.(yaml|json|html) > Set a variable # example QUERY POST /config/DB_TYPE.yaml # example POSTDATA value: "mysql" ; POST /config.(yaml|json|html) > Set all the configuration variables # example QUERY POST /config.yaml # example POSTDATA DB_TYPE: 'mysql' DB_HOSTNAME: 'localhost' DB_PORT: '3306' ... Do we make checks??? Mandatory variables; types; syntax;? ===== Main Milestones ===== ===== Tips ===== ===== TODO list ===== ==== Mentor ==== ==== Student ==== * OAR Admin specification (URLs, input/output,...) ===== Links to look at ===== * http://www.masonhq.com/ * https://2009.jres.org/planning_files/summary/html/85.htm