dugoh / advancedscheduler Goto Github PK

Need an autorep script that will:

* Default behavior is to report current status for a list of jobs matching
a pattern. Patterns will use SQL-style '%' for wildcards (E.g. %job would
match anything ending in 'job'. My%job% would match My_New_job_1, My_job,
Myjob, etc. )
* Output JIL for a list of jobs matching the same-style pattern. This is
selected via the '-q' switch. 
* Job names are specified with the -J parameter.

Sample command lines:

# report the status for all jobs matching the pattern.
autorep -J My%Job   

# output JIL for all jobs matching a pattern
autorep -qJ MyJob%xyz

More advanced reporting functionality will come along later. I think this
covers the general use case. 

SQL Makefiles are currently organized by object type. Tables are built
first, then views, then functions... The problem is that some functions are
based on views, and some views are based on functions. In order to properly
fix this, makefiles need to be reorganized so that dependencies can be set
up between different types of objects. 

This is currently causing a problem with the PendingJobs view and the
StartConditionsMet function. These objects must currently be deployed by
hand due to this deficiency. 

Need an agent daemon to actually run the jobs on the specified machine.
Daemon should be written in Python, the Job class in Database.py to
maintain the job states.

This daemon should query a configurable database for a list of pending
jobs. The logic to determine the list should reside in stored procedures.
The daemon should be as small as possible, while achieving the following goals:

   * Set appropriate job status: 
      * 'ST' (starting) -- Job session setup, before job is actually started
      * 'RU' (running) -- Job has actually started running
      * 'TE' (terminated) -- Job was killed via ADS control command
      * 'FA' (failed) -- Job exited with non-zero status
      * 'SU' (success) -- Job exited with zero status

   * Run each pending job in a multi-process or multi-threaded manner
   * Listen on a control connection (dbus?) for control commands
      * Agent shutdown
      * Terminate job (SIGTERM)
      * Kill job (SIGKILL)