Difference between revisions of "P3s"

From DUNE
Jump to navigation Jump to search
Line 17: Line 17:
  
 
=DAG=
 
=DAG=
 +
==DAG Model in p3s==
 
* Vertex and Edge tables: vertices are jobs and the edges are data
 
* Vertex and Edge tables: vertices are jobs and the edges are data
 
* Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.
 
* Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.
 +
 +
==DAG as a state machine in p3s==
 +
A design idea:
 +
* have an attribute in the Job class which specifies whether children of that particular node have been created (can be one child, of course)
 +
* this assumes that the data which is input to each child have been created and is already available
 +
* special (but trivial) case is the leaf of a DAG - no further jobs need to be generated. Finding whether a node is a leaf can be done by looking for edges with "source" attribute corresponding to this node - the leaf condition is ascertained when there are no such edges.

Revision as of 22:49, 4 December 2016

!!!UNDER CONSTRUCTION!!!

States of Pilots and Jobs

Pilots

States:

  • active
  • dispatched
  • running
  • finished (completion of a job)
  • stopped (timeout w/o getting a job)

Jobs

States:

  • defined
  • dispatched (sent to a pilot)
  • running
  • finished

DAG

DAG Model in p3s

  • Vertex and Edge tables: vertices are jobs and the edges are data
  • Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.

DAG as a state machine in p3s

A design idea:

  • have an attribute in the Job class which specifies whether children of that particular node have been created (can be one child, of course)
  • this assumes that the data which is input to each child have been created and is already available
  • special (but trivial) case is the leaf of a DAG - no further jobs need to be generated. Finding whether a node is a leaf can be done by looking for edges with "source" attribute corresponding to this node - the leaf condition is ascertained when there are no such edges.