Difference between revisions of "P3s"

From DUNE
Jump to navigation Jump to search
Line 18: Line 18:
 
=DAG=
 
=DAG=
 
==DAG Model in p3s==
 
==DAG Model in p3s==
* Vertex and Edge tables: vertices are jobs and the edges are data
+
* Vertex and Edge tables: vertices are jobs and the edges are data. The edge class has the following attributes at a minimum
 +
** ID
 +
** Source node
 +
** Target node
 +
 
 
* Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.
 
* Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.
  

Revision as of 22:50, 4 December 2016

!!!UNDER CONSTRUCTION!!!

States of Pilots and Jobs

Pilots

States:

  • active
  • dispatched
  • running
  • finished (completion of a job)
  • stopped (timeout w/o getting a job)

Jobs

States:

  • defined
  • dispatched (sent to a pilot)
  • running
  • finished

DAG

DAG Model in p3s

  • Vertex and Edge tables: vertices are jobs and the edges are data. The edge class has the following attributes at a minimum
    • ID
    • Source node
    • Target node
  • Leaves of a DAG: can only be a job, not data (since all data are edges and not vertices). This also has the benefit of not having final data unaccounted for - it must me either flushed or moved to permanent storage in most cases. The job/task responsible for either of these operations forms the leaf.

DAG as a state machine in p3s

A design idea:

  • have an attribute in the Job class which specifies whether children of that particular node have been created (can be one child, of course)
  • this assumes that the data which is input to each child have been created and is already available
  • special (but trivial) case is the leaf of a DAG - no further jobs need to be generated. Finding whether a node is a leaf can be done by looking for edges with "source" attribute corresponding to this node - the leaf condition is ascertained when there are no such edges.