Skip to main content

Workflow concepts

Three objects describe workflows on Rescale: a workflow is the template, a workflow run is one execution of it, and a node is one unit of work inside it.

Workflow​

A workflow is a directed acyclic graph (DAG) that describes work to run on Rescale. It is a template. Running it produces a workflow run.

A workflow has two things: a list of nodes and a list of edges. Nodes hold the work (a job, a workstation, a storage device, a decision). Edges are dependencies. A node runs after every upstream node it depends on has completed.

Workflows are persistent. You create one, then submit it as many times as you need. Each submission creates a new run with its own status, logs, and outputs.

For the workflow object's exact fields and types, see the Workflows section of the API reference. That is generated from the OpenAPI spec, so it cannot drift from the API the way a hand-written table can.

Workflow run​

A workflow run is one execution of a workflow. It tracks the status of every node as the DAG progresses.

Submitting a workflow creates a new run. The run starts in queued, moves to running when the first node starts, and ends in one of completed, failed, or stopped. Node-level status lives inside the run object and updates as the DAG advances.

A run references the workflow by id but does not share mutable state with it. Editing the workflow after submission does not change any run that was already created.

StatusMeaning
queuedAccepted, waiting for a scheduler slot
runningAt least one node is executing
completedEvery node finished with status completed
failedAt least one node finished with status failed and the failure stopped the run
stoppedA user or system stopped the run before it finished

Retried runs​

A retried run continues a previous failed run rather than starting over. It references the run it came from and reuses the outputs of nodes that succeeded and have not changed since. Only failed nodes, nodes whose configuration changed, and the nodes downstream of those execute again.

Node​

A node is one unit of work inside a workflow. Every node has a type and a data payload specific to that type.

Nodes are the vertices of the DAG. Edges connect them. A node runs after every upstream dependency has completed. The payload shape depends on the node type: a job node describes hardware, software, and input files; a workstation node describes an interactive session; a decision node routes the graph based on an expression.

Each run carries per-node status and logs. Inspect a node by reading its entry in the run object.

NodePurpose
JobBatch HPC execution
WorkstationInteractive session
Storage deviceCloud Filesystem mount
DecisionConditional branching
AgentAn automated step, such as an input check or failure analysis
EdisonAdds context to a pre-built agent, or runs a prompt you write yourself

The exact type strings the API accepts are in the API reference.