Writing an automation
An automation is a Python or Bash script you attach to a job, which Rescale runs at a defined point in the job's lifecycle. Automations covers what they are for and how to create and attach one. This page is the part that matters while you are writing the script: what it can assume about where it runs, how it gets its configuration, and what it can call back into.
Pick the timing first
It changes what the script can rely on existing.
A during-execution automation is a side-car, not a step in the run, so it should tolerate files being partially written.
Where the script runs
The script runs on the job's own cluster with access to the job's working directory. For batch
jobs that directory is ~/work/shared, and process output for the run lands in
~/work/process_output.log. Both are reachable over SSH while the cluster is up, which is the
practical way to work out paths for a script you are still developing. See
SSH access.
Take configuration as variables
An automation declares variables, and whoever attaches it supplies the values for that particular job. Write the script to read its inputs from those variables rather than hardcoding paths, filenames, thresholds or webhook URLs. One parameterised automation attaches to many jobs; a hardcoded one gets copied and the copies drift.
Variables are also the right place for any credential the script needs for a third-party system, such as a Slack or Teams webhook.
Calling the Rescale API from inside a job
Every job receives a short-lived API key in the RESCALE_API_KEY environment variable, so
a script does not need a token shipped inside it. The key expires when the job completes. See
embedded per-job keys.
import os
import requests
api_key = os.environ["RESCALE_API_KEY"]
job_id = "..." # the job this automation is attached to
requests.post(
f"https://platform.rescale.com/api/v2/jobs/{job_id}/tags/",
headers={"Authorization": f"Token {api_key}"},
json={"name": "converged"},
)
RESCALE_API_KEY is the only environment variable Rescale sets for you. Anything else the
script needs, including the job id, is yours to supply through the automation's variables.
If the work has to outlive the job, use a key you created yourself and pass it in as an automation variable instead of relying on the embedded one.
Swap the host if your account is on another regional platform. See platform regions.
The operations automations mostly use
Writing tags and custom fields is what makes metadata capture reliable, because a value the run wrote is a value nobody had to remember to enter. See tags and custom fields for what you are populating.
The full operation reference lists all 111 operations.
Automation, submit script, or workflow
Three different places to put code around a job, and they are not interchangeable.
The signal that you want an automation rather than submit-script logic is reuse: if the same post-processing has been pasted into three submit scripts, it is an automation.
Related
- Automations — what they are for, and creating and attaching one in the UI
- REST API — conventions, authentication, and the operation reference
- CLI —
#RESCALE_*directives and the submit-script shape - Tags and custom fields — what a metadata automation writes into
- Job templates — saving a configuration, including its attached automations
- AI Physics — the surrogate-model training that AI extraction automations feed