Stop a job
Stop shuts a job down cleanly and saves your files. Force Stop ends it faster, at the cost of anything not yet written. Force Stop appears only after you click Stop.
Which to use
Default to Stop. Reach for Force Stop when you don't need restart files, or when the job was misconfigured and none of its output matters.
The computer analogy from the original docs is apt: Stop is a normal shutdown. Force Stop with saving is "save my files, then pull the plug." Force Stop without saving is pulling the plug.
Stop a job
- Click Stop on the running job.
- Confirm in the dialog.
Files are then saved to Rescale Cloud Storage and appear on the Results tab.
Why a graceful stop takes a while
Four things happen in sequence, which is why the total isn't instant:
- The application prepares to shut down. It may write restart files, which can be large, slow to write, and slow to upload. Your license stays checked out for this entire step.
- Remaining processes are terminated.
- Files are uploaded to Rescale. Files start appearing on Results as this progresses.
- The cluster is terminated.
Step 1 is usually what makes a stop feel slow. A solver writing a large restart file is doing exactly what it should.
Force stop a job
Force Stop is hidden until you click Stop. You can't reach it directly.
- Click Stop on the Status tab.
- In the pop-up, confirm with Stop. (Stopping here gives you a normal graceful shutdown.)
- Click Force Stop, next to Stop.
- Choose either:
- Save all files and upload them to Results, or
- Terminate the cluster without saving.
Job Logs then shows Stopping job (User terminated)…. It takes a few minutes.
No output files are uploaded. The Results tab stays empty, permanently. There's no recovery. The cluster and its disk are gone.
Use it only when you're certain nothing on that cluster matters.
Other things that stop a job
You aren't the only cause:
Interruptible capacity being reclaimed doesn't stop your job. It triggers a restart or requeue. See why a job is queued.
"A run failed" versus "Error executing job"
Worth distinguishing before you contact Support:
A run failed— the platform worked; your application returned a non-zero exit code. Start with reading logs and exit codes.Error executing job— the platform hit an error it couldn't handle. This one is Rescale's to investigate. Share the job with Support and include the job ID.
Walltime
Walltime is the maximum hours a job may run. Set it on the Hardware Settings panel when creating a job, and edit it on a running job from the Run Time column of the jobs list.
Extending walltime on a job that's about to hit its limit is much cheaper than letting it stop and rerunning.
Stopping a workstation
Workstations stop differently, and can be suspended instead, which preserves the disk. See suspend and resume.
Related
- Monitoring & status — watching before you stop
- Checkpoint long simulations — making a stop resumable
- Why a job is queued — restarts versus stops
- Download your results — retrieving what was saved
- Troubleshooting — diagnosing a failure