High-Performance Storage (HPS)
An HPS device is a persistent, NFS-like filesystem you provision, size, and attach to jobs and workstations. It provides a central shared file store with SSH and SFTP access, reachable from any cluster for compute, post-processing, or visualization.
HPS or CFS?
Both mount a shared filesystem into your jobs. They suit different situations, and neither is a replacement for the other.
Choose HPS when you need:
- SSH or SFTP access to the storage itself — CFS has neither.
- Several independent devices, sized and shared separately. CFS allows one per workspace.
- Windows workstations.
- No region lock. A CFS constrains every job in the workspace to its region; an HPS doesn't.
- A device you control — you set the size, walltime, and alerts yourself.
Choose CFS when you need:
- High parallel throughput. Lustre substantially outperforms NFS for I/O-heavy simulation.
- No walltime and no compute charge.
- Many jobs attached at once — HPS has NFS node limits.
- Workspace-wide sharing through Rescale Projects, with no per-device setup.
HPS versus CFS versus Rescale Files
The lifecycle to plan around
HPS devices are billed hourly and terminate on walltime, which makes them unlike the other two storage options.
Walltime and an exhausted project budget both kill an HPS device. Walltime is capped at 365 days and counts from creation, and once started, it can't be extended beyond 365 days from the cluster's start time.
If you're relying on an HPS, watch its walltime.
The 365-day cap is deliberate. HPS exists so jobs don't have to re-copy data, not to hold data indefinitely. For long-term retention use Rescale Files.
What the shutdown sync does and doesn't preserve
Only files under the cluster's work/ directory are synced to Rescale Files at shutdown. And the sync is lossy in a way that surprises people:
Files synced at shutdown land in Rescale Files as input files with only their name and size preserved. Relative paths, timestamps, and POSIX permissions are not kept.
A carefully organized 50,000-file tree comes back as a flat list. If structure matters, archive it yourself, tar it up before the device terminates.
Job output is unaffected: files in a job's work/ upload on job completion exactly as they would without an HPS.
If a device terminates before you're ready
Data isn't necessarily gone. HPS disks are retained for a short grace period (about 3 days on most platforms) and can be recovered onto a new device.
Recovery is a manual engineering process, so contact Rescale Support immediately with the device ID. Waiting past the grace period makes it impossible.
What an HPS actually costs
Unlike the other storage options, an HPS runs a compute instance. You're billed for three or four things at once:
A storage device is billed like a running job, not like object storage. An HPS you've forgotten about accrues compute charges around the clock.
Extending capacity adds to the storage charge for the remaining lifetime. If a device is only occasionally needed, terminating it (letting the sync move data to Rescale Files) is usually cheaper than leaving it up.
Pages in this section
- Create a device — sizing, walltime, alerts, and auto-extend
- Attach to jobs and workstations — mounting, and the node limits that queue jobs
- Manage a device — extending, sharing, and moving data off
Related
- Cloud Filesystem (CFS) — the other mounted-filesystem option
- Rescale Files — where files land at walltime
- Workstation files — the termination dependency
- Budgets and deposits — the other thing that terminates devices