Solver run commands
The command line each solver expects on Rescale, plus the license flags and per-solver details that aren't obvious from the software catalogue. Look up your solver; the entries are independent.
The command goes in the Command field on Software Settings. See Create and submit a job for where that sits in job setup, and Command Templates for the suggested forms Rescale offers.
Host and machine files
MPI solvers need the list of nodes in the cluster. Rescale writes it into the $HOME
folder of your workspace in the common formats, so reference it by full path:
--hostfile $HOME/machinefile.openmpi
List $HOME to see what's available for your run. The usual set is hosts,
mpd.hosts, machinefile, machinefile.gpu, machinefile.openmpi and PCF.xml.
Some solvers want the host list somewhere specific rather than as an argument:
Abaqus
abaqus job=<job> cpus=$RESCALE_CORES_PER_SLOT mp_mode=mpi interactive
<job> is your .inp file without the extension. $RESCALE_CORES_PER_SLOT picks up
every core in the hardware you selected, so the same command works on any coretype.
Common optional arguments:
For the complete list, SSH into the cluster and run abaqus -help.
Global and local models
A two-stage global/local analysis is two commands in one job, not two jobs. Upload both
.inp files, then run the global model and pass its name to the local model with
globalmodel=. Rescale runs the commands in order:
abaqus job=input_global cpus=$RESCALE_CORES_PER_SLOT mp_mode=mpi \
output_precision=full double=both parallel=domain interactive
abaqus job=input_local cpus=$RESCALE_CORES_PER_SLOT mp_mode=mpi \
output_precision=full double=both parallel=domain globalmodel=input_global interactive
Fortran user subroutines
A user subroutine extends Abaqus where the normal input syntax cannot express what you
need, most often a custom material model. Pass the source or object file with user=:
abaqus job=<job> user=<user-subroutine> cpus=$RESCALE_CORES_PER_SLOT mp_mode=mpi interactive
Abaqus on Rescale is linked against gfortran. A subroutine that depends on Intel Fortran extensions will not compile.
Using a Dassault Systèmes License Server
To check out from a DSLS server rather than FlexLM, prepend these two lines to your command on the Software step:
export ABQLMIMPL=DSLS
export DSLS_CONFIG=$(IFS=:; DSLS=$HOME/DSLS ; if [ ! -f $DSLS ] ; then for i in $LM_LICENSE_FILE ; do echo ${i#*@}:${i%@*} >> $DSLS;done;fi;echo $DSLS)
Simcenter STAR-CCM+
A typical batch invocation:
starccm+ -power -rsh ssh -np $RESCALE_CORES_PER_SLOT -machinefile $HOME/machinefile -batch macro.java -load input.sim
The -power flag is required when you use the Rescale On-Demand License or your own
Power Session license.
- To mesh only, use
-batch mesh. - To mesh and then run, use
-batch mesh,run.
For a Power-on-Demand (POD) license, select Provide Existing License on the
Software Setup page, enter 1999@flex.cd-adapco.com as the license, and put your
product code in the Project Code field.
LS-DYNA
LS-DYNA ships with a Rescale wrapper. The core arguments are -i for the input,
-p for precision, -n for DMP ranks, -s for SMP threads, and -a for
additional options.
# SMP
ls-dyna -s $RESCALE_CORES_PER_SLOT -i <input-file> -p <precision>
# double precision
ls-dyna -i <input-file> -p double
# hybrid: DMP ranks with SMP threads per rank
ls-dyna -n <dmp-ranks> -s <smp-threads-per-rank> -i <input-file> -p <precision> -a "memory=600m memory2=60m"
Sending a sense switch to a running job
To write a restart or check progress mid-run, SSH into the cluster, change to
work/shared, and write the switch to d3kil:
echo sw2 > d3kil
The file is consumed and removed after execution.
CONVERGE
Run converge. If you need super licenses, add the flag for your major version:
converge -a super # CONVERGE 2.x
converge -S # CONVERGE 3.x
Rescale recommends running CONVERGE on a multiple of the cores-per-node for the coretype you chose. Check the cores per node for your coretype in Coretype Explorer rather than assuming a number.
COMSOL
To reach a running COMSOL server from your own machine:
-
Start a COMSOL Multiphysics Server or COMSOL Server job on Rescale.
-
Note the username and password printed in
process_output.log. -
Open an SSH tunnel to the cluster, with source and destination port
2036:ssh -i </path/to/key> <user>@<ip> -L 2036:localhost:2036 -N -v -
Connect your local COMSOL client to
localhost:2036.
From a Rescale Desktop, skip the tunnel and use the hostname printed in the log.
When you're finished, stop the server from the Rescale job page so the cluster shuts down.
LAMMPS
The installed package list depends on the version you select. Recent versions ship with the standard package set plus most USER packages: USER-ATC, USER-CGDNA, USER-DIFFRACTION, USER-INTEL, USER-OMP, USER-PHONON, USER-REAXC and others. Older versions differ. Check the specific version you plan to use before relying on a particular package.
Related
- Browsing the Catalog for finding a package and picking a version
- Software for how software works on Rescale: licensing, versions, publishing your own
- Create and submit a job for where the command goes
- Hardware & Coretypes for matching a coretype to a solver