services:cluster:usetmpforio
Use /tmp for I/O intensive single node jobs
Jobs that do a lot of I/O operations on a shared cluster filesystem like /scratch
can severely slow down the whole system. If your job does not use multiple nodes and is not reading and writing very large files, it might be a good idea to move input and output files to the /tmp
folder on the compute node itself.
/tmp
is a RAM based filesystem, meaning that anything you store there is actually stored in memory. So space is quite limited. Currently all jobs on a node can use at most 20% of the total system memory for space in /tmp
. If you need more space, you should consider using /dev/shm
, where you can use up to 50% of the total system memory per job.
Usable space below /tmp and /dev/shm counts towards your job's memory usage and thus is limited by the
–mem
option
services/cluster/usetmpforio.txt · Last modified: 2024/04/26 14:19 by hoffmac00