services:cluster:gpunodes
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
services:cluster:gpunodes [2015/06/18 12:28] – [GROMACS example using GPU acceleration] dreger | services:cluster:gpunodes [2024/04/26 14:37] (current) – hoffmac00 | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Introduction to GPU accelerated jobs ====== | ====== Introduction to GPU accelerated jobs ====== | ||
+ | |||
+ | <note info> | ||
Currently we have 31 nodes in the yoshi cluster (ygpu01-ygpu31) equipped with GPU boards. The exact hardware config is: | Currently we have 31 nodes in the yoshi cluster (ygpu01-ygpu31) equipped with GPU boards. The exact hardware config is: | ||
Line 72: | Line 74: | ||
</ | </ | ||
+ | While your jobs run you can log in to the node and call '' | ||
+ | < | ||
+ | dreger@ygpu01: | ||
+ | Thu Jun 18 14:25:21 2015 | ||
+ | +------------------------------------------------------+ | ||
+ | | NVIDIA-SMI 340.65 | ||
+ | |-------------------------------+----------------------+----------------------+ | ||
+ | | GPU Name Persistence-M| Bus-Id | ||
+ | | Fan Temp Perf Pwr: | ||
+ | |===============================+======================+======================| | ||
+ | | | ||
+ | | N/A | ||
+ | +-------------------------------+----------------------+----------------------+ | ||
+ | | | ||
+ | | N/A | ||
+ | +-------------------------------+----------------------+----------------------+ | ||
+ | |||
+ | +-----------------------------------------------------------------------------+ | ||
+ | | Compute processes: | ||
+ | | GPU | ||
+ | |=============================================================================| | ||
+ | | 0 | ||
+ | | 1 | ||
+ | +-----------------------------------------------------------------------------+ | ||
+ | </ | ||
+ | Please check your job logfiles to see if your program has some problems using the GPUs. In case of GROMACS this might look like: | ||
+ | < | ||
+ | **NOTE: GPU(s) found, but the current simulation can not use GPUs | ||
+ | To use a GPU, set the mdp option: cutoff-scheme = Verlet | ||
+ | (for quick performance testing you can use the -testverlet option)** | ||
+ | |||
+ | Using 8 MPI threads | ||
+ | |||
+ | 2 GPUs detected: | ||
+ | #0: NVIDIA Tesla M2070, compute cap.: 2.0, ECC: yes, stat: compatible | ||
+ | #1: NVIDIA Tesla M2070, compute cap.: 2.0, ECC: yes, stat: compatible | ||
+ | |||
+ | **2 compatible GPUs detected in the system, but none will be used. | ||
+ | Consider trying GPU acceleration with the Verlet scheme!** | ||
</ | </ | ||
+ | |||
+ | In this case a cutoff-scheme was specified that can not be used with GPU acceleration. | ||
+ | |||
+ | Compare the timings with a test run on the same node, that does not use the GPUs. In some cases the GPUs will not help at all, even though '' | ||
+ | |||
+ | < | ||
+ | dreger@ygpu01: | ||
+ | dreger@ygpu01: | ||
+ | dreger@ygpu01: | ||
+ | |||
+ | Core t (s) Wall t (s) (%) | ||
+ | | ||
+ | | ||
+ | Performance: | ||
+ | </ | ||
+ | |||
+ | So in this case the calculation runs about three times faster with two GPU cards. | ||
+ | |||
+ | ===== Example batch file ===== | ||
+ | |||
+ | A job script for the example given above could look like: | ||
+ | |||
+ | < | ||
+ | #!/bin/bash | ||
+ | |||
+ | #SBATCH --mail-user=dreger@physik.fu-berlin.de | ||
+ | #SBATCH --mail-type=end | ||
+ | |||
+ | #SBATCH --output=job%j.out | ||
+ | #SBATCH --error=job%j.err | ||
+ | #SBATCH --ntasks=8 | ||
+ | #SBATCH --mem-per-cpu=1024 | ||
+ | #SBATCH --time=01: | ||
+ | #SBATCH --gres=gpu: | ||
+ | #SBATCH --nodes=1 | ||
+ | #SBATCH --partition=gpu-main | ||
+ | |||
+ | module load gromacs/ | ||
+ | |||
+ | TAG=" | ||
+ | |||
+ | grompp -f run.mdp -c waterbox.gro -p topol.top -o output-$TAG | ||
+ | mdrun -nt ${SLURM_CPUS_ON_NODE} -testverlet -v -deffnm output-$TAG | ||
+ | </ | ||
+ | |||
+ | Please make sure you change the email if you use this for your own tests ;) | ||
+ |
services/cluster/gpunodes.1434630509.txt.gz · Last modified: 2015/06/18 12:28 by dreger