| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| services:cluster:start [2014/06/11 14:51] – [Information about the HPC-Cluster] dreger | services:cluster:start [2016/10/21 09:17] (current) – [General documentation] tomannecke |
|---|
| In order to get access to the department of physics HPC resources you need to send an email to hpc@physik.fu-berlin.de. Please give some information on the kind of jobs you are planing to run and software you plan to use, if possible. | In order to get access to the department of physics HPC resources you need to send an email to hpc@physik.fu-berlin.de. Please give some information on the kind of jobs you are planing to run and software you plan to use, if possible. |
| |
| **2014-06-11**: Currently there are two different HPC clusters in production at the physics department. The old cluster is still running Debian/Squeeze and uses Torque/MAUI as queueing system, while the new cluster is running Debian/Wheezy an uses Slurm as queueing system. Despite the fact that documentation for the new cluster is far from complete, we will create new accounts on the new system only. If you absolutely need access to the old system for some reason, drop us a note at hpc@physik.fu-berlin.de. | **2015-06-16**: Currently there are two HPC clusters in production at the physics department. One is located in the HLRN datacenter, the other is located at ZEDAT. See the table below for further information on available resources. Both cluster only share a common /home. Everything else like the queuing system ot /scratch are distinct. Both are running Debian/Wheezy and utilize the Slurm scheduling system. |
| |
| ===== Slurm documentation (new) ===== | ===== Slurm documentation ===== |
| |
| * [[important|Important notes]] on cluster usage | * [[important|Important notes]] on cluster usage |
| * Start with the [[slurm|Introduction to the Slurm HPC cluster]]. | * Start with the [[slurm|Introduction to the Slurm HPC cluster]]. |
| | * Using [[interactivesessions|interactive sessions]] with the queuing system. |
| ===== Torque documentation (old) ===== | * How to make use of the [[gpunodes|GPU-nodes]]. |
| | |
| * Use [[localstorage|local storage on the nodes]] if at all possible. | |
| * Start with the [[queuing-system|Introduction to the Torque HPC cluster]]. | |
| |
| ===== General documentation ===== | ===== General documentation ===== |
| |
| * Robert Hübener from AG-Eisert has written a HOWTO for using [[mmacluster|Mathematica on a HPC-Cluster]]. | * Robert Hübener from AG-Eisert has written a HOWTO for using [[mmacluster|Mathematica on a HPC-Cluster]]. |
| * A current python version as been built for cluster usage. The [[pythoncluster|Python on the HPC-Cluster]] tutorial describes how to set it up. | * A current python version has been built for cluster usage. The [[pythoncluster|Python on the HPC-Cluster]] tutorial describes how to set it up. |
| | * Try to [[usetmpforio|use /tmp for I/O intensive single node jobs]] |
| |
| ===== Overview of available resources ===== | ===== Overview of available resources ===== |
| |
| The following table lists all HPC resources available at the physics department. At the end of the table we also list the resources for the ZEDAT [[http://www.zedat.fu-berlin.de/Compute/Soroban|soroban]] cluster. The torque system will soon be replaced by SLURM. All torque nodes are running Debian/Squeeze, while all SLURM nodes are running Debian/Wheezy. | The following table lists all HPC resources available at the physics department. At the end of the table we also list the resources for the ZEDAT [[http://www.zedat.fu-berlin.de/HPC/Soroban|soroban]] cluster. The tron cluster at Takustrasse 9 is currently restructured and will most likely start running Debian/Jessie. |
| | |
| | ^ Hosts ^ Manager ^ Nodes ^ Form ^ Hardware ^ CPU ^ Speed ^ Core/Node ^ RAM/Core ^ RAM/Node ^ #RAM ^ #Cores ^ |
| | | @#cfc:**tron cluster** - FB Physik - Location: Takustrasse 9 |||||||||||| |
| | | @#cfc:n010-n041 | offline | 32 | 2U Twin<sup>2</sup> | Dell C6100 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 96G | 3072G | 384 | |
| | | @#cfc:n110-n111 | offline | 2 | 2U | Dell C6145 | 4x Opteron 6128HE | 2.0GHz | 32 | 4G | 128G | 256G | 64 | |
| | | @#cfc:n112-n127 | offline | 16 | Blade | Dell M600 | 2x Xeon E5450 | 3.00GHz | 8 | 2G | 16G | 256G | 128 | |
| | | @#cfc:n128-n143 | offline | 16 | Blade | Dell M600 | 2x Xeon E5450 | 3.00GHz | 8 | 2G | 16G | 256G | 128 | |
| | | @#cfc:n144-n175 | offline | 32 | Blade | Dell M610 | 2x Xeon X5570 | 2.93GHz | 8 | 6G | 48G | 1536G | 256 | |
| | | @#cfc:n176-n183 | offline | 8 | 4U | HP DL580 | 4x Xeon X7560 | 2.26Ghz | 32 | 8G | 256G | 2048G | 256 | |
| | | @#cfc:**#Taku9** | | **~~=sum(range(col(),1,col(),row()-1))~~** | | | | | | | | **~~=sum(range(col(),1,col(),row()-1))~~G** | **~~=sum(range(col(),1,col(),row()-1))~~** | |
| | | | | | | | | | | | | | | |
| | | @#ced:**sheldon cluster** - FB Physik - Location: HLRN |||||||||||| |
| | | @#ced:x001-x192 | SLURM<sup>1)</sup> | 192 | Blade | SGI Altix ICE 8200 | 2x Xeon X5570 | 2.93GHz | 8 | 6G | 48G | 9216G | 1536 | |
| | | @#ced:uv1000 | none | 1 | 42U | SGI UV 1000 | 64x Xeon X7560 | 2.26Ghz | 512 | 4G | 2T | 2048G | 512 | |
| | | @#ced:**#HLRN** | | **193** | | | | | | | | **11264G** | **2048** | |
| | | | | | | | | | | | | | | |
| | | @#cef:**yoshi cluster** - FB Physik - Location: ZEDAT |||||||||||| |
| | | @#cef:y001-y128 | SLURM<sup>1)</sup> | 128 | Blade | HP BL460c G6 | 2x Xeon X5570 | 2.93GHz | 8 | 6G | 48G | 6144G | 1024 | |
| | | @#cef:ygpu01-ygpu31 | SLURM<sup>2)</sup> | 31 | 2U GPU Nodes (2x Nvidia Tesla M2070) | IBM iDataPlex dx360 M3 | 2x Xeon X5570 | 2.93GHz | 8 | 3G | 24G | 744G | 248 | |
| | | @#cef:**#Ph-ZEDAT** | | **159** | | | | | | | | **6888G** | **1272** | |
| | | | | | | | | | | | | | | |
| | | @#ccf:**soroban cluster** - ZEDAT-HPC - Location: ZEDAT |||||||||||| |
| | | @#ccf:node001-002 | SLURM | 2 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 48G | 96G | 24 | |
| | | @#ccf:node003-030 | SLURM | 28 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 4G | 24G | 672G | 336 | |
| | | @#ccf:node031-100 | SLURM | 70 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 48G | 3360G | 840 | |
| | | @#ccf:node101-112 | SLURM | 12 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 16G | 96G | 1152G | 144 | |
| | | @#ccf:**#ZEDAT** | | **112** | | | | | | | | **5280G** | **1344** | |
| | | | | | | | | | | | | | | |
| | | @#ccc:**ausgemusterte Systeme** |||||||||||| |
| | | @#ccc:Abacus4 | | 8 | | IBM p575 | 16x POWER 5+ | 1.9Ghz | 32 | 4G | 128G | 1024G | 256 | |
| | |
| | |
| | |
| | |
| | |
| |
| ^ Hosts ^ Manager ^ Nodes ^ Form ^ Hardware ^ CPU ^ Speed ^ Core/Node ^ RAM/Core ^ RAM/Node ^ #RAM ^ #Cores ^ | |
| | @#cfc:**FB Physik - Standort Takustrasse 9** |||||||||||| | |
| | @#cfc:n010-n041 | Torque | 32 | 2U Twin<sup>2</sup> | Dell C6100 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 96G | 3072G | 384 | | |
| | @#cfc:n042-n079 | Torque | 38 | 1U Twin | Supermicro X7DWT | 2x Xeon E5440 | 2.83GHz | 8 | 4G | 32G | 1216G | 304 | | |
| | @#cfc:n110-n111 | Torque | 2 | 2U | Dell C6145 | 4x Opteron 6128HE | 2.0GHz | 32 | 4G | 128G | 256G | 64 | | |
| | @#cfc:n112-n127 | Torque | 16 | Blade | Dell M600 | 2x Xeon E5450 | 3.00GHz | 8 | 2G | 16G | 256G | 128 | | |
| | @#cfc:n128-n143 | SLURM<sup>1)</sup> | 16 | Blade | Dell M600 | 2x Xeon E5450 | 3.00GHz | 8 | 2G | 16G | 256G | 128 | | |
| | @#cfc:n144-n175 | SLURM<sup>1)</sup> | 32 | Blade | Dell M610 | 2x Xeon X5570 | 2.93GHz | 8 | 6G | 48G | 1536G | 256 | | |
| | @#cfc:n176-n183 | SLURM<sup>2)</sup> | 8 | 4U | HP DL580 | 4x Xeon X7560 | 2.26Ghz | 32 | 8G | 256G | 2048G | 256 | | |
| | @#cfc:**#Taku9** | | **~~=sum(range(col(),1,col(),row()-1))~~** | | | | | | | | **~~=sum(range(col(),1,col(),row()-1))~~G** | **~~=sum(range(col(),1,col(),row()-1))~~** | | |
| | | | | | | | | | | | | | | |
| | @#cee:**FB Physik - Standort HLRN** |||||||||||| | |
| | @#cee:x001-x192 | SLURM<sup>1)</sup> | 192 | Blade | SGI Altix ICE 8200 | 2x Xeon X5570 | 2.93GHz | 8 | 6G | 48G | 9216G | 1536 | | |
| | @#cee:uv1000 | SLURM<sup>1)</sup> | 1 | 42U | SGI UV 1000 | 64x Xeon X7560 | 2.26Ghz | 512 | 4G | 2T | 2048G | 512 | | |
| | @#cee:**#HLRN** | | **193** | | | | | | | | **11264G** | **2048** | | |
| | | | | | | | | | | | | | | |
| | @#ccf:**ZEDAT-HPC** |||||||||||| | |
| | @#ccf:node001-002 | SLURM | 2 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 48G | 96G | 24 | | |
| | @#ccf:node003-030 | SLURM | 28 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 4G | 24G | 672G | 336 | | |
| | @#ccf:node031-100 | SLURM | 70 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 8G | 48G | 3360G | 840 | | |
| | @#ccf:node101-112 | SLURM | 12 | 1U Twin | Asus Z8NH-D12 | 2x Xeon X5650 | 2.66GHz | 12 | 16G | 96G | 1152G | 144 | | |
| | @#ccf:Abacus4 | | 8 | | IBM p575 | 16x POWER 5+ | 1.9Ghz | 32 | 4G | 128G | 1024G | 256 | | |
| | @#ccf:**#ZEDAT** | | **120** | | | | | | | | **6304G** | **1600** | | |
| |
| |
