User Tools

Site Tools


services:gitlab:gitlab-runner

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
services:gitlab:gitlab-runner [2021/05/19 09:37] hoffmac00services:gitlab:gitlab-runner [2022/07/08 15:58] (current) hoffmac00
Line 1: Line 1:
-====== Shared GitLab Runner of the Physics department ======+====== Shared GitLab Runners of the Physics Department ======
  
 All repositories have the ability to use the shared GitLab Runners. They can be used to compile, test and deliver code automatically. They must not be used for numerical computations. All repositories have the ability to use the shared GitLab Runners. They can be used to compile, test and deliver code automatically. They must not be used for numerical computations.
  
-===== Intro into GitLab CI (Continuous Integration) =====+===== Intro to GitLab CI (Continuous Integration) =====
  
 If CI is enabled and there is a push event, GitLab will automatically start a pipeline. A pipeline will run a series of specified jobs on the code and report errors it encounters. Using pipelines, you can see if a change causes an issue with existing code and you can test it for different environments. This works for all branches as well as merge requests or on a schedule. If CI is enabled and there is a push event, GitLab will automatically start a pipeline. A pipeline will run a series of specified jobs on the code and report errors it encounters. Using pipelines, you can see if a change causes an issue with existing code and you can test it for different environments. This works for all branches as well as merge requests or on a schedule.
Line 27: Line 27:
 </code> </code>
  
-This will define a job called **test** and in that job it will run ''%%./testscript%%'' from the root of the project. If this testscripts exits successfully, the job will succeed and so will the pipeline.+This will define a job called **test** and in that job it will run ''%%./testscript%%'' from the root of the project. If this testscript exits successfully, the job will succeed and so will the pipeline.
  
-Often you want to store the results of a job, for example to use it in further jobs. For this, you can use //artifacts//. Below is a sample configuration which will build a $LaTeX$ project with bibtex to a PDF and upload the resulting PDF to our GitLab instance.+Often you will want to store the results of a job, for example to use it in further jobs. For this, you can use //artifacts//. Below is a sample configuration that will build a $LaTeX$ project with bibtex to a PDF and upload the resulting PDF to our GitLab instance.
  
 <code yaml> <code yaml>
Line 121: Line 121:
 ===== Parallel and matrix ===== ===== Parallel and matrix =====
  
-Inside a job, you can use ''%%parallel%%'' to run it multiple times simultaneously. Some languages, e.g. ruby also have the ability to split tests across these parallel jobs.+Inside a job, you can use ''%%parallel%%'' to run it multiple times simultaneously. Some languages, e.g. ruby also have the ability to split tests across these parallel jobs.
  
 <code yaml> <code yaml>
Line 193: Line 193:
 will use the checksum of ''%%requirements.txt%%'' as the key. This means your cache only has to be recreated when ''%%requirements.txt%%'' changes. will use the checksum of ''%%requirements.txt%%'' as the key. This means your cache only has to be recreated when ''%%requirements.txt%%'' changes.
  
-Jobs that update the cache should always be idempotent (i.e. should not change anything when run again) and reuse existing files. This means you wouldn’t need to reinstall, say 2GiB in a python venv with every run of a pipeline but only update the existing modules.+Jobs that update the cache should always be idempotent (i.e. should not change anything when run again) and reuse existing files. This means you wouldn’t need to reinstall, say 2GiB in a python venv with every run of a pipeline but only update the existing modules.
  
 ===== Images ===== ===== Images =====
Line 199: Line 199:
 GitLab CI allows for choosing images. All available images are based on debian. There are multiple variants and multiple releases available. GitLab CI allows for choosing images. All available images are based on debian. There are multiple variants and multiple releases available.
  
-Available releases: * buster * bullseye * sid * stable (stable release, installed on workstations) * testing (next stable release)+Available releases: 
 +  * buster 
 +  * bullseye 
 +  bookworm (julia is not packaged for this release currently, therefore the julia variant is not available) 
 +  * oldstable (last stable release) 
 +  * stable (stable release, installed on workstations) 
 +  * testing (next stable release)
  
 These releases correspond to the debian codenames. These releases correspond to the debian codenames.
  
-Available variants: * base (simplest variant e.g. for simple shell scripts) * dev (for C/C++ projects) * python (comes with the same python libs as the workstations) * tex (comes with the same TeX packages as the workstations) * fortran (comes with gfortran) * haskell (comes with ghc) * julia (comes with julia) * ruby (comes with ruby) * deb (for debian packaging) * full (Includes everything from above and is close to the workstation configuration)+Available variants: 
 +  * base (most simple variant e.g. for simple shell scripts) 
 +  * dev (for C/C++ projects) 
 +  * python (comes with the same python libs as the workstations) 
 +  * tex (comes with the same TeX packages as the workstations) 
 +  * fortran (comes with gfortran) 
 +  * haskell (comes with ghc) 
 +  * julia (comes with julia) 
 +  * ruby (comes with ruby) 
 +  * deb (for debian packaging) 
 +  * full (Includes everything from above and is close to the workstation configuration)
  
 Launch time increases with the size of the image which is why the full variant will be slowest to start. Launch time increases with the size of the image which is why the full variant will be slowest to start.
Line 210: Line 226:
  
 <code yaml> <code yaml>
-image: buster-python+image: bullseye-python
 </code> </code>
  
Line 216: Line 232:
  
 <code yaml> <code yaml>
-image: buster-python+image: bullseye-python
  
 ... ...
  
 deploy: deploy:
-    image: buster-base+    image: bullseye-base
     script:     script:
         - ./deploy.sh         - ./deploy.sh
 </code> </code>
  
-This will use ''%%buster-python%%'' for all jobs, except ''%%deploy%%'', which will use the minimal base image.+This will use ''%%bullseye-python%%'' for all jobs, except ''%%deploy%%'', which will use the minimal base image.
  
 ===== Notes on Resource Usage ===== ===== Notes on Resource Usage =====
Line 235: Line 251:
   * Artifacts that are not meant to be investigated manually (which should be most, except final results like LaTeX pdfs) should have a low expiration time to save disk space.   * Artifacts that are not meant to be investigated manually (which should be most, except final results like LaTeX pdfs) should have a low expiration time to save disk space.
   * The maximum size for artifacts is 4 GiB.   * The maximum size for artifacts is 4 GiB.
-  * Each job has TBD Cores and TBD GiB RAM+  * Each job can use up to 8 Cores and GiB RAM
  
 ===== Pipeline Editor ===== ===== Pipeline Editor =====
services/gitlab/gitlab-runner.1621417039.txt.gz · Last modified: 2021/05/19 09:37 by hoffmac00

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki