Differences

This shows you the differences between two versions of the page.

--- services:cluster:pythoncluster [2014/01/17 16:44] – created behrmj87
+++ services:cluster:pythoncluster [2024/04/26 14:39] (current) – hoffmac00
@@ Line 1: / Line 1: @@
+<note warning>
+This information is outdated, please use the system python version already included in debian for our bookworm clusters
+</note>
 ======Python on the Cluster======
-This document describes how to use python for numerical and/or batch jobs on the cluster with a heavy focus on using IPython for management of the multiprocessing. Other solutions are possible, but not the focus of this introductory document. Furthermore, this document focuses on how to setup a parallel IPython environment, but will not discuss its usage. The usage is
+This document describes how to use python for numerical and/or batch jobs on the cluster with a heavy focus on using IPython for management of the multiprocessing. Other solutions are possible, but not the focus of this introductory document. Furthermore, this document focuses on how to setup a parallel IPython environment, but will not discuss its usage.
-This is version 0.1(1389963359) of this document. If you have any suggestions, send them to <behrmann@physik.fu-berlin.de>.
+This is version 0.2(1390214905) of this document. If you have any suggestions, send them to <behrmann@physik.fu-berlin.de>.
 =====Setting up the Environment=====
@@ Line 45: / Line 50: @@
 </code>
 which will install the package  //packagename// into ''$HOME/.local/lib/python2.7/site-packages''. If you install packages locally to your home folder, than you need to change the ''PYTHONPATH'' variable mentioned above as follows ''PYTHONPATH=${HOME}/.local/lib/python2.7/site-packages:/opt/python2.7/lib/python2.7/site-packages''.
+The subset of already installed packages can also be found packaged as ''wheel''-packages [[services:cluster:pythoncluster#Sources|[1]]] in ''/opt/python2.7/share/wheel'' and can be installed with the command
+<code>
+pip install --user --ignore-installed --use-wheel --no-index --find-links=/opt/python2.7/share/wheel packagename
+</code>
+This is for particular use in virtualenvs, for which, for one reason or the other you do not want to use the provided system packages.
 =====Setting up IPython=====
@@ Line 60: / Line 71: @@
 patch < /opt/python2.7/share/configs/ipcluster/ipcluster_config.patch
 patch < /opt/python2.7/share/configs/ipcluster/ipcontroller_config.patch
+patch < /opt/python2.7/share/configs/ipcluster/ipengine_config.patch
 </file>
 which will immediately patch the the configuration to start the engines using ''mpiexec'' and the controller to listen not only on the local host, which is important if you are using multiple nodes.
@@ Line 73: / Line 85: @@
 Both files are a recommended reading for further options that can be changed.
+Furthermore, you may want to set ''c.IPEngineApp.startup_script = u'/path/to/my/startup.py''' to a startup script for importing all needed packages into the engines.
 =====Using IPython on the Cluster=====
@@ Line 163: / Line 177: @@
 MPI can be used with IPython, as described [[http://ipython.org/ipython-doc/stable/parallel/parallel_mpi.html|here]]. An advanced example of using IPython in a parallel workflow, both with and without MPI can be found [[https://github.com/ipython/ipython/tree/master/examples/parallel/wave2D|here]].
+====Sources====
+[[http://wheel.readthedocs.org/en/latest/|[1]]] Documentation for the wheel package