Verified Commit ad86acf0 authored by Adel Kara Slimane's avatar Adel Kara Slimane
Browse files

Add Docker config files and gitlab-ci file to enable pipelines

parent e6795173
Pipeline #35238 failed with stages
in 7 seconds
image: gitlab.kwant-project.org:5005/kwantoperator/debian
stages:
- build
- test
# Build
build:package:
stage: build
script:
- python3 setup.py build
- python3 setup.py build_ext -i
artifacts:
untracked: true
expire_in: 1 hour
# Test
test:package:
stage: test
script:
- py.test -rs --cache-clear
build:documentation:
stage: test
script:
- make -C doc clean ; make -C doc doctest html
artifacts:
paths:
- doc/build/html/
expire_in: 1 week
\ No newline at end of file
# Docker image for building and testing tkwantoperator
FROM debian:stable
MAINTAINER Kwant developers <authors@kwant-project.org>
RUN apt-get update && apt-get install -y --no-install-recommends\
# CI requirements
apt-transport-https file openssh-client rsync\
# all the hard non-Python dependencies
git g++ make patch gfortran libblas-dev liblapack-dev librsb-dev\
libmumps-scotch-dev pkg-config libfreetype6-dev texlive texlive-latex-extra\
dvipng\
# all the hard Python dependencies -- use the distribution's
# repositories so we don't rely on versions that are too new
python3-all-dev python3-pip python3-setuptools python3-tk python3-matplotlib\
cython3 python3-numpy python3-scipy python3-mpi4py python3-sympy python3-dill
RUN pip3 install wheel
# install tkwant along with its dependencies
RUN pip3 install --upgrade\
git+https://gitlab.kwant-project.org/kwant/tkwant.git
# install testing and documentation building dependencies
# remove deprecated pytest-flakes pytest-pep8 at some point
RUN pip3 install --upgrade\
pytest pytest-cov pytest-flake8 pytest-flakes pytest-pep8\
# and documentation building dependencies
sphinx sphinx_rtd_theme numpydoc nbsphinx ipykernel jupyter_client\
jupyter-sphinx
## Dockerfiles
This folder contains a set of docker config files, usually named [Dockerfile](https://docs.docker.com/engine/reference/builder/), that are used to produce container images which then will serve as environments for running programs.
Docker images are used in Gitlab CI runners to do pipeline tests : Docker images are specified at each `image: foo` line in the `.gitlab-ci.yml` file (at the root of this repository), the image is then used as an environment to run scripts in. In our specific case, we use Docker images to test that our code 'works' but also to check compatibility with specific versions of specific Linux distributions.
### Modifying a Dockerfile
Each docker file is named after the Linux distribution it bundles (ont its latest stable version) using the following pattern `Dockerfile.${Linux_distro_name}` (in lowercase), for example `Dockerfile.debian`.
To learn more about the actual writing of a Dockerfile, a start can be here: https://docs.docker.com/engine/reference/builder/
### Create a Docker image from a config file and upload it
From a given Dockerfile in this repository, one can locally (in his own machine) create a Docker image then upload it the Gitlab repository to be used by the runners for the pipeline.
To create a Docker image, one needs to have Docker installed. On Debian derivatives:
```shell
$ sudo apt install docker.io
$ sudo systemctl enable --now docker
```
Then, one would do the following commands, for the file `Dockerfile.debian`:
```shell
$ cd path/to/tkwantoperator/
$ sudo docker build -t gitlab.kwant-project.org:5005/kwant/tkwantoperator/debian -f docker/Dockerfile.debian .
$ sudo docker login gitlab.kwant-project.org:5005
$ sudo docker push gitlab.kwant-project.org:5005/kwant/tkwantoperator/debian
$ sudo docker logout gitlab.kwant-project.org:5005
```
#### Some problems that may be encountered
Docker tries to upload several files at once when pushing, if kept at its default configuration. And that can lead to aborts if the server is busy or refuses multi-uploads. If it is happening, one can tweak this this behavior for the push to work. In a shell, one can stop the Docker service, change a config file, and then re-start the service:
```shell
$ sudo systemctl stop docker
$ sudo nano /etc/docker/daemon.json
```
Then add the two following lines:
```json
{
"max-concurrent-uploads": 1,
"max-concurrent-downloads": 1
}
```
Then save and exit (`CTRL+X` then `Y` then `ENTER`) then restart the docker service:
```shell
$ sudo systemctl start docker
```
The previous push command should work after that.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment