Skip to content

WIP: proposed conda-based image

Anton Akhmerov requested to merge conda into master

I have played around quite a bit, and here's what I have so far. I'm not convinced this is the best path, so let me share my thoughts.

We want to test different dependency versions (including different python versions). The sources that provides said dependencies are: apt, pip, conda, manual installation. apt would require maintaining several different images and doesn't allow checking against pre-releases. pip isn't very reliable since we're at the mercy of the dependencies being pip-installable. This leaves conda and conda environments. Based on the image in the MR, it's easy to make Kwant run and build by using this variant of build.conf and corresponding LD_LIBRARY_PATH & LIBRARY_PATH.

The next part that we want to achieve is that it should be easy to plug-and-play different versions. Here's what we need:

  1. Install with latest cython and custom numpy & python
  2. test with same numpy & python as in 1. and custom scipy/mpl
  3. build docs with latest sphinx, same numpy, and custom scipy/mpl (we should probably stick to building the docs only once though), it seems enough.

We don't want to test all combinations, so I guess we'll have around 3x step 1, same amount of step 2, and 1x step 3.

How do we pass cache between different runs? .gitlab-ci.yml only allows to specify cache within the project folder, and I don't like messing with conda config to force it to use a different folder for cache. On the other hand, we do provide the kwant runner as a service to anyone else who wants to develop kwant. If they had access to the same cache, they would be able to contaminate cache with malicious dependency versions that disclose secret variables or something like that.

So we'd need to also set up the environments inside the Kwant project folder, which is where I am now.

Merge request reports