kwant issueshttps://gitlab.kwant-project.org/kwant/kwant/-/issues2020-10-26T15:23:39Zhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/395Improve Hamiltonian evaluation for vectorized systems2020-10-26T15:23:39ZChristoph GrothImprove Hamiltonian evaluation for vectorized systemsSo far, the new `system.VectorizedSystem` offers a `hamiltonian_submatrix` method that offers a subset of the functionality of the eponymous method of `system.System`.
This is a good first step, but we are not bound by backwards compati...So far, the new `system.VectorizedSystem` offers a `hamiltonian_submatrix` method that offers a subset of the functionality of the eponymous method of `system.System`.
This is a good first step, but we are not bound by backwards compatibility here (the new `hamiltonian_submatrix` already breaks it), since there is no legacy code that enables vectorization.
This gives as a unique opportunity to modernize the API for evaluating Hamiltonians.
Here are some problems with the old API:
- It's incompatible with ND-systems.
- It's not possible to selectively evaluate terms (for example only terms that depend on a particular parameter could be of interest).
- By default, a dense matrix is returned (#244).
- It's impossible to act with the Hamiltonian on a vector without creating a sparse matrix in memory.
- `inter_cell_hopping` may return non-square matrices.vectorizationhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/368Using smatrix on a system with a SelfEnergyLead fails2024-01-09T20:07:17ZJoseph WestonUsing smatrix on a system with a SelfEnergyLead failsI don't have time to make an example right now, but if you construct a system that has a `kwant.builder.SelfEnergyLead` as one of the leads, and call `kwant.smatrix` on it Kwant will raise an `AttributeError` because the `SelfEnergyLead`...I don't have time to make an example right now, but if you construct a system that has a `kwant.builder.SelfEnergyLead` as one of the leads, and call `kwant.smatrix` on it Kwant will raise an `AttributeError` because the `SelfEnergyLead` does not have a `modes` method.
IMO this is a bug. As long as you have at least 1 lead that has `modes`, I would expect everything to work.
A naïve fix is simple: simply evaluate the self-energy of the lead and add it to the scattering region Hamiltonian.
What is a bit trickier is how to handle lead numbering. Imagine I had 3 leads attached (0, 1, 2) and lead 1 is a self-energy lead.
When asking for submatrices of the scattering matrix we would be "unable" to use lead index 1, as we have no way of getting the scattering matrix elements to the SelfEnergyLead (because it does not have `modes` defined).
What we could do is raise an exception if `smatrix` is used and there is a `SelfEnergyLead` and no `in_lead` and `out_leads` have been provided, notifying the user that they must specify this, omitting the self-energy lead. AFAICT this would solve the numbering problem.https://gitlab.kwant-project.org/kwant/kwant/-/issues/190Add Kwant to Zenodo2019-03-01T16:37:02ZJoseph WestonAdd Kwant to Zenodo[Zenodo](https://zenodo.org/) allows us to get DOIs for Kwant releases.
Publishing Kwant releases there will be helpful for reproducible computing, as people may cite the exact Kwant release that they used.
It is a bit of manual work t...[Zenodo](https://zenodo.org/) allows us to get DOIs for Kwant releases.
Publishing Kwant releases there will be helpful for reproducible computing, as people may cite the exact Kwant release that they used.
It is a bit of manual work to fill in all the fields, but the only unanswered question is whether to open a new account on that platform (and then somehow share the credentials between the authors).Kwant 1.4https://gitlab.kwant-project.org/kwant/kwant/-/issues/343Consider adding a global switch for vectorization2020-02-13T09:59:07ZAnton AkhmerovConsider adding a global switch for vectorizationRight now there's a per-builder switch `vectorized=False`. @slavoutich correctly observed that vectorized code is unlikely to be mixed with unvectorized code, so in order to reduce boilerplate and ease the transition to the vectorized fo...Right now there's a per-builder switch `vectorized=False`. @slavoutich correctly observed that vectorized code is unlikely to be mixed with unvectorized code, so in order to reduce boilerplate and ease the transition to the vectorized format we may instead implement this flag as a global configuration option and instead require that the user calls `kwant.enable_vectorization()`.vectorizationhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/233Remove `show=...` argument from Kwant plotter2019-02-10T00:13:12ZAnton AkhmerovRemove `show=...` argument from Kwant plotterMatplotlib's own plotting functions have a separate logic for when to show figures or not. As far as I understand, we don't gain anything by adding a `show` argument to the plots. At the same time, because the figures we create are withi...Matplotlib's own plotting functions have a separate logic for when to show figures or not. As far as I understand, we don't gain anything by adding a `show` argument to the plots. At the same time, because the figures we create are within pyplot's figure registry, we now may do things like
```python
kwant.plot(syst, show=False)
pyplot.title('My awesome system')
```
If we remove `show`, the extra argument will not be needed. The downside is that the users running a *non-interactive* session will need to call `pyplot.show()` manually. This is consistent with how matplotlib works.Plotly PlotterAnton AkhmerovAnton Akhmerovhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/108Interface compatibility between Lattice.shape and Builder.fill2017-05-11T15:20:59ZViacheslav OstroukhInterface compatibility between Lattice.shape and Builder.fillThere were two minor changes in interface:
- `lat.shape` accepts real-space position in shape function, `builder.fill` accepts site. This is a good change, as for me, and allows shorter code in some cases. However, it may be reasonable t...There were two minor changes in interface:
- `lat.shape` accepts real-space position in shape function, `builder.fill` accepts site. This is a good change, as for me, and allows shorter code in some cases. However, it may be reasonable to keep backward compatibility and check type of an argument for the purpose of not annoying users (at least before issuing Kwant 2).
- `lat.shape` accepted real-space position of an origin for flood-fill, `builder.fill` accepts site tag. When (0, 0) can't be chosen as an origin, user has routinely to use something like:
```python
lat = next(iter(proto.sites())).family
start = lat.closest((x0, y0))
```
I don't see any good reason for not doing this automatically in `kwant.Builder.fill` method and keep old-style interface.Kwant 1.3https://gitlab.kwant-project.org/kwant/kwant/-/issues/63Be more consistent in dependency specification2020-12-14T10:37:35ZAnton AkhmerovBe more consistent in dependency specificationKwant >=1.2 depends on python v3.4 released March 16th, 2014. At that time the following versions of scipy libraries already existed:
* matplotlib v1.3.1 (but we don't support it due to bugs).
* scipy v0.13.3.
* numpy v1.8.1.
The...Kwant >=1.2 depends on python v3.4 released March 16th, 2014. At that time the following versions of scipy libraries already existed:
* matplotlib v1.3.1 (but we don't support it due to bugs).
* scipy v0.13.3.
* numpy v1.8.1.
The dependencies we list in setup.py are:
* mpl >=1.2.
* scipy >=0.11.0
* numpy > 1.6.1.
An obvious improvement would to bump all of these versions, but we can also consider being more proactive in increasing the dependencies if there is something to be gained.
EDIT: fixed the version numbers (I wasn't thorough enough checking them initially), and slightly rephrased the description.https://gitlab.kwant-project.org/kwant/kwant/-/issues/372Use structural subtyping for defining interfaces to the different parts of Kwant2020-04-01T16:46:58ZJoseph WestonUse structural subtyping for defining interfaces to the different parts of Kwant## Problem definition
In principle Kwant has several more or less independent components separated by interfaces: Builder, System, solvers, plotter
In practice the specific interfaces that are used are only loosely defined, and it is q...## Problem definition
In principle Kwant has several more or less independent components separated by interfaces: Builder, System, solvers, plotter
In practice the specific interfaces that are used are only loosely defined, and it is quite challenging (even for one well-versed in the internals of Kwant) to create new classes which adhere to the correct interface.
Take, for example `kwant.system.FiniteSystem` which can, in principle, be plotted by `kwant.plot` if it provides a `pos` method (this is not even documented in the docstring of `plot`; it just says that the `sys` parameter may be a `Builder` or `system.Finitesystem`). However, if the leads do not provide a `symmetry` attribute that has the interface of a `kwant.lattice.TranslationalSymmetry` then they are simply not plotted.
This is not documented in `plot`. Even if it were, having ad-hoc interfaces defined in docstrings is probably not the way to go.
In practice people use Kwant as a monolithic package, and it is often unacceptable to e.g. be able to define custom system classes but be unable to plot them properly.
## Proposed Solution
I propose to use structural subtyping a la [PEP 544](https://www.python.org/dev/peps/pep-0544) to allow use to more precisely specify the interfaces required by the different parts of Kwant.
We can either wait until we can rely on Python 3.8, and make use of [`typing.Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol) or roll our own with `__subclass_hook__`.https://gitlab.kwant-project.org/kwant/kwant/-/issues/364Consider switching to "src" package layout and using "tox" for test running2020-03-18T17:53:42ZJoseph WestonConsider switching to "src" package layout and using "tox" for test runningWhen hacking on Kwant it is recommended [by the Kwant docs](https://kwant-project.org/contribute#setting-up-for-development) to use `python setup.py build_ext --inplace && pytest` for testing.
There are a couple of problems I have obser...When hacking on Kwant it is recommended [by the Kwant docs](https://kwant-project.org/contribute#setting-up-for-development) to use `python setup.py build_ext --inplace && pytest` for testing.
There are a couple of problems I have observed with this workflow:
1. the `build_ext --inplace` step is kind of non-standard, and if done incorrectly (e.g. by omitting the `--inplace`) then `python -c 'import kwant'` complains about not importing Kwant from the correct place (this has confused me more times than I care to mention)
2. placing the `.c` files next to the source an be problematic. `Cython` is not re-run if the associated `.pyx` file has not changed, but if we are using different versions of Cython, or targeting different CPython APIs (can happen when jumping around between `stable`/`master`) then the `.c` files are *still* not regenerated.
3. locally testing against different environments becomes a pain (manually deactivating / reactivating different Conda environments / whoops I just wasted 10 minutes because I was in the wrong environment)
I believe that points 1 and 3 can be addressed by using [the `src` layout](https://hynek.me/articles/testing-packaging/) (i.e. have the Kwant source code under `src/kwant` rather than directly in `kwant`) and by using an environment-aware test-runner such as [tox](https://tox.readthedocs.io/en/latest/) (which also has support for Conda environments).
I am not sure if this will address 2, but 2/3 ain't bad.https://gitlab.kwant-project.org/kwant/kwant/-/issues/237name of Builder.substitute2018-10-12T17:55:55ZAnton Akhmerovname of Builder.substituteThe method name `Builder.substitute` may give an impression that the parameter name substitutions are done in-place. Instead this operation returns a copy (as it probably should).
We should *consider* if there is a less ambiguous name b...The method name `Builder.substitute` may give an impression that the parameter name substitutions are done in-place. Instead this operation returns a copy (as it probably should).
We should *consider* if there is a less ambiguous name before %"Kwant 1.4".Kwant 1.4Anton AkhmerovAnton Akhmerovhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/148Polyatomic lattices could automatically create attributes (a, b, c, etc.) for...2017-07-20T15:22:44ZChristoph GrothPolyatomic lattices could automatically create attributes (a, b, c, etc.) for all of their sublatticesSo far, this feature is only provided by the likes of `kwant.lattice.honeycomb()`, but I guess it would be useful generally, and probably it wouldn't hurt. It would be also easier to explain. Does anyone see any problems with this? If...So far, this feature is only provided by the likes of `kwant.lattice.honeycomb()`, but I guess it would be useful generally, and probably it wouldn't hurt. It would be also easier to explain. Does anyone see any problems with this? If, for example, there is some lattice whose sublattices are habitually called X and Y, this could be confusing.https://gitlab.kwant-project.org/kwant/kwant/-/issues/141Color sites by family by default in 'kwant.plot'2017-09-14T10:32:59ZJoseph WestonColor sites by family by default in 'kwant.plot'This will provide more information in the default plot.
This could be implemented in 2 ways:
1. Assign site families colors in the order that they come out of the system (i.e. the first time a new site family is seen
it is assigned ...This will provide more information in the default plot.
This could be implemented in 2 ways:
1. Assign site families colors in the order that they come out of the system (i.e. the first time a new site family is seen
it is assigned the next color in some sequence)
2. First sort the site families, then assign them colors
The advantage of first sorting is that identical systems constructed in 2 different ways (imagine adding sites from family A before those from B, then another system doing the reverse)
will have the same site coloring, however we would have to iterate over the sites first (`O(N)`) in order to get all the site families.https://gitlab.kwant-project.org/kwant/kwant/-/issues/137Factor the Brillouin zone logic out of wraparound.2019-02-09T23:09:20ZAnton AkhmerovFactor the Brillouin zone logic out of wraparound.Calculation of a Brillouin zone probably belongs in `lattice`, but whether it should play with `TranslationalSymmetry` or `PolyatomicLattice` depends on #73. Also we should probably keep in mind further symmetry generalizations planned f...Calculation of a Brillouin zone probably belongs in `lattice`, but whether it should play with `TranslationalSymmetry` or `PolyatomicLattice` depends on #73. Also we should probably keep in mind further symmetry generalizations planned for #38.futurehttps://gitlab.kwant-project.org/kwant/kwant/-/issues/81providing params via system interface2020-05-08T21:02:13ZAnton Akhmerovproviding params via system interfaceAfter #77 the call signature of `kwant.smatrix` and other consumers of `System` have a rather lengthy call signature: `kwant.smatrix(syst, energy, args=..., params=...)`. It is counter-intuitive to supply arguments that the system consum...After #77 the call signature of `kwant.smatrix` and other consumers of `System` have a rather lengthy call signature: `kwant.smatrix(syst, energy, args=..., params=...)`. It is counter-intuitive to supply arguments that the system consumes using awkward syntax and via an unrelated function.
I propose an alternative pathway for doing so. A method `System.update_parameters(*args, **kwargs)` that will return a shallow copy of the system that stores its arguments and keyword arguments such that the user doesn't need to provide them elsewhere. Example usage:
```python
[kwant.smatrix(syst.update_parameters(0, B=5)) for B in fields]
```
Extra considerations
* `System`s already have a method `precalculate` that returns a modified shallow copy.
* This change is fully backwards-compatible (old way of providing parameters would still work).futurehttps://gitlab.kwant-project.org/kwant/kwant/-/issues/393Improve user experience when dealing with densities2020-10-23T15:46:27ZChristoph GrothImprove user experience when dealing with densitiesThe original Kwant made it quite easy to define TB systems in terms of sites and to evaluate transport properties like transmission probabilities between leads. In this workflow low-level systems and their inner workings were an impleme...The original Kwant made it quite easy to define TB systems in terms of sites and to evaluate transport properties like transmission probabilities between leads. In this workflow low-level systems and their inner workings were an implementation detail of Kwant.
When the operator module was added, it was natural for it to deal with low-level systems. However this meant that suddenly the user was exposed to details of low-level systems like the order of sites and their indices.
We would like to provide an intuitive way to handle densities, and the [vectorization effort](https://gitlab.kwant-project.org/kwant/kwant/-/milestones/9) seems like a good moment to introduce a new improved API because we have to [review operators for ND systems](#333) anyway.
To proposals exist that offer different solutions to this problem:
- #9
- #65
I propose to discuss the topic generally here and decide on a course of action. At the end of the discussion, one of the above two issues could be updated, and the other closed (along with this one).vectorizationhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/379Remove link to INAC website2020-05-13T15:37:22ZJoseph WestonRemove link to INAC websiteThe link to @waintal's homepage is [causing the linkcheck build step to fail](https://gitlab.kwant-project.org/kwant/kwant/-/jobs/120706).
It seems that http://inac.cea.fr is down and has been for a few weeks. We should either point to ...The link to @waintal's homepage is [causing the linkcheck build step to fail](https://gitlab.kwant-project.org/kwant/kwant/-/jobs/120706).
It seems that http://inac.cea.fr is down and has been for a few weeks. We should either point to Xavier's actual homepage or remove the link.https://gitlab.kwant-project.org/kwant/kwant/-/issues/367Treat deprecation warnings as errors in the tests2020-03-20T17:24:34ZJoseph WestonTreat deprecation warnings as errors in the tests#366 highlights that our tests do not catch a certain subset of potential bugs that (IMO) they ought to.
One way to fix this specific class of bugs (where we are using deprecated API) is to make deprecation warnings errors in the tests....#366 highlights that our tests do not catch a certain subset of potential bugs that (IMO) they ought to.
One way to fix this specific class of bugs (where we are using deprecated API) is to make deprecation warnings errors in the tests.
I haven't looked but I am almost certain pytest can be configured to do this.
~"Needs Discussion" because we might also catch lots of cruft with this.https://gitlab.kwant-project.org/kwant/kwant/-/issues/365Keep separate testing images for "master" and "stable"2020-03-30T12:56:20ZJoseph WestonKeep separate testing images for "master" and "stable"Currently we test against the different supported environments (Debian / Ubuntu / Conda on Debian) by having separate Docker images, which are stored [in the container registry](https://gitlab.kwant-project.org/kwant/kwant/container_regi...Currently we test against the different supported environments (Debian / Ubuntu / Conda on Debian) by having separate Docker images, which are stored [in the container registry](https://gitlab.kwant-project.org/kwant/kwant/container_registry).
There is a manual job that can be triggered to build these images (from the Dockerfiles in `docker/`), and which then pushes the images to the registry.
However we do not currently do any kind of versioning of these images. Each time the image-build jobs are run they overwrite whatever images are currently in the registry.
This is not great, as in principle `stable` and `master` need different testing environments. For example, `stable` should work with whatever the minimum requirements were at the time the last *minor* Kwant version was released. We are, however, allowed to update the minimum requirements of `master` (e.g. if the supported OS's bumped the package versions in their repos). Currently, for example, we are not actually testing `stable` against the minimum requirements as stated in the documentation (in the state the documentation is on `stable).
Fixing this is in principle quite simple, but comes with its own problems.
In the `.gitLab-ci.yml` file there are lines like this:
```
.ubuntu-env: &ubuntu_env
image: gitlab.kwant-project.org:5005/kwant/kwant/ubuntu
```
We would have to modify these lines and stick a `:stable` or something on the end, like:
```
.ubuntu-env: &ubuntu_env
image: gitlab.kwant-project.org:5005/kwant/kwant/ubuntu:stable
```
Simple enough, right?
Except that whenever we merge `stable` into `master` (or vice versa) we would have to ensure that these lines are not taken over (i.e. that we do not have the `:stable` images used by `master` or vice versa). Git's default merge strategy will, unfortunately, work against us.
In principle only a few people are merging `master` and `stable` together, and as long as they are aware of this it should "all be good" (and if we mess up it's easy to fix in a new commit), but it still makes me a little uneasy.
It is also not possible (AFAIK) to tell git to apply a particular merge strategy (`ours` in this case) to individual files.https://gitlab.kwant-project.org/kwant/kwant/-/issues/355Vectorization of builders2020-05-07T20:34:58ZChristoph GrothVectorization of buildersCurrently, builders store the Hamiltonian as a Python dictionary of adjacency lists (interleaved with values), as described in a comment in `builder.py`:
```
# For a given `tail` site, H[tail] is a list alternately storing
# head...Currently, builders store the Hamiltonian as a Python dictionary of adjacency lists (interleaved with values), as described in a comment in `builder.py`:
```
# For a given `tail` site, H[tail] is a list alternately storing
# heads and values. (The heads occupy even locations followed by the
# values at odd locations.) Each pair of entries thus describes a single
# directed edge of the graph.
```
Dealing with a system of N sites requires O(N) Python operations. With the vectorization of low-level systems, this can become a bottleneck, or at least a nuisance.
Since we need update the builder finalization code anyway (so that builders with more than a single translational symmetry can be finalized), why not try to solve both problems in one go and vectorize builders?
Fortunately, in terms of API (almost) nothing needs to change. Already now builders offer a vectorized interface in that many elements can be set with a single Python operation: a builder can be indexed with a sequence of sites. A natural evolution of this would be to also allow sitearrays and "tuple arrays", i.e. 2-tuples of sitearrays. This would allow to vectorize `shape`, `neighbors`, and `fill` and as such most old user scripts, without changing anything on the user side.
The more complicated part is reworking the data structures used internally by builders to efficiently support vectorization. Let’s try to analyze in a language-agnostic way what data structure would be best suited:
- We should optimize for a small number of site families, just like we already do in the new low-level format. This means that instead of storing a single graph indexed by (pairs of) sites, we could store separate graphs for each pair of families. These vertices of these graphs would be indexed by tags. I think that we could restrict valid tags to sequences of numbers without loosing anything. Together, this means that what we need is an efficient data structure to store graphs over vertices that are identified by sequences of numbers.
- We could also optimize for a small number of different values, since this is the most common usage by far. Ideally, the use case of having a different constant value for each site/hopping would be still viable, but I am not even sure if such systems were ever made.
A hash table of adjacency lists still seems the right data structure, only that it should be vectorized and it would be indexed by sequences of numbers (of same length) and not by generic Python objects.
I can’t see how to implement something like this efficiently in pure Python, but writing it as a C extension is not a very big deal - much simpler then, say, writing tinyarray. Perhaps, actually, a vectorized dictionary with keys that are sequences of numbers would be a useful addition to tinyarray?vectorizationhttps://gitlab.kwant-project.org/kwant/kwant/-/issues/352Idea: reduce the number of options through "configuration variables"2020-02-27T10:59:56ZChristoph GrothIdea: reduce the number of options through "configuration variables"Some Kwant functions take options that are rarely set, and if so, they are often set for a whole session. The prime example is the `show` argument to all plotting functions, but there's also `dpi`, `fig_size`, and perhaps also `cmap` fo...Some Kwant functions take options that are rarely set, and if so, they are often set for a whole session. The prime example is the `show` argument to all plotting functions, but there's also `dpi`, `fig_size`, and perhaps also `cmap` for example. Outside of plotter, there are parameters to the MUMPS solver like `nrhs` that are very rarely set, because it's cumbersome to do so.
Another problem with these options is that API-functions that call other API-functions (like `plotter.current` that calls `plotter.streamplot` and `plotter.interpolate_current`) need to re-expose options if the user is able to set them, or pass on kwargs. This leads to the kind of kwargs-hell known from matplotlib.
Now we could replace these options with global variables, and that could be done even in a backwards-compatible way (because the options would remain, and could even have precedence). Something like this:
```
old_value = kwant.plotter.show
kwant.plotter.show=False
# code that is executed with show set to False.
kwant.plotter.show=old_value
```
But this solution has several grave problems. Notably, there's no error when mistyping, and it's a mess when different configuration values are used throughout the same program. With async/multithreaded programs it becomes unmanageable.
Here is a possible solution that has been haunting me for some time: we could implement a special type of configuration variables that would emulate [dynamic scope from Lisp](https://www.gnu.org/software/emacs/emacs-paper.html#SEC17). The syntax could be taken over verbatim from the [contextvars stdlib module](https://docs.python.org/3/library/contextvars.html). In fact, contextvars could (and probably should) be used internally to make the configuration vars async-proof.
```
token = kwant.plotter.show.set(False)
# code that is executed with show set to False.
kwant.plotter.dpi.reset(token)
```
However, that's a bit on the heavy side. There could be also, additionally, a lighter context-manager-based syntax:
```
with kwant.plotter.show.set(False):
# code that is executed with show set to False.
```
(The stdlib context vars lack the context-manager syntax, but only because it was initially not deemed necessary for this low-level feature: https://mail.python.org/archives/list/python-ideas@python.org/thread/E4BB2WFMSC7D2GVOT4AFW3EBDY57HNQA.)
We could start slowly only using this for the likes of `show`, but I believe that the above could be actually useful for most optional arguments to functions. The configuration variables could be even under the scope of the function that they belong to, e.g.:
```
with plotter.streamplot.cmap.set(something):
plotter.current(...)
```
Configuration options could be also made to inherit from each other when useful. For example, `kwant.plotter.streamplot.cmap` could inherit from `kwant.plotter.cmap`. This would allow to easily use one colormap for all streamplots, but another one for all other plots.
While this may seem a little bit technical and complicated, the usage is very simple, and this change would be fully backwards compatible.
---
This may deserve to be turned into a separate module on PyPI, but it's also small enough to be included into Kwant at least initially.