README.md 6.44 KB
Newer Older
1
# A collection of tutorials and potentially reusable code snippets.
Bas Nijholt's avatar
Bas Nijholt committed
2 3 4

Add links to the code with short descriptions below.

Bas Nijholt's avatar
Bas Nijholt committed
5
## [Set-up automatic syncing between your computer and `io`](syncthing.md)
Bas Nijholt's avatar
Bas Nijholt committed
6 7


Bas Nijholt's avatar
Bas Nijholt committed
8
## [Set-up ssh to `io`](ssh-setup.md)
Bas Nijholt's avatar
Bas Nijholt committed
9

10

Bas Nijholt's avatar
Bas Nijholt committed
11
## `ipynb`s with git: automatically remove output
Bas Nijholt's avatar
Bas Nijholt committed
12 13
Works on macOS and Linux, copy paste this whole command into a terminal (at once):
```
Bas Nijholt's avatar
Bas Nijholt committed
14 15 16
mkdir -p ~/.config && mkdir -p ~/.config/git
echo '*.ipynb filter=nbstripout' > ~/.config/git/attributes
cat > ~/.config/git/config << EOF
Bas Nijholt's avatar
Bas Nijholt committed
17
[filter "nbstripout"]
18
    clean = jupyter-nbconvert --stdin --ClearOutputPreprocessor.enabled=True --to notebook --stdout
Bas Nijholt's avatar
Bas Nijholt committed
19 20
    smudge = cat
    required = true
Bas Nijholt's avatar
Bas Nijholt committed
21
EOF
Bas Nijholt's avatar
Bas Nijholt committed
22
```
23

24
## [Handle data in your git repo with submodules](data-git-submodule.md)
25 26


Bas Nijholt's avatar
Bas Nijholt committed
27 28
## [Git Snippets](git.md)
Useful Git commands that serve as a summary of the excellent [Git Book](https://git-scm.com/book/en/v2/)
29

Bas Nijholt's avatar
Bas Nijholt committed
30 31
## [ArXiv feed watcher](arxivfv2.py)
Keep track of new papers on the arXiv without having to look at all the new abstracts every day. It fetches the RSS feed, runs a simple search for keywords in the title/abstract and authors and sends an email with the abstracts that match any of the criteria. It uses `feedparser`, which is on `io`. I run it with cron at 03:00 Monday to Friday. To set it up you need to fill out the search criteria and the SMTP email options so it can send an email from user1@server1.com to user2@server2.com.
32

Bas Nijholt's avatar
Bas Nijholt committed
33 34
## [Sparse diagonalization with MUMPS](mumps_sparse_diag.py)
Example on using `mumps` with `scipy.sparse.linalg.eigsh`.
35

36
## [Vagrant](vagrant/readme.md)
Bas Nijholt's avatar
Bas Nijholt committed
37 38
Vagrant is simple wrapper around VirtualBox that allows to easily setup configured
virtual machines. See the basic usage commands and a few examples.
39 40 41

[Centos](vagrant/examples/centos/Vagrantfile) could be probably very useful for admins
to see how things work on Centos7 system in an easy way.
Bas Nijholt's avatar
Bas Nijholt committed
42

Bas Nijholt's avatar
Bas Nijholt committed
43 44
## [Set-up your connection to hpc05 and explain usage](https://gitlab.kwant-project.org/qt/cookbook/tree/master/hpc05)

Bas Nijholt's avatar
Bas Nijholt committed
45
## [hpc05 usage checker](hpc05-usage.py)
Bas Nijholt's avatar
Bas Nijholt committed
46
See how many cores are currently in total and per user. Add an `alias` to your `~/.bash_profile` for easy usage:
Bas Nijholt's avatar
Bas Nijholt committed
47
```
Bas Nijholt's avatar
Bas Nijholt committed
48
alias stat="python /home/$USER/full_path_to_script"
Bas Nijholt's avatar
Bas Nijholt committed
49
```
50
<img width="274" alt="screen shot 2016-10-12 at 13 02 19" src="https://cloud.githubusercontent.com/assets/6897215/19307824/e4980ba0-907c-11e6-88e4-dca1948d957b.png">
Bas Nijholt's avatar
Bas Nijholt committed
51 52 53 54

## [colormap normalizers](cmap-normalizers.py)
Nomalize your colorscale by pinning the middle of the colormap with `MidpointNormalize` or by binning the data such in different parts of the colormap such that every range of data points is represented in the plot with `HistogramNormalize`.

Bas Nijholt's avatar
Bas Nijholt committed
55 56 57
## Spyview
Did an experimentalist sent you a `.mtx` Spyview file and you don't run Windows? Do not despair:
```
Bas Nijholt's avatar
Bas Nijholt committed
58 59 60 61 62 63
def open_mtx(fname):
    with open(fname, 'rb') as f:
        line1 = f.readline()
        *shape, entry_size = [int(i) for i in  f.readline().split()]
        data = np.fromfile(f, (np.float64 if entry_size == 8 else np.float32))
        data = data.reshape(shape).squeeze()
64 65

    keys = ['units', 'plot_settings', 'xname', 'xmin', 'xmax',
Bas Nijholt's avatar
Bas Nijholt committed
66 67 68 69
            'yname', 'ymin', 'ymax', 'zname', 'zmin', 'zmax']
    values = line1.decode("utf-8").replace('\n', '').split(',')
    meta_data = dict(zip(keys, values))
    return meta_data, data
70
```
71

Bas Nijholt's avatar
Bas Nijholt committed
72
## [What is axis orientation when doing 2D plots](http://nbviewer.jupyter.org/urls/gitlab.kwant-project.org/qt/cookbook/raw/master/axis_2d.ipynb)
73 74 75


## [Insallation of kwant from source into conda env](kwant-from-source.md)
76

77
## [WIP: xArray + HoloViews: Finding and ultimte way for gridded data](xarray_holoviews_gridded.ipynb)
78 79 80 81 82 83 84

## [LCAO Hamiltonian generator](lcao.py)

Tight-binding Hamiltonians are often given in the LCAO (linear combination of atomic orbitals) formalism.
You may have encountered large tables of parameters containing entries like $V_{pp\pi}$.
This utility computes the corresponding hopping matrices automatically, for example `lcao_term(1, 1, 1, [0, 0, 1])` will return $H_{pp\pi}$ for a bond in the `[0, 0, 1]` direction.
See docstring in [lcao.py](lcao.py) for more details.
85

Artem Pulkin's avatar
Artem Pulkin committed
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108
## [Guide for uploading Gitlab CI artifacts to Dropbox](artifacts_and_dropbox.md)

## `owncloud` / `nextcloud` in your CLI

You may use it, for example, to perform real-time visualisations of your running
hpc jobs. You will be able to track the progress without the need to vpn/login
to the cluster. This example is for [surfdrive](https://surfdrive.surf.nl).

First, create a new `webdav` password in your
[settings page](https://surfdrive.surf.nl/files/index.php/settings/personal?sectionid=security).

Then, upload any file with `curl -u "user-id@tudelft.nl:webdav-pass" -X PUT "https://surfdrive.surf.nl/files/remote.php/nonshib-webdav/path/to/folder/$1" --data-binary @"$1"`.
You have to replace `user-id`, `webdav-pass`, `$1` with your user name, the
generated password and the file name, respectively.

- The file will appear as an ordinary file in your cloud. You may, for example,
share it via link: new uploads under the same name will keep sharing settings.
- Uploaded images can be displayed directly in the browser (and shared with your
colleagues). Sometimes, browsers prefer to not download updated image previews:
this might depend on OC/NC server settings.
- Make sure to keep your `webdav` password in secret. If you prepare a script
please make sure to set `chmod 700 your-script` as a minimal measure.
- similarly, you may download and remove files, create folders in your cloud by
109 110 111 112 113 114
replacing `PUT` with a proper `http` directive.

## Voice notifications in executable cells

If you’re running long simulations and want some sound notification when they’ve finished, [someone worked out how to do it](https://gist.github.com/parente/41a13f4c8fa6165d345cc7703be291a3). All you need is one handy function:
```
115 116 117 118
def speak(text, voice=0):
    '''
    You can select a voice by putting in an integer. If you overshoot the length of the voice list, the default voice (0) is used.
    '''
119 120 121
    from IPython.display import Javascript as js, clear_output
    # Escape single quotes
    text = text.replace("'", r"\'")
122 123 124
    display(
        js(
            f"""
125 126
    if(window.speechSynthesis) {{
        var synth = window.speechSynthesis;
127 128 129 130
        var voices = synth.getVoices();
        var utterThis = new window.SpeechSynthesisUtterance('{text}');
        utterThis.voice = voices[{voice}];
        synth.speak(utterThis);
131
    }}
132 133 134
    """
        )
    )
135 136 137 138 139
    # Clear the JS so that the notebook doesn't speak again when reopened/refreshed
    clear_output(False)

```