run profiler
Passed Started
by
@Kostas
Kostas Vilkelis
1Running with gitlab-runner 15.4.0 (43b2dc3d)2 on universal-tn4 d3cf99854Using Docker executor with image gitlab.kwant-project.org:5005/qt/research-docker ...5Authenticating with credentials from job payload (GitLab Registry)6Pulling docker image gitlab.kwant-project.org:5005/qt/research-docker ...7Using docker image sha256:fe97fe7a9f41d572be990afbad4d2f2c0bfb33344f892f2d812a915b9a3814d5 for gitlab.kwant-project.org:5005/qt/research-docker with digest gitlab.kwant-project.org:5005/qt/research-docker@sha256:7f1cd32463b6a333d77cea77aaef1b1edf62a9126dde067838e17ef03fe43e2a ...9Running on runner-d3cf9985-project-3642-concurrent-1 via tnw-tn4.tudelft.net...11Fetching changes with git depth set to 20...12Reinitialized existing Git repository in /builds/qt/kwant-scf/.git/13Checking out 92fc4b63 as release_version...14Removing memoryProfile.bin15Removing pymf/__pycache__/16Removing pymf/kwant_helper/__pycache__/17Removing pymf/tb/__pycache__/18Removing timeProfile.html19Skipping Git submodules setup21Using docker image sha256:fe97fe7a9f41d572be990afbad4d2f2c0bfb33344f892f2d812a915b9a3814d5 for gitlab.kwant-project.org:5005/qt/research-docker with digest gitlab.kwant-project.org:5005/qt/research-docker@sha256:7f1cd32463b6a333d77cea77aaef1b1edf62a9126dde067838e17ef03fe43e2a ...22$ export PYTHONPATH="."23$ pip install memray pyinstrument24WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.25Collecting memray26 Downloading memray-1.12.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)27Collecting pyinstrument28 Downloading pyinstrument-4.6.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)29Requirement already satisfied: jinja2>=2.9 in /opt/conda/lib/python3.12/site-packages (from memray) (3.1.2)30Requirement already satisfied: rich>=11.2.0 in /opt/conda/lib/python3.12/site-packages (from memray) (13.7.1)31Collecting textual>=0.41.0 (from memray)32 Downloading textual-0.58.1-py3-none-any.whl.metadata (5.6 kB)33Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.12/site-packages (from jinja2>=2.9->memray) (2.1.5)34Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/lib/python3.12/site-packages (from rich>=11.2.0->memray) (3.0.0)35Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/lib/python3.12/site-packages (from rich>=11.2.0->memray) (2.16.1)36Requirement already satisfied: typing-extensions<5.0.0,>=4.4.0 in /opt/conda/lib/python3.12/site-packages (from textual>=0.41.0->memray) (4.8.0)37Requirement already satisfied: mdurl~=0.1 in /opt/conda/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich>=11.2.0->memray) (0.1.2)38Collecting linkify-it-py<3,>=1 (from markdown-it-py[linkify,plugins]>=2.1.0->textual>=0.41.0->memray)39 Downloading linkify_it_py-2.0.3-py3-none-any.whl.metadata (8.5 kB)40Requirement already satisfied: mdit-py-plugins in /opt/conda/lib/python3.12/site-packages (from markdown-it-py[linkify,plugins]>=2.1.0->textual>=0.41.0->memray) (0.4.0)41Collecting uc-micro-py (from linkify-it-py<3,>=1->markdown-it-py[linkify,plugins]>=2.1.0->textual>=0.41.0->memray)42 Downloading uc_micro_py-1.0.3-py3-none-any.whl.metadata (2.0 kB)43Downloading memray-1.12.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)44 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 90.4 MB/s eta 0:00:0045Downloading pyinstrument-4.6.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (106 kB)46 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.5/106.5 kB 192.7 MB/s eta 0:00:0047Downloading textual-0.58.1-py3-none-any.whl (549 kB)48 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 549.8/549.8 kB 159.2 MB/s eta 0:00:0049Downloading linkify_it_py-2.0.3-py3-none-any.whl (19 kB)50Downloading uc_micro_py-1.0.3-py3-none-any.whl (6.2 kB)51Installing collected packages: uc-micro-py, pyinstrument, linkify-it-py, textual, memray52Successfully installed linkify-it-py-2.0.3 memray-1.12.0 pyinstrument-4.6.2 textual-0.58.1 uc-micro-py-1.0.353$ python profiling/graphene.py54Single SCF loop takes 3.320030429586768 whereas a single diagonalization of a corresponding system takes 1.760068050399422655$ memray stats memoryProfile.bin56📏 Total allocations:57 36858558📦 Total memory allocated:59 1.273GB60📊 Histogram of allocation size:61 min: 8.000B62 ---------------------------------------------63 < 47.000B : 358822 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇64 < 285.000B : 29 ▇65 < 1.663KB : 21 ▇66 < 9.930KB : 19 ▇67 < 59.292KB : 9604 ▇68 < 354.032KB: 8 ▇69 < 2.064MB : 58 ▇70 < 12.326MB : 14 ▇71 < 73.599MB : 4 ▇72 <=439.453MB: 6 ▇73 ---------------------------------------------74 max: 439.453MB75📂 Allocator type distribution:76 MALLOC: 36852877 MMAP: 5578 REALLOC: 279🥇 Top 5 largest allocating locations (by size):80 - tb_to_khamvector:/builds/qt/kwant-scf/pymf/tb/transforms.py:40 -> 439.704MB81 - c2cn:/opt/conda/lib/python3.12/site-packages/scipy/fft/_pocketfft/basic.py:149 -> 264.305MB82 - density_matrix_kgrid:/builds/qt/kwant-scf/pymf/mf.py:33 -> 175.782MB83 - ifftn_to_tb:/builds/qt/kwant-scf/pymf/tb/transforms.py:60 -> 105.833MB84 - eigh:/opt/conda/lib/python3.12/site-packages/numpy/linalg/linalg.py:1487 -> 98.879MB85🥇 Top 5 largest allocating locations (by number of allocations):86 - ifftn_to_tb:/builds/qt/kwant-scf/pymf/tb/transforms.py:60 -> 35886587 - c2cn:/opt/conda/lib/python3.12/site-packages/scipy/fft/_pocketfft/basic.py:149 -> 962788 - eigh:/opt/conda/lib/python3.12/site-packages/numpy/linalg/linalg.py:1487 -> 1189 - tb_to_khamvector:/builds/qt/kwant-scf/pymf/tb/transforms.py:34 -> 690 - add_tb:/builds/qt/kwant-scf/pymf/tb/tb.py:19 -> 691$ memray summary memoryProfile.bin92┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┓93┃ ┃ ┃ Total ┃ ┃ Own ┃ ┃94┃ ┃ <Total ┃ Memo… ┃ Own ┃ Memo… ┃ Allo… ┃95┃ Location ┃ Memor… ┃ % ┃ Memo… ┃ % ┃ Count ┃96┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━┩97│ density_matrix at │ 554.9… │ 97.4… │ 0.00… │ 0.00% │ 15 │98│ /builds/qt/kwant-scf/pymf/mf.py │ │ │ │ │ │99│ mfield at │ 554.9… │ 97.4… │ 0.00… │ 0.00% │ 15 │100│ /builds/qt/kwant-scf/pymf/model.py │ │ │ │ │ │101│ scf_loop at │ 554.9… │ 97.4… │ 0.00… │ 0.00% │ 15 │102│ /builds/qt/kwant-scf/profiling/gra… │ │ │ │ │ │103│ <module> at │ 554.9… │ 97.4… │ 0.00… │ 0.00% │ 15 │104│ /builds/qt/kwant-scf/profiling/gra… │ │ │ │ │ │105│ _wrapreduction at │ 88.01… │ 15.4… │ 88.0… │ 15.4… │ 4 │106│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │107│ sum at │ 88.01… │ 15.4… │ 0.00… │ 0.00% │ 4 │108│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │109│ tb_to_khamvector at │ 27.46… │ 4.82% │ 27.4… │ 4.82% │ 2 │110│ /builds/qt/kwant-scf/pymf/tb/trans… │ │ │ │ │ │111│ meshgrid at │ 96.00… │ 0.00% │ 32.0… │ 0.00% │ 3 │112│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │113│ _broadcast_to at │ 64.00… │ 0.00% │ 64.0… │ 0.00% │ 2 │114│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │115│ broadcast_arrays at │ 64.00… │ 0.00% │ 0.00… │ 0.00% │ 2 │116│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │117│ tensordot at │ 32.00… │ 0.00% │ 32.0… │ 0.00% │ 1 │118│ /opt/conda/lib/python3.12/site-pac… │ │ │ │ │ │119└─────────────────────────────────────┴────────┴───────┴───────┴───────┴───────┘121Uploading artifacts...122timeProfile.html: found 1 matching files and directories 123memoryProfile.bin: found 1 matching files and directories 124Uploading artifacts as "archive" to coordinator... 201 Created id=420900 responseStatus=201 Created token=glcbt-64126Job succeeded