Figure out how to add custom folders to PYTHONPATH
I am not a big fan of PYTHONPATH tricks, but frequently it is needed to emulate Jupyter notebook behaviour, that treats .py files in the notebook directory as modules. In the case of Dask Jobqueue, I have written a tutorial for that, which utilized env_extra
argument for SLURMCluster
:
dask_jobqueue_demo.ipynb
In the case of Dask Gateway (and dask-quantumtinkerer
) it is not yet clear how to do it.
Clarification
If you have file funcs.py
in the folder with your IPython notebook, probably you want it to be accessible by your Dask workers. Their working directory is probably $HOME
, so they won't see the source folder, for example, in ~/Sync
. One needs to figure out if this is true and how to fix it, if it is true.
Edited by Viacheslav Ostroukh