2D: fix the scaling in the plotting of the polygons

92084364 · Bas Nijholt · c77b3d87 · 92084364
Commit 92084364 authored 7 years ago by Bas Nijholt
--- a/learner.ipynb
+++ b/learner.ipynb
@@ -206,13 +206,13 @@
    "%%opts Contours (alpha=0.3)\n",
    "from adaptive.learner import *\n",
    "\n",
-    "def plot(self):\n",
-    "    tri = self.ip().tri\n",
-    "    return hv.Contours([p for p in tri.points[tri.vertices]])\n",
+    "def plot(learner):\n",
+    "    tri = learner.ip().tri\n",
+    "    return hv.Contours([p for p in learner.unscale(tri.points[tri.vertices])])\n",
    "\n",
-    "def plot_poly(self):\n",
-    "    tri = self.ip().tri\n",
-    "    return hv.Polygons([p for p in tri.points[tri.vertices]])\n",
+    "def plot_poly(learner):\n",
+    "    tri = learner.ip().tri\n",
+    "    return hv.Polygons([p for p in learner.unscale(tri.points[tri.vertices])])\n",
    "\n",
    "\n",
    "(adaptive.live_plot(runner) +\n",
@@ -614,7 +614,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.2"
+   "version": "3.6.3"
  }
 },
 "nbformat": 4,

 %% Cell type:markdown id: tags:

 # Adaptive

 %% Cell type:markdown id: tags:

 [`adaptive`](https://gitlab.kwant-project.org/qt/adaptive-evaluation) is a package for adaptively sampling functions with support for parallel evaluation.

 This is an introductory notebook that shows some basic use cases.

 `adaptive` needs the following packages:

 + Python 3.6
 + holowiews
 + bokeh

 %% Cell type:code id: tags:

 ``` python
 import adaptive
 adaptive.notebook_extension()
 ```

 %% Cell type:markdown id: tags:

 # 1D function learner

 %% Cell type:markdown id: tags:

 We start with the most common use-case: sampling a 1D function $\ f: ℝ → ℝ$.

 We will use the following function, which is a smooth (linear) background with a sharp peak at a random location:

 %% Cell type:code id: tags:

 ``` python
 from functools import partial
 from random import random

 offset = random() - 0.5

 def f(x, offset=0, wait=True):
    from time import sleep
    from random import random

    a = 0.01
    if wait:
        sleep(random()*10)
    return x + a**2 / (a**2 + (x - offset)**2)
 ```

 %% Cell type:markdown id: tags:

 We start by initializing a 1D "learner", which will suggest points to evaluate, and adapt its suggestions as more and more points are evaluated.

 %% Cell type:code id: tags:

 ``` python
 learner = adaptive.learner.Learner1D(f, bounds=(-1.0, 1.0))
 ```

 %% Cell type:markdown id: tags:

 Next we create a "runner" that will request points from the learner and evaluate 'f' on them.

 By default the runner will evaluate the points in parallel using local processes ([`concurrent.futures.ProcessPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor)).

 %% Cell type:code id: tags:

 ``` python
 # The end condition is when the "loss" is less than 0.1. In the context of the
 # 1D learner this means that we will resolve features in 'func' with width 0.1 or wider.
 runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.01)
 ```

 %% Cell type:markdown id: tags:

 When instantiated in a Jupyter notebook the runner does its job in the background and does not block the IPython kernel.
 We can use this to create a plot that updates as new data arrives:

 %% Cell type:code id: tags:

 ``` python
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 We can now compare the adaptive sampling to a homogeneous sampling with the same number of points:

 %% Cell type:code id: tags:

 ``` python
 if not runner.task.done():
    raise RuntimeError('Wait for the runner to finish before executing the cells below!')
 ```

 %% Cell type:code id: tags:

 ``` python
 import numpy as np

 learner2 = adaptive.learner.Learner1D(f, bounds=(-1.01, 1.0))

 xs = np.linspace(-1.0, 1.0, len(learner.data))
 learner2.add_data(xs, map(partial(f, wait=False), xs))

 learner.plot() + learner2.plot()
 ```

 %% Cell type:markdown id: tags:

 # 2D function learner

 %% Cell type:markdown id: tags:

 Besides 1D functions, we can also learn 2D functions: $\ f: ℝ^2 → ℝ$

 %% Cell type:code id: tags:

 ``` python
 def func(xy):
    import numpy as np
    from time import sleep
    from random import random
    sleep(random())
    x, y = xy
    a = 0.2
    return x + np.exp(-(x**2 + y**2 - 0.75**2)**2/a**4)

 learner = adaptive.learner.Learner2D(func, bounds=[(-1, 1), (-1, 1)])
 ```

 %% Cell type:code id: tags:

 ``` python
 runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.01)
 ```

 %% Cell type:code id: tags:

 ``` python
 %%output size=100
 %%opts Contours (alpha=0.3)
 from adaptive.learner import *

-def plot(self):
-    tri = self.ip().tri
-    return hv.Contours([p for p in tri.points[tri.vertices]])
-
-def plot_poly(self):
-    tri = self.ip().tri
-    return hv.Polygons([p for p in tri.points[tri.vertices]])
+def plot(learner):
+    tri = learner.ip().tri
+    return hv.Contours([p for p in learner.unscale(tri.points[tri.vertices])])
+
+def plot_poly(learner):
+    tri = learner.ip().tri
+    return hv.Polygons([p for p in learner.unscale(tri.points[tri.vertices])])


 (adaptive.live_plot(runner) +
 adaptive.live_plot(runner, plotter=plot_poly) +
 adaptive.live_plot(runner) * adaptive.live_plot(runner, plotter=plot))
 ```

 %% Cell type:code id: tags:

 ``` python
 import numpy as np
 learner2 = adaptive.learner.Learner2D(func, bounds=[(-1, 1), (-1, 1)])
 lin = np.linspace(-1, 1, len(learner.points)**0.5)
 xy = [(x, y) for x in lin for y in lin]
 learner2.add_data(xy, map(func, xy))
 learner2.plot().relabel('Homogeneous grid') + learner.plot().relabel('With adaptive')
 ```

 %% Cell type:markdown id: tags:

 # Averaging learner

 %% Cell type:markdown id: tags:

 The next type of learner averages a function until the uncertainty in the average meets some condition.

 This is useful for sampling a random variable. The function passed to the learner must formally take a single parameter,
 which should be used like a "seed" for the (pseudo-) random variable (although in the current implementation the seed parameter can be ignored by the function).

 %% Cell type:code id: tags:

 ``` python
 def g(n):
    import random
    from time import sleep
    sleep(random.random() / 5)
    # Properly save and restore the RNG state
    state = random.getstate()
    random.seed(n)
    val = random.gauss(0.5, 1)
    random.setstate(state)
    return val
 ```

 %% Cell type:code id: tags:

 ``` python
 learner = adaptive.AverageLearner(g, None, 0.01)
 runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 1)
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 # Balancing learner

 %% Cell type:markdown id: tags:

 The balancing learner is a "meta-learner" that takes a list of multiple leaners. The runner wil find find out which points of which child learner will improve the loss the most and send those to the executor.

 The balancing learner can for example be used to implement a poor-man's 2D learner by using the `Learner1D`.

 %% Cell type:code id: tags:

 ``` python
 from adaptive.learner import Learner1D, BalancingLearner

 learners = [Learner1D(partial(f, offset=2*random()-1, wait=False), bounds=(-1.0, 1.0)) for i in range(10)]
 learner = BalancingLearner(learners)
 runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.02)
 ```

 %% Cell type:code id: tags:

 ``` python
 import holoviews as hv
 adaptive.live_plot(runner, plotter=lambda learner: hv.Overlay([L.plot() for L in learner.learners]))
 ```

 %% Cell type:markdown id: tags:

 ## Alternative executors

 %% Cell type:markdown id: tags:

 Often you will want to evaluate the function on some remote computing resources. `adaptive` works out of the box with any framework that implements a [PEP 3148](https://www.python.org/dev/peps/pep-3148/) compliant executor that returns `concurrent.futures.Future` objects.

 %% Cell type:markdown id: tags:

 ### `concurrent.futures`

 %% Cell type:markdown id: tags:

 By default a runner creates a `ProcessPoolExecutor`, but you can also pass one explicitly e.g. to limit the number of workers:

 %% Cell type:code id: tags:

 ``` python
 from concurrent.futures import ProcessPoolExecutor

 executor = ProcessPoolExecutor(max_workers=4)

 learner = adaptive.learner.Learner1D(f, bounds=(-1, 1))
 runner = adaptive.Runner(learner, executor=executor, goal=lambda l: l.loss() < 0.1)
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 ### IPyparallel

 %% Cell type:code id: tags:

 ``` python
 import ipyparallel

 client = ipyparallel.Client()
 # f is a closure, so we have to use cloudpickle -- this is independent of 'adaptive'
 client[:].use_cloudpickle()

 learner = adaptive.learner.Learner1D(f, bounds=(-1, 1))
 runner = adaptive.Runner(learner, executor=client, goal=lambda l: l.loss() < 0.1)
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 ---

 %% Cell type:markdown id: tags:

 # Advanced Topics

 %% Cell type:markdown id: tags:

 ## Cancelling a runner

 %% Cell type:markdown id: tags:

 Sometimes you want to interactively explore a parameter space, and want the function to be evaluated at finer and finer resolution and manually control when the calculation stops.

 If no `goal` is provided to a runner then the runner will run until cancelled:

 %% Cell type:code id: tags:

 ``` python
 learner = adaptive.learner.Learner1D(f, bounds=(-1.0, 1.0))
 runner = adaptive.Runner(learner)
 adaptive.live_plot(runner)
 ```

 %% Cell type:code id: tags:

 ``` python
 runner.task.cancel()
 ```

 %% Cell type:markdown id: tags:

 ## Debugging Problems

 %% Cell type:markdown id: tags:

 Runners work in the background with respect to the IPython kernel, which makes it convenient, but also means that inspecting errors is more difficult because exceptions will not be raised directly in the notebook. Often the only indication you will have that something has gone wrong is that nothing will be happening.

 Let's look at the following example, where the function to be learned will raise an exception 10% of the time.

 %% Cell type:code id: tags:

 ``` python
 def will_raise(x):
    from random import random
    from time import sleep

    sleep(random())
    if random() < 0.1:
        raise RuntimeError('something went wrong!')
    return x**2

 learner = adaptive.Learner1D(will_raise, (-1, 1))
 runner = adaptive.Runner(learner)  # without 'goal' the runner will run forever unless cancelled
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 The above runner should continue forever, but we notice that it stops after a few points are evaluated.

 First we should check that the runner has really finished:

 %% Cell type:code id: tags:

 ``` python
 runner.task.done()
 ```

 %% Cell type:markdown id: tags:

 If it has indeed finished then we should check the `result` of the runner. This should be `None` if the runner stopped successfully. If the runner stopped due to an exception then asking for the result will raise the exception with the stack trace:

 %% Cell type:code id: tags:

 ``` python
 runner.task.result()
 ```

 %% Cell type:markdown id: tags:

 ### Logging runners

 %% Cell type:markdown id: tags:

 Runners do their job in the background, which makes introspection quite cumbersome. One way to inspect runners is to instantiate one with `log=True`:

 %% Cell type:code id: tags:

 ``` python
 learner = adaptive.learner.Learner1D(f, bounds=(-1, 1))
 runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.1,
                         log=True)
 adaptive.live_plot(runner)
 ```

 %% Cell type:markdown id: tags:

 This gives a the runner a `log` attribute, which is a list of the `learner` methods that were called, as well as their arguments. This is useful because executors typically execute their tasks in a non-deterministic order.

 This can be used with `adaptive.runner.replay_log` to perfom the same set of operations on another runner:

 %% Cell type:code id: tags:

 ``` python
 reconstructed_learner = adaptive.learner.Learner1D(f, bounds=(-1, 1))
 adaptive.runner.replay_log(reconstructed_learner, runner.log)
 ```

 %% Cell type:code id: tags:

 ``` python
 learner.plot().opts(style=dict(size=6)) * reconstructed_learner.plot()
 ```

 %% Cell type:markdown id: tags:

 ## Using Runners from a script

 %% Cell type:markdown id: tags:

 Runners can also be used from a Python script independently of the notebook:

 ```python
 import adaptive

 def f(x):
    return x

 learner = adaptive.Learner1D(f, (-1, 1))

 runner = adaptive.Runner(learner, goal=lambda: l: l.loss() < 0.1)
 runner.run_sync()  # Block until completion.
 ```

 Under the hood the runner uses [`asyncio`](https://docs.python.org/3/library/asyncio.html). You don't need to worry about this most of the time, unless your script uses asyncio itself. If this is the case you should be aware that instantiating a `Runner` schedules a new task on the current event loop, and you can simply

 ```python
    await runner.task
 ```

 inside a coroutine to await completion of the runner.