add runner.max_retries

This PR (mainly) introduces runner.max_retries and runner.raise_if_max_retries.

With this the following function would be "learned":

import adaptive
adaptive.notebook_extension()

def f(x, offset=0):
    from random import random
    a = 0.01
    if random() < 0.9:
        raise Exception('Oops, this failed.')
    return x + a**2 / (a**2 + (x - offset)**2)

learner = adaptive.Learner1D(f, bounds=(-1, 1))
runner = adaptive.BlockingRunner(learner, goal=lambda l: l.loss() < 0.05,
                         max_retries=20, log=True, raise_if_max_retries=False)

I also introduce BlockingRunner.overhead and the corresponding timing functions and put the shared code of BlockingRunner and AsyncRunner in BaseLearner methods.

changed the description

added 1 commit

• 0947266a - return 'loss_improvements' with the BalancingLearner

Compare with previous version

added 1 commit

• ffdc5cf0 - add 'runner.latest_traceback'

Compare with previous version

added 14 commits

• ffdc5cf0...95529480 - 6 commits from branch master
• c00c5af3 - add runner.max_retries and runner.raise_if_max_retries
• d66b4b4f - make 'xs' into an attribute 'xs_pending'
• 0420fd7c - add 'runner.overhead' to the BlockingRunner too
• c98f27e2 - make 'AsyncRunner._run' and 'BlockingRunner._run' more similar
• bccccb4b - simplify '_run' for both runners
• fff38172 - make 'do_log' into a property such that logging can be turned on
• 5ee3d08f - return 'loss_improvements' with the BalancingLearner
• 136c0cc1 - add 'runner.latest_traceback'

Compare with previous version

added 1 commit

• 23b50d40 - simplify 'runner._ask'

Compare with previous version

Should we have multiple error-handling strategies? Some errors may be reproducible, and e.g. cquad can deal with those efficiently.

added 4 commits

• eb15456e - homogenize doc-strings and improve 'runner.elapsed_time'
• 027abfe4 - rename 'elapsed_function_time' -> '_elapsed_function_time'
• fa01a033 - make some methods and attributes hidden
• ee432008 - rename 'runner.latest_traceback' -> 'runner.last_traceback'

Compare with previous version

added 1 commit

• 8f6296d5 - fix comment

Compare with previous version

added 3 commits

• 81a98b18 - rename 'done' -> 'done_futs'
• 52f6e9eb - remove unused import
• 123085aa - rename 'runner.xs_pending' -> 'runner.pending_points'

Compare with previous version

added 1 commit

• f0f205c2 - be consistent in the doc-strings

Compare with previous version

added 9 commits

• a141829b - homogenize doc-strings and improve 'runner.elapsed_time'
• 2eade7e3 - rename 'elapsed_function_time' -> '_elapsed_function_time'
• 83c55760 - make some methods and attributes hidden
• 4ca9a6bd - rename 'runner.latest_traceback' -> 'runner.last_traceback'
• c7434dcd - fix comment
• fcd0aa69 - rename 'done' -> 'done_futs'
• 15b435af - remove unused import
• 57af7620 - rename 'runner.xs_pending' -> 'runner.pending_points'
• 19435772 - be consistent in the doc-strings

Compare with previous version

added 1 commit

• 02b3074f - add doc-strings to 'AsyncRunner' and 'BlockingRunner'

Compare with previous version

added 1 commit

• 673e3024 - add doc-string to 'runner.overhead'

Compare with previous version

added 1 commit

• 96e18ab5 - add doc-string to 'runner.overhead'

Compare with previous version

assigned to @jbweston

@basnijholt can you squash down the commits a bit so that there is 1 commit per bit of functionality?

There's quite a bit of stuff here to review at once.

assigned to @basnijholt