LearnerND scale output values before computing loss

import adaptive
from adaptive import LearnerND
import numpy as np
from adaptive.tests.test_learners import *

def f(xyz, d=0.4850418018218789):
    a = 0.2
    x, y, z = xyz
    return x + math.exp(-(x**2 + y**2 + z**2 - d**2)**2 / a**4) + z**2

learner_kwargs = dict(bounds=((-1, 1), (-1, 1), (-1, 1)),
                      loss_per_simplex=adaptive.learner.learnerND.uniform_loss)

control_kwargs = dict(learner_kwargs)
control = LearnerND(f, **control_kwargs)

xscale, yscale = 212.71376207339554, 905.2031939573303

l_kwargs = dict(learner_kwargs)
l_kwargs['bounds'] = xscale * np.array(l_kwargs['bounds'])
learner = LearnerND(lambda x: yscale * f(np.array(x) / xscale),
                       **l_kwargs)

learner._recompute_losses_factor = 1
control._recompute_losses_factor = 1

npoints = 201

for n in range(npoints):
    cxs, _ = control.ask(1)
    xs, _ = learner.ask(1)
    control.tell_many(cxs, [control.function(x) for x in cxs])
    learner.tell_many(xs, [learner.function(x) for x in xs])

    # Check whether the points returned are the same
    xs_unscaled = np.array(xs) / xscale
    assert np.allclose(xs_unscaled, cxs)

Actually, the problem occurs earlier, at point 57. Check with

for n in range(npoints):
    ...
    assert control._simplex_queue == learner._simplex_queue  # add this line

I'm not entirely sure if it's possible to fully mitigate all errors. Because if we have a simplex with some loss and then divide this equally among 2 children with equal area, obviously you expect the loss to be equal, yet by numerical precision it may have some error.

Now if you happen to have the following loss rounded to say 4 decimal places (4 because it's easy)

0.1111 then dividing it in two will yield 0.05555 for each of the two children. Now we apply some numerical error to it: loss1 = 0.055550000001 and loss2 = 0.0555499999999 or something like that. Now we will round the thing to four decimal places again:

loss1_rounded = 0.0556 and loss2_rounded = 0.0555. The same applies if we round to more digits. This is even possible if we round the loss to 12 digits and the subdivided loss to 8 digits. No matter how we round there is allways the possibility that we may end up slightly below the rounding cutoff or slightly above it, resulting in a different number.

ok. Seems we can't fix this in general then.... Should probably keep the test as xfailing, and merge?

Yeah I would suggest to do so. However maybe we should insert some test that is not randomized, that does succeed now, and as such we will be notified whenever something that works now gets broken.

Because if we just x-fail, we will never notice

And also possibly open an issue, maybe it's something we want to solve, maybe it is not. We will have to see in the future I suppose

added 5 commits

e0841808...fa4696ed - 2 commits from branch master
e7ad124b - scale the losses when the scale increases with a factor 1.1
a995d832 - round the loss to a certain precision
1a002b1b - xfail(LearnerND) for test_learner_performance_is_invariant_under_scaling

Compare with previous version

enabled an automatic merge when the pipeline for 1a002b1b succeeds

mentioned in commit a90d2579

merged

LearnerND scale output values before computing loss

Merged by Bas Nijholt 6 years ago (Dec 7, 2018 4:13pm UTC) 6 years ago

Activity

LearnerND scale output values before computing loss

Merge request reports

Merged by Bas Nijholt 6 years ago (Dec 7, 2018 4:13pm UTC) 6 years ago

Activity