(BalancingLearner) make new balancinglearner that looks at the total loss rather than loss improvement
Suggestion is to implement a new balancing-learner that is a drop-in replacement for the old one, which doesn't requests points from the learners unless they are actually going to be used. So we would look at the total loss of each learner rather than the loss-improvement.
- This would probably be faster as we do not need to suggest a new point in order to sort the learners. The actual difference should be tested
- We do not require the learners to support
ask(..., add_data=False)
- This would very likely also solve #87
- This would also be nice for the ND-learner
- Probably in practice, the outcome would be the similar to the normal balancinglearner
- The logic should be simpler (as in less code to maintain) and we do not need to keep track of the suggested points
- We should analyse in what cases the current implementation behaves differently from the proposed implementation and if it is an improvement or worse.
Suggestion is to test it out, if it works nicely we will adopt