(LearnerND) make default point choosing better
Closes #76 (closed), #77 (closed)
This is a first attempt at making the point choosing better, this is done by a modified loss and choose_point_in_simplex method.
The new loss function is the hypervolume of the simplex in the N+M space. eg, if we have a function
The choose-point-in-simplex works by taking the center of the simplex, unless the shape of the simplex is sufficiently bad, then it will take the center of the longest edge. It does this by checking whether the circumcenter lies within the simplex. In 2D this means that the largest angle has to be less than 90 degrees, (in higher dimensions, well something similar I imagine).
Merge request reports
Activity
- Resolved by Joseph Weston
The new loss function is the hypervolume of the simplex in the N+M space
...
In essence this is the extension of the loss of the 1D-learner into ND-space
Actually for the 1D learner we essentially treat the different components of the output vector as independent functions and the loss of an interval is the max of the losses for the individual components. e.g. for learning
f: ℝ → ℝ^2the loss ismax(hypot(dx, dy[0]), hypot(dx, dy[1]))The idea of instead taking the volume of a higher-dim simplex is intriguing, but I'm not sure how to visualize this, and what it means...
- Last reply by Joseph Weston
so I created a pdf so you could compare the Learner2D vs LearnerND(master branch) vs LearnerND(this MR). In the pdf
!86refers to this merge requestso here is the pdf compare_learners.pdf
The advantage of the Learner2D is that it still uses less points to map the circle, so evaluation would be faster.
The disadvantage of the Learner2D being that you could easily insert a peak around (0,0) with radius ~0.1 inside the circle and it would not be noticed, while the LearnerND with the current loss would find it.
So this does seem like an matter of focus: either more focussing on steep regions to have more detail there or exploring to find other potential regions of interest.
-
@Jorn that's really nice
I also notice that with the loss implemented here the points are much "tighter" to the ring than with the default loss of the Learner2D, which is good.
The advantage of the Learner2D is that it still uses less points to map the circle, so evaluation would be faster.
I can't see how the PDF supports this claim. There isn't a single panel where Learner2D has mapped the whole circle and LearnerND still has more to map. They seem pretty equal to me. LearnerND puts more points in the "flat" reagions, but the points in the steep regions are tighter to the ring.
- Resolved by Joseph Weston
So this does seem like an matter of focus: either more focussing on steep regions to have more detail there or exploring to find other potential regions of interest.
I think that what has been implemented in LearnerND strikes this balance quite well.
If it's not too much work could you have a go at implementing a loss that prioritizes detailing steep regions over exploring flat ones? It might be as simple as just scaling the
ycoordinates... - Last reply by Joseph Weston
added 18 commits
-
17369ae2...926a48b3 - 12 commits from branch
master - 4d03a752 - make loss function based on simplex volume in higher dimensional space
- d9a45e1b - make point_in_simplex and circumcircle global methods
- 5f1eb3b3 - change choose_point_in_simplex such that it sometimes takes the center
- b90a25d3 - pep8
- 9ae74260 - update docstring
- ba9c6d7c - Merge branch '76-77-improve-point-selection' of…
Toggle commit list-
17369ae2...926a48b3 - 12 commits from branch
added 1 commit
- d1c51924 - move simplex_volume function to triangulation file
added 2 commits
