A simple fix for h_inv overflow with large dimensions about crfmnes HOT 4 CLOSED

I've run some quick trials tracking the value of h_ind during the while loop in get_h_inv. Note that I am using the early exiting check suggested above to approximate convergence and avoid extreme numbers of iterations.

The legend shows pairs of (initial value of h_ind, dim):

For very small dim values it may converge faster when h_ind starts at 1 rather than 5, but this quickly changes as dim grows.
Any run where an OverflowError occurs is omitted, which includes dim >= ~2048 when h_ind starts at 1. For comparison you can see how consistent the runs are when h_ind starts at 5 (dashed lines), allowing much higher values for dim.

Here you can see how f(h_inv, dim) explodes, then re-converges over time when h_inv starts at 1 (f_prime(h_inv) has similar behavior). Note the y-axis is changed to log-scale:

from crfmnes.

Fixed. Thank @Obliman again!
If you encounter any problems again, I would be very happy if you could report it with an issue or create a PR.

Best,

from crfmnes.

Also, for some "dim" values abs(f(h_inv, dim)) will never reach the 1e-10 target (ex: dim = 800000 -> ~2.5e-9). I suggest either an iteration limit resulting in an error, or track the last value of h_inv and exit early when it stops changing.

# Inside the while loop of get_h_inv:
last = h_inv
h_inv = ...
if abs(h_inv - last) < 1e-16:
    # Exit early since no further improvements are happening
    break

from crfmnes.

@Obliman
Thank you for reporting the important problem!
In fact, we have only conducted experiments up to 2000 dimensions, and have not investigated the behavior in higher dimensions.
How many dimensions do you know that there is a problem with behavior?
We may need more investigations to make this improvement.

from crfmnes.