Serialization about pynndescent HOT 5 OPEN

This seems to be related to how Numba handles things. Hopefully there is a reasonable work-around. It will take some time to figure out the right way to handle this.

from pynndescent.

Potentially I have a fix here. Hopefully this works... I won't have time to test it for a while as I'm travelling very soon.

from pynndescent.

Thanks a lot, I'll give it a try!

from pynndescent.

Thanks! This patch seems to work but only for the sparse case. To handle the dense case I changed renumbaify_tree to the following, which seems to fix the issue:

def renumbaify_tree(tree):
    if tree.hyperplanes[0].ndim == 1:
        hyperplanes = numba.typed.List.empty_list(dense_hyperplane_type)
    else:
        hyperplanes = numba.typed.List.empty_list(sparse_hyperplane_type)
    ....

With this change I'm able to serialize a UMAP instance using joblib, and load it back.

from pynndescent.

@adilosa's snippet also got me unstuck.

from pynndescent.