How to use SNOMED trained model about medcat HOT 4 CLOSED

Thank you for letting us know of this issue! We've not encountered this before.
And after having looked at this in some detail, I can confirm that it is indeed a general issue not just something to do with your specific installation.

This has something to do with how the weighted average function is pickled within the model pack (or more specifically, the config).

The easiest workaround would be:

cat.config.linking.filters.weighted_average_function = lambda step: max(0.1, 1 - (step ** 2 * 0.0004))

This will ensure there's a valid method there to be called.

PS:
I'll be looking into the specific issue and a general resolution.
PPS:
Seems to work fine on python 3.10. So there's some issue with the differences between python 3.10 and python 3.11.

from medcat.

Works gracefully after downgrading to Python 3.10. BTW I'll leave the issue open until a general resolution is available.

Thanks!

I'm glad downgrading to Python 3.10 worked for you (I expected it would).
However, overall this still remains an issue for us with python 3.11. Downgrading may not always be an option. And even if it is, it may not be desirable (e.g the speed improvements in 3.11). So we do not wish to have that as a general recommendation.

With that said, I've prepared a PR (#411) that should fix this on the fly. Though I'm still not fully sure what the underlying issue is. I've managed to identify that there is some issue with dumping functools.partial methods using dill with older python versions (3.10 and below) and subsequently loading them in later ones (3.11 and onwards). And the fix I've proposed really only addresses the default value. I don't expect many models to have vastly different weighted average functions set, but it would be better if we were able to solve it in a more general manner.

from medcat.

Works gracefully after downgrading to Python 3.10.
BTW I'll leave the issue open until a general resolution is available.

Thanks!

from medcat.

Closing as #411 was merged in. Internal discussion suggests we're unlikely to come to a more robust solution with a reasonable time investment.

from medcat.