duplicate terms in name affecting scoring about openstreetmap HOT 3 OPEN

Oh yeah, this is a good one. It's kinda inherent to the scoring mechanisms of Elasticsearch.

Maybe we can solve this by modifying the scoring settings of Elasticsearch, which is easier and more flexible in newer versions. We might be able to split altnames into separate fields in a clever way as well. But I think this sort of case will be pretty hard, though not impossible, to solve.

from openstreetmap.

Last night I went into a deep-dive on tuning the BM25 algorithm, I suspect that we could reduce k1 to either 0 or near 0 to resolve this issue.

It's a big topic in itself so I'll start a discussion over on pelias/schema

[edit] I think maybe we actually want to set k1=1?

from openstreetmap.

Since we've decided this issue is mostly unsolvable without major changes to how we index data in Elasticsearch, let's talk mitigation:

In this particular case, one name is a prefix of the other. Is it worth it to detect that and skip the shorter version?

In the other main case I've noticed with OSM data, the names only differ by whitespace. That one might be trickier, but we could probably still do it.

from openstreetmap.