Improve ranking about npm-search HOT 27 CLOSED

yup sorry with all the holidays I did not had time to look into it.
What we did in the past is to pop a new instance that will create a new index and once it's ready swap the index.

I'll try to setup that today or tomorrow, full indexing takes more than a week.

from npm-search.

Yes it's currently running ☺️

from npm-search.

no frontend reordering :p I thought putting popular sooner in the ranking would help but it's the rule that promote those package indeed.

we could remove the rule for promoting alternativeName? (I'm not entirely when it's relevant)
and also add a rule to demote deprecated packages.
wdyt @Haroenv?

from npm-search.

Oh I see, I previously looked only at the index config but there's a whole bunch of logic that adds alternative names in the code. Then my previous suggestion stands - do this only if the package seems relevant based on other metadata.

from npm-search.

Awesome, it works really well! I tested all packages from my initial post here as well as those from comments in #951 and all of them return relevant results now.

from npm-search.

Sorry for the delay, it's hard to allocate time internally 🙇🏻 Thanks a lot for your patience

from npm-search.

Technically I'm not sure how this could be done, but we could indeed have a certain download number threshold before adding alternative names maybe? That way at least those three examples would not show up anymore, it bothered me too.

The one that will be harder to avoid is re-act as Algolia sees that as a single typo, which still may list higher than other matches usually would.

from npm-search.

just ordering by download should help no?

from npm-search.

The problem is that custom ranking comes after word relevance, unless you mean resorting in the frontend @bodinsamuel

from npm-search.

Downloads are already used as part of the ranking but as I understand they have lower priority than the match on project name, so they don't help here.

Something I'm not 100% sure about though - I would expect that "ignored words" apply only to whole words, not parts of them so in "vuejs" the "js" suffix would not be ignored. But based on the search results, it either is ignored even when it's a suffix (in which case my proposed solution would help) or it's some other algolia feature that makes this rank higher (in which case the threshold idea may not help).

from npm-search.

The js is not an ignored word, but it's listed as an alternativeName, which is listed after the official name in searchableAttributes

from npm-search.

deprecated should already show up lower according to ranking, but if you spell it correctly, it should still be the first result, as sometimes you would be looking for that.

I think we can add a certain popularity threshold before packages get alternative names. Will look into a PR

from npm-search.

I agree you should get deprecated packages when spelled correctly but not sure about alternative names - there it's a similar problem that if the name is similar enough, deprecation has too low priority to push the result down. My idea was to use multiple attributes for this because downloads may not always be enough, e.g. add it only if all:

1. Is not deprecated.
2. Has 1k npm downloads or 10k jsDelivr downloads.
3. Has non-empty description field or a readme file or a source repository linked or homepage.

If you didn't want to use the other rules then I suppose a higher popularity threshold.

from npm-search.

#951 should actually fullfill most of those use cases, at the cost of not showing popular alternative names that match in a first or second position. I think those cases are fairly small though, only one I could think of is places.js, hogan.js or floating.js (small one) but they all still eventually show up

from npm-search.

Any update on the reindex progress? Are we able to test the new attributes now?

from npm-search.

I don't think we started a full reindex, as it stops the synchronisation, and would take a very long time to catch up, while we don't yet have code for handling multiple instances.

from npm-search.

@bodinsamuel did you have time to set up the other instance?

from npm-search.

NB: you can test what you want using the tmp index named npm-search-bootstrap-6 it will be renamed npm-search-6 at some point. And then will replace the main index.

from npm-search.

The index doesn't use the newly added field yet because we wanted to test it first after all packages are updated (0e5194d) without affecting the main index. Considering the bootstrap runs in a separate index anyway, what would be the best course of action? I suppose we could update the config right away?

from npm-search.

ah true, forgot about that. Since we did not merge, all manual changes will be erased after some time by the script so not good. We could merge the PR in master and do not deploy the main process

from npm-search.

I have swapped the index since it had fully reindexed.
Duplicated it with the name npm-search-test and manually applied your changes from #994 on it

from npm-search.

@bodinsamuel when you have time please check and swap the indexes if possible 😄

from npm-search.

@bodinsamuel @Haroenv ping, can we get this live, please? 😄

from npm-search.

I'll merge 👌🏻

from npm-search.

Also not sure if the indexes were swapped yet.

from npm-search.

Applied on the main index 👍🏻

from npm-search.

Awesome! https://twitter.com/jsDelivr/status/1552660335219429377

from npm-search.