Search Show Notes about hemppa-bot HOT 7 OPEN

Good thought! The search is a fancy dynamic thing which I don't want to deconstruct, it would be nice if search sharing was enabled and then we could generate that url and parse its output instead.

I don't know if there's a machine-consumable endpoint for mkdocs, I can't find one atm.

from hemppa-bot.

So looking into the mkdocs plugin, it looks like it does provide the search_index at https://notes.jupiterbroadcasting.com/search/search_index.json. That's a pretty bulky json, but we only need to fetch it once a week (the notes site is built every friday).
So no easy way to just query an endpoint, but we could build the index cache as part of the module.

I'll create a pr against the notes site to enable search sharing.

from hemppa-bot.

One design paradigm Hemppa uses is that it stores as much state as possible in userdata on the server. Storing the whole search index would be wasteful; pushing it up to the server every time a state change happens locally is not good.

My first opinion for bot command and subcommands:

Using a local cache of the search_index.json:

• !mkdocs add <name><url> ... sets the url to pull from for a given name.
• !mkdocs pull <name> pulls the newest search_index for <name>. We can use !cron to pull notes regularly.
• !mkdocs search <name> <search terms> does a search against the site <name>. Using the alias system, we can alias !jbnotes/ !notes or similar to !mkdocs search jbnotes

Note that I'm leaving it flexible enough so that we can support multiple sites. iirc, some of the hosts have public mkdocs instances that we might want to support, or there might be a few projects which are close to the community that we could support searchable docs.

from hemppa-bot.

from hemppa-bot.

To make sure we are on the same page, are you saying that we should store
the search index locally?

Yes. This is assuming we go with the "download the giant json and just search it" method.

Would we want to have some sort of TTL on the
index and automatically update it after x days?

The laziest approach is to have a command to manually do it, and then use the existing !cron module to update it regularly.

from hemppa-bot.

@xPMo I've got a PoC put together, what have you been doing to test? I saw your update to the Readme about the .env file. I guess I can create a private room and create a user for my testing, unless you have an easier route.

from hemppa-bot.

I've been testing on prod so far, but we should change that. Check vranki/hemppa for more info on getting an access token.

from hemppa-bot.