Add hook for summarization about core HOT 2 CLOSED

It's useful to have inside CheshireCat a way to summarize text/documents before saving embeddings.

That's a great idea, summarization could be useful for:

• saving in memory conversation chunks and not only user utterances / documents
• when a document is ingested, we should also memorize summaries and not only simple text chunks
• when the prompt or any of its contents gets too large, summarize parts of it

So I'll propose you this:

• let's introduce a method for summarization in the main Cat class, similar to present methods like get_hyde_embedding or recall_memories_from_text.
• we than close the PR and reason on how to design all of this methods to be pluggable as hooks, with a good default in the plugin folder

Sounds good? Any more idea?

We can use Hugging Face maybe for the default one? let me know :)

We just need a summarizer chain on the already present LLM, with a default prompt.
A good example is here with HyDE embedding.
Take a look at langchain chains to see what is possible.
I'm here for any question! Thanks a lot

from core.

Sounds good!

from core.