Implement whole document "chunker" about fms-guardrails-orchestrator HOT 2 CLOSED

For code organization purposes, we likely want to implement this not as branching logic in the orchestrator itself but as a separate chunker that can get called via some name (still implemented within the orchestrator repo, but from a deployment/user configuration POV, this name can be still be specified in the config).

Hmm, I don't think this approach will fit with the current design. We delegate to the ChunkerClient, which wraps the underlying ChunkersServiceClients. As this "default" implementation would not be an actual grpc ChunkersServiceClient (which is a struct, not a trait that we can implement a DefaultChunkerServiceClient for), this wouldn't work.

I can propose some alternative ways to implement this, if interested.

from fms-guardrails-orchestrator.

@declark1 definitely let us know some alternatives. The idea here was to not have yet another branching logic, like if chunker not configured or if chunker is default and instead have a stock implementation of a chunker that instead of making an external calls, returns the response based on the input text directly but in the format that the "chunker" itself would do.

This could be added in the chunk function, or we could add the default chunking mechanism as a special case handling in the chunk client itself 🤔

from fms-guardrails-orchestrator.