Does llm-guard's text content detection technology support languages other than English, such as Chinese? about llm-guard HOT 2 CLOSED

@dongxinghua I have discovered llm-guard this week and I like to add this to one of our projects. I also bumped into question similar as yours (but Filipino instead of Chinese language). One work-around I came up with is to implement an input scanner following the implementation of Toxicity class.

https://github.com/laiyer-ai/llm-guard/blob/main/llm_guard/input_scanners/toxicity.py

You can swap with existing or your own hugging face models to support other languages or other use cases that you may have.

from llm-guard.

Hey @dongxinghua ,
thanks for raising this! In theory, some of these checks should work with Chinese but we don't guarantee that. It needs some testing, and we are planning to work on multi-language features in the coming months.

Best regards,
Oleksandr

from llm-guard.