Use GLM-130B on machine translation task. about glm-130b HOT 4 CLOSED

@BrightXiaoHan Hi,

Using pre-trained LLMs for machine translation would be interesting. However, I am sorry that we do not have enough time to dive into the topic. I think it is absolutely possible as we also observe some evidence in zero-shot/few-shot translation of GLM-130B.

If you would conduct experiments with GLM/GLM-130B on English-Chinese translation, we are more than glad to include your results and cite you in our updated paper. Thanks again for your interest in GLM-130B.

from glm-130b.

Why didn't you consider adding TLM-like objective from XLM in the training process, which seems more beneficial to cross-language tasks between Chinese and English?

from glm-130b.

Thanks for your suggestion. However, translation is not a main purpose of GLM-130B's training. Our goal is to build a language model that understand people's query well. Therefore, we'd like to invest more expensive computation on training it on general corpora rather than translation. On the other hand, the zero-shot/few-shot translation ability could arise automatically, which is enough for common use.

If you wanna improve GLM-130B's translation between English and Chinese, we encourage you to fine-tune it on translation datasets via the code repo for training. We have been too busy recently and are afraid that we can't spend more time on fine-tuning GLM-130B on translation datasets.

from glm-130b.

@BrightXiaoHan Hi, do you have to try it on downstream tasks?

from glm-130b.