[Disscussion] Can we align GLM-130B to human like chatgpt? about glm-130b HOT 7 OPEN

@smeyerhot Trl is Transformer Reinforcement Learning a library built by Huggingface for training language models with PPO. Trlx is an extension of Trl built by CarperAI. Both cover the same use-case for training models using reinforcement learning with human feedback. You can also build the same functionality with actor-critic ppo in PyTorch although it would require more extensive domain knowledge. You do not have to use glm-10b but it is publicly available on Huggingface's model hub unlike 130b which requires you to apply for access. You can use any encoder-decoder or decoder-only model. We are on an issue relating to GLM for aligning human feedback with the model which is why I suggested using the 10b parameter one.

from glm-130b.

Certainly. The alignment for GLM-130B could be important, and we are on preliminary surveying.

from glm-130b.

You could use the current glm-10b on huggingface with trl/trlx to construct a model with rlhf.

from glm-130b.

What is trl/trlx? I am very interested in this use case. Why must the 10-b parameter model be used for rlhf?

from glm-130b.

I am actively working on this task and would be very interested in further development coordination.

from glm-130b.

chatgpt can generate the format text and image. this need to keep the pertaining data in original format

from glm-130b.

hi gays, I use bloom to implement ppo successfully.

But I found the Bloom model use the AutoModelForCausalLM function.

however, the glm is using the AutoModelForSeq2SeqLM function.

there is no LM in AutoModelForSeq2SeqLM model, so do u know how to correct ?

from glm-130b.