Code Monkey home page Code Monkey logo

textgeneration's Introduction

README

  • based on pytorch / transformers and pytorch_lightning

train data description

  • json
[
    {"title": "xxxx", "body": "xxxxx"},
    {"title": "xxxx", "body": "xxxxx"}
    ...
]

GPT2

  • pretrained model: CKIP Lab GPT2-base
  • Two generation methods:
    • contextual calibration
    • normal beam search

fine-tune

python train_script.py --model_type gpt2 --train_data data/covid_100000.json --max_len 300 --batch_size 4 --num_workers 15 --lr 3e-4 --gpus 1 --max_epochs 4 --save_top_k 2
  • python train_script.py --help to see more info.

Generation

Normal Beam Search

python generate.py --model_type gpt2 --ckpt lightning_logs/version_4/checkpoints/epoch=3-step=21971.ckpt --prompt 疫苗 --maxlen 500 --num_seq 2
  • python generate.py --help to see more info.

Contextual Calibration

python generate.py --ckpt gpt2_ckpt/epoch=3  --prompt "日本禁止進口鳳梨"  --maxlen 150 --num_seq 5 --to result_3.txt --model_type gpt2-calibration --device cuda --gpu 0

Bert2Bert

finetune

python train_script.py --model_type bert2bert --train_data data/covid_100000.json --max_len 300 --batch_size 4 --num_workers 15 --lr 3e-4 --gpus 1 --max_epochs 4 --save_top_k 2
  • python train_script.py --help to see more info.

Generation

python generate.py --model_type bert2bert --ckpt lightning_logs/version_4/checkpoints/epoch=3-step=21971.ckpt --prompt 疫苗 --maxlen 500 --num_seq 2
  • python generate.py --help to see more info.

textgeneration's People

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.