yangsu2022 / gpt2-cpp-ov Goto Github PK

This is a simple repo with an implementation of the GPT2 tokenizer + execution of the GPT2 ONNX model provided here.

The cmake script will look for onnxruntime header files and dynamic library using the repo structure based from the $HOME directory. After compiling make sure that the dynamic library can be found by the runtime, ie. set LD_LIBRARY_PATH accordingly.

The other three dependencies are already included in this project:

• simdjson
• ctre
• cxxopts

Compiling the binary requires a C++17 compliant compiler. Additionally the ONNX model is downloaded during the build process from the ONNX Model Zoo repo. The model is 634 MB large, so it may take a while to download it :)

The vocabulary and merges files are provided in this repository, but were originally obtained from the transformers repo.

mkdir build && cd build
cmake ..
make

cd build
./gpt2-generate -t "I've got a q" -n 5
Prediction: "I've got a qwerty knife!!"