ftarlaci / gpt2sqa Goto Github PK
View Code? Open in Web Editor NEWFine-tuning GPT-2 Small for Question Answering
License: Apache License 2.0
Fine-tuning GPT-2 Small for Question Answering
License: Apache License 2.0
Hi! I'm researching this for a college project and I need a bit of help setting this up.
I cloned the repo and created a virtual env in python 3.6, install the requirements.txt and I ran python setup.py build
and then python setup.py install
after that I try to run the the training command:
python gpt2_squad.py --output_dir=output/ --train_file=dataset/dataset/train-v2.0.json --do_train --train_batch_size=32 --predict_file=dataset/dataset/dev-v2.0.json --do_predict
but it returns this Error:
Traceback (most recent call last): File "gpt2_squad.py", line 39, in <module> from gpt2sqa.file_utils import PYTORCH_PRETRAINED_GPT2_CACHE, WEIGHTS_NAME, CONFIG_NAME File "/home/domainproxy/NateCusack/QA_Proj/GPT2sQA/gpt2sqa/__init__.py", line 5, in <module> from gpt2sqa.gpt2.gpt2model import GPT2Model File "/home/domainproxy/NateCusack/QA_Proj/GPT2sQA/gpt2sqa/gpt2/gpt2model.py", line 7, in <module> from gpt2_question_answering.gpt2.layer_norm import LayerNorm ModuleNotFoundError: No module named 'gpt2_question_answering'
Sorry if this is a really basic question I'm still pretty new to ML.
Thanks in Advance!
Nathan
It would be great if you could share with us some rough results on SQuAD v2.0/v1.1 using the smallest model (117M). Thanks in advance!
Could you please specify the System requirements to run this program like GPU.
Because I have 8GB RAM and 4GB Graphics card and I am facing runtime errors while trying to run this code
Created a fork, did some minor changes. Used the following pip installs
#!pip install torch>=0.4.1 tqdm boto3 regex
!pip install tqdm boto3 regex
!pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu117
https://github.com/thistleknot/GPT2sQA
Get this as soon as epoch starts
total_missed:0
12/27/2022 20:56:23 - INFO - main - ***** Running training *****
12/27/2022 20:56:23 - INFO - main - Num orig examples = 130319
12/27/2022 20:56:23 - INFO - main - Num split examples = 130334
12/27/2022 20:56:23 - INFO - main - Batch size = 32
12/27/2022 20:56:23 - INFO - main - Num steps = 12216
0% 0/4073 [00:00<?, ?it/s]
Epoch: 0% 0/3 [00:00<?, ?it/s]^C
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.