Code Monkey home page Code Monkey logo

Comments (6)

aws-ennst avatar aws-ennst commented on July 22, 2024 1

Hi @philschmid. We have reproduced the issue and have a fix which will be available in an upcoming release.

from transformers-neuronx.

aliseyfi avatar aliseyfi commented on July 22, 2024 1

Can you confirm this is fixed with latest release ?

This is working for me with batch_size=2 and tp_degree=4. tp_degree should be a divisor of number of attention heads, just fyi.

from transformers-neuronx.

philschmid avatar philschmid commented on July 22, 2024

FYI. Running the sample with a batch_size=2 works.

from transformers-neuronx.

aliseyfi avatar aliseyfi commented on July 22, 2024

Hi @aws-ennst any updates on this?

from transformers-neuronx.

dacorvo avatar dacorvo commented on July 22, 2024

Can you confirm this is fixed with latest release ?

from transformers-neuronx.

jeffhataws avatar jeffhataws commented on July 22, 2024

Hi @dacorvo , I have confirmed that the GPT-2 code snippet from README.md is now working with release 2.12:

(aws_neuron_venv_pytorch) ubuntu@ip-10-0-10-149:~$ python test.py
.
Compiler status PASS
.
Compiler status PASS
Downloading (…)olve/main/vocab.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.04M/1.04M [00:00<00:00, 64.1MB/s]
Downloading (…)olve/main/merges.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 456k/456k [00:00<00:00, 38.3MB/s]
Downloading (…)/main/tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.36M/1.36M [00:00<00:00, 34.0MB/s]
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
['Hello, I\'m a language model, you know. So I\'ll give you an example.\n\nNow this is really important, and you know: if you\'re a programmer, you\'re going to have to understand these things. I don\'t mean programming, or programming about languages. I mean writing. And so the first thing I need to do is write a program for the program, and then I need to write a function. And that\'s a lot of stuff, and it\'s not something that you could just throw away.\n\nThen you don\'t have to write a compiler. You can write a language, and you can write a function, and they\'ll look at your code and say, "Okay, this is going to be a function."\n\nNow, because languages are so powerful, and because they\'re so complicated, if it\'s a function, then you can write a function, and you\'re going to be able to write a function, and you\'re going to be able to write a function. And so that\'s a big problem, and it\'s one that\'s really hard to fix.\n\nSo the first thing I need to do is write a program. And then I need to write a function, and I need to write']

Packages:

(aws_neuron_venv_pytorch) ubuntu@ip-10-0-10-149:~$ pip list | grep neuron
aws-neuronx-runtime-discovery 2.9
libneuronxla                  0.5.391
neuronx-cc                    2.8.0.25+a3ad0f342
neuronx-distributed           0.1.0
neuronx-hwm                   2.8.0.3+2b7c6da39
torch-neuronx                 1.13.1.1.9.0
torch-xla                     1.13.1+torchneuron8
transformers-neuronx          0.5.58

from transformers-neuronx.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.