pytorch-rnn-text-classification's Introduction

RNN-based short text classification

This is for multi-class short text classification.
Model is built with Word Embedding, LSTM ( or GRU), and Fully-connected layer by Pytorch.
A mini-batch is created by 0 padding and processed by using torch.nn.utils.rnn.PackedSequence.
Cross-entropy Loss + Adam optimizer.
Support pretrained word embedding (GloVe).

Model

Embedding --> Dropout --> LSTM(GRU) --> Dropout --> FC.

Preprocessing

The following command will download the dataset used in Learning to Classify Short and Sparse Text & Web with Hidden Topics from Large-scale Data Collections from here and process it for training.
Also it download GloVe embeddings.

python preprocess.py

Training

The following command starts training. Run it with -h for optional arguments.

python main.py

pytorch-rnn-text-classification's People

Contributors

Stargazers

Watchers

pytorch-rnn-text-classification's Issues

UnicodeDecodeError: 'gbk' codec can't decode byte 0xa2 in position 4759: illegal multibyte sequence

When i run the ''process.py'', i got the error at
'create_tsv_file('data-web-snippets/train.txt', 'data/train.tsv')':

  File "H:/my work/Pytorch-RNN-text-classification-master\util.py", line 21, in create_tsv_file
    for line in f:
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa2 in position 4759: illegal multibyte sequence

i wonder why i use the same download txt but went wrong??
thank u.

TypeError: new(): data must be a sequence (got map)

Which Python version should I use?

When I used Python3 run the main.py I got an error:
Traceback (most recent call last):
File "main.py", line 179, in
train(train_loader, model, criterion, optimizer, epoch)
File "main.py", line 102, in train
for i, (input, target, seq_lengths) in enumerate(train_loader):
File "/media/yu/Work/ml/dl/case/Pytorch-RNN-text-classification-master/dataloader.py", line 108, in iter
yield self._create_batch()
File "/media/yu/Work/ml/dl/case/Pytorch-RNN-text-classification-master/dataloader.py", line 82, in _create_batch
seq_lengths = torch.LongTensor(map(len, string))
TypeError: new(): data must be a sequence (got map)

Recommend Projects

keishinkickback / pytorch-rnn-text-classification Goto Github PK

pytorch-rnn-text-classification's Introduction

RNN-based short text classification

Model

Preprocessing

Training

pytorch-rnn-text-classification's People

Contributors

Stargazers

Watchers

Forkers

pytorch-rnn-text-classification's Issues

UnicodeDecodeError: 'gbk' codec can't decode byte 0xa2 in position 4759: illegal multibyte sequence

TypeError: new(): data must be a sequence (got map)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent