dmlc / gluon-nlp Goto Github PK

NLP made easy

License: Apache License 2.0

Makefile 0.15% Python 95.68% Shell 2.48% Dockerfile 0.17% C++ 0.24% C 0.01% Cuda 1.28%

mxnet gluonnlp nlp gluon deep-learning machine-learning natural-language-processing numpy nlu natural-language-understanding

gluon-nlp's Introduction

Distributed Machine Learning Common Codebase

DMLC-Core is the backbone library to support all DMLC projects, offers the bricks to build efficient and scalable distributed machine learning libraries.

Developer Channel

What's New

Note on Parameter Module for Machine Learning

Known Issues

RecordIO format is not portable across different processor endians. So it is not possible to save RecordIO file on a x86 machine and then load it on a SPARC machine, because x86 is little endian while SPARC is big endian.

Contributing

Contributing to dmlc-core is welcomed! dmlc-core follows google's C style guide. If you are interested in contributing, take a look at feature wishlist and open a new issue if you like to add something.

DMLC-Core uses C++11 standard. Ensure that your C++ compiler supports C++11.
Try to introduce minimum dependency when possible

CheckList before submit code

Type make lint and fix all the style problems.
Type make doc and fix all the warnings.

NOTE

deps:

libcurl4-openssl-dev

gluon-nlp's People

Contributors

Stargazers

Watchers

Forkers

lishengfever stevenlol huangshizhi fendaq yanghaha11514 statml ml-ai-nlp-ir akinswin yorick76ee kyocen jeanru jamesliu sweetcard anguoyang myjeffxie shujian2015 swayfreeda magicknight shaunstanislauslau mehrdad-shokri yusirkuan yelzhang szha leezu asherchan muximuxi xxcharles szhengac huangpeng1126 jxfruit george86028 cgraywang ishitori qq547276542 zhiheng-huang jason8kang wh-forker nikolayvoronchikhin vinaygupta1234 kjeanclaude berryhn godfatherace ai3dvision ethanchiu7 shubhampachori12110095 milog17 tutty427 lalalland yeying213 abc3436645 mdmustafizurrahman anirudhacharya 0xqq kingsuo zhuyangda gdh756462786 rajathjain fuxianghua eric-haibin-lin tandysony arfu2016 oxleyobjects alexjinlei beethovenvirus gaoyangkuanglong gertie123 chinpeng melodylail longchuan1985 lianxiaolei hbcbh1999 hzitoun garima3292 anirband shu13720902 tangoscut wqw123 github-jinwei vvvictorlee ruimao1988 scotteven letteropener 952298769 powerbijay myonlywayup carsondahlberg starevelyn isr-wang julianslzr jepsonwong astonzhang ascenoputing fantasycheng hhexiy ankkhedia ahanmr fancycheung pq7799 vivek2319 8023jiao

gluon-nlp's Issues

beam search early stop

Feature request:
Stop search after beam_size EOS generated in the beam.

beam search should support skip_values

When decoding, a common use case is to skip some of the output values, such as padding, end-of-sequence, unknown token, etc.

Importing gluonnlp giving :'ImportError: cannot import name '_get_repo_url' '

All libraries were upgraded; running on Ubuntu
A simple import of gluonnlp giving the following error

Traceback (most recent call last):
  File "lm.py", line 25, in <module>
    import gluonnlp as nlp
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/__init__.py", line 23, in <module>
    from . import data
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/__init__.py", line 23, in <module>
    from .utils import *
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/utils.py", line 32, in <module>
    from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'

SimLex-999 dataset link broken

@szhengac noted a CI failure due to a broken link. Hopefully the link can be made to work again or the author can give us permission to redistribute the dataset. Progress is tracked at fh295/fh295.github.io#1

Notebooks should be converted to markdown/notedown format for easier review

In #166, I set up a notedown (i.e. notebook in markdown format) pipeline and converted word embedding notebook to use it.

We still have three more to go:

Word Embeddings Training and Evaluation
Language Modeling using NLP Toolkit
Sentiment Analysis (SA) with pretrained Language Model (LM)

'name' is an undefined name twice in evaluation.py

An undefined name may cause NameError to be raised at runtime.

flake8 testing of https://github.com/dmlc/gluon-nlp on Python 3.6.3

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

../gluonnlp/embedding/evaluation.py:380:67: F821 undefined name 'name'
                '{} is not a WordEmbeddingAnalogyFunction'.format(name))
                                                                  ^
./gluonnlp/embedding/evaluation.py:439:67: F821 undefined name 'name'
                '{} is not a WordEmbeddingAnalogyFunction'.format(name))
                                                                  ^

Improve examples with ready to use examples

It would be nice to have examples for Language Models and NMT that show an end to end example for a given sentence.

nmt:

text = "I love playing football"
...
print(output)
"ich liebe es Fußball zu spielen"

language model:

text = "the winner of the 2018 world cup is"
...
print(output)
"France"

Batch-dependent behavior in SoftmaxCEMaskedLoss?

Hi all,

Consider the following (placed in scripts/nmt/):

import mxnet as mx
import loss

if __name__ == '__main__':
    BATCH_SIZE = 3
    MAX_SEQ_LEN = 100
    VOCAB_SIZE = 26
    # Each time step has a uniform distribution over the vocabulary
    uniform_tensor = 1./VOCAB_SIZE * mx.nd.ones(shape=(BATCH_SIZE, MAX_SEQ_LEN, VOCAB_SIZE))
    # Sequences have different lengths
    valid_lens = mx.nd.array((1, 10, 100))

    loss = loss.SoftmaxCEMaskedLoss(sparse_label=False, from_logits=False)
    ce_loss = loss(uniform_tensor, uniform_tensor, valid_lens)
    print(ce_loss)

This outputs:

[0.03258096 0.32580966 3.2580965 ]
<NDArray 3 @cpu(0)>

However, these should all be 3.2580965 (which is ln(26)), as this is the (average) CE across all valid timesteps, per sequence.

The problem is the averaging step of CE is not aware of valid_length, which leads to "masking zeros" being included in the mean. Compare with, e.g., Sockeye's implementation.

One workaround is to increase sample weights so their weighted sum compensates for the zeros. For example, adding the following line before calling super in SoftmaxCEMaskedLoss:

sample_weight = sample_weight * (sample_weight.shape[1] / F.reshape(valid_length, shape=(-1, 1, 1)))

I have not evaluated how this affects current MT models. This causes short sequences to have smaller per-token loss, weighting long sequences more.

However, perhaps we should edit SoftmaxCEMaskedLoss for correctness (or someone point out if I'm mistaken); I can submit a PR. The CE loss of an sequence should not depend on the batch it is in.

[DISCUSS] How do we reorganize the homepage for easier feature discovery

right now it's not obvious what is available, especially lots of the code are in scripts. as a starter, here are some ideas:

rename scripts to Toolkits, and list them in the home page
split the scripts into their individual pages instead of one unified page

Pre-trained Embeddings should be converted to npz for faster downloading and loading

Implementing Differentiable Neural Computer

We could implement DNC and test it on the bAbI dataset to illustrate how to implement memory networks in Gluon-NLP.

DNC paper
DNC code

stochastic beam search / sampling

Need sampling with temperature for inference.

At each search step, sample beam_size tokens from the token distribution instead of taking the top k.

Reshape argument issues in running attention cells

a = mx.nd.array([[1,2,3],[2,3,4]])
print(mx.nd.reshape(a, shape=(3,2), reverse=True))
print(a.reshape(shape=(3,2), reverse=True))

The first one works, the second one says "TypeError: Only 'shape' is supported as keyword argument. Got: shape, reverse."
Versions (latest):
gluonnlp.version = '0.3.3post1'
mxnet.version = '1.2.1'

Originally hitting this issue in the attention_cell code in gluonnlp, reproduced it in the aforementioned toy example based on that.

Original exception thrown at:

~/anaconda3/envs/mxnet_latest_p36/lib/python3.6/site-packages/gluonnlp/model/attention_cell.py in _compute_weight(self, F, query, key, mask)
    250         query = F.transpose(query.reshape(shape=(0, 0, self._num_heads, -1)),
    251                             axes=(0, 2, 1, 3))\
--> 252                  .reshape(shape=(-1, 0, 0), reverse=True)

some comments

I tried to read the document by assuming I have very limited knowledge about NLP and MXNet. It's not necessary to address all of them before the first lease.

Front-page

Add some comments into the quick example. such why we need to specify the name 'glove', what does 'glove.6B.50d.txt' mean, try to add print(baby). explain what is mx.nd.dot (do we have an inner op, i feel it's more straightforward)
Probably add a link to the gluon crash course (but the crash course may contains other things not quite necessary for NLP)

Get started

I feel we can remove this page, it's kinds of duplicated to the a quick example and the later embedding example.

Examples

Using Pre-trained Word Embeddings

general comments,

we should explain the key concepts in details, such as "pre-trained", "word embedding", "attach", "vocab". think about we are writing a book for beginner users. or add links to refer some easy read documents
avoid to use too much inline code blocks, may use gluon or just gluon instead of gluon. otherwise there are too many red blocks and make the doc ugly.
use intersphinx to refer mxnet.gluon.text and other mxnet apis
print as much as possible, it can save a lot of explanation words

details

remove

import os
import sys
curr_path = os.getcwd()
sys.path.append(os.path.join(curr_path, '..', '..'))

it's not necessary if already installed gluonnlp

I cannot get the meaning of "let us index words, attach pre-trained word embeddings for them, " , let's explain a little bit more here.
for In [2], split into two blocks and explain them separatly

import re # move to begin?

text = " hello world \n hello nice world \n hi world \n"

tokens = filter(None, re.split(' |\n'))  # why we need filter
print(tokens)

counter = data.count_tokens(tokens)

I didn't list all details here, we can update this document according to the principals how we make these comments.

LM and Sentiment Analysis

documents are too short. consider explain both the model (may be briefer than our gluon book) and APIs (should in rich details)

API

Data API

add examples how to use the dataset API. since they are similar to each other, we can add an example at the begging of each section. We can apply this principal to other APIs as well.
add a table to list all supported datasets with columns such as "# examples, vocab size, data size, ..."
gluon used train, gluonvision used splits, while gluonnlp used segment. I feel we should be consistent here.
is it necessary to keep skip_empty? namely we have cases to keep empty? If not, we can remove this argument. If users asks for it, we can add back later. Same principal can be applied to other argument to keep the API simple
gluon put root as the first argument, should we follow it?

Vocab APIs

It's good we have examples to explain the APIs. But I feel they are kinds of duplicated with the embedding example. My suggestions how to distinguish them are: examples will explain the concepts and some common usages of the APIs. Users are suppose to read them to get familiar with APIs first. Then the API doc examples will concise but cover more usages, namely specifying various arguments.

Model API

I feel the model description is too brief. Users are hard to know what it is. I think we should explain them more
May create a model zoo page to list all pre-traineded model, e.g. http://gluon-vision-staging.s3-website-us-west-2.amazonaws.com/PR-40/4/model_zoo/object_detection.html

Scripts

break the long command into multiple lines.
Suggest to make the training log or a summary of the log availble. users will doubt if they are good and therefore hesitate to try them

gluonnlp should have guidelines on how to contribute notebooks vs examples

testing requirements
styles

Pypi Readme doesn't render well

As pointed out by @szha in #204 our Readme does not render well on Pypi due to relative links in the Readme.
In particular there are Quick Start Guide , Resources links at the top of the Readme, linking to the second half of the document.

Should we remove the relative links? As the document is not too long I wonder if having these links is worth having the Pypi Readme not being rendered properly.

[Discussion] Add a C Library to GluonNLP

As reported in #136, some functions can be greatly accelerated using C implementations. Would it be worthwhile to add a C library to hold these functions? In fact, FastText are generally implemented in C++ (https://github.com/facebookresearch/fastText/tree/master/src) and the bleu score in fairseq is also implemented in C (https://github.com/pytorch/fairseq/tree/master/fairseq/clib/libbleu).

NameError: name 'long_description' is not defined

pip install -U gluonnlp

Looking in indexes: http://pypi.douban.com/simple
Collecting gluonnlp
  Downloading http://pypi.doubanio.com/packages/cc/60/f2d2cbedc60f338bddf7fe820d1ee695bd8e773b9fab2537b90171d9ba18/gluonnlp-0.3.3.post0.tar.gz (146kB)
    100% |████████████████████████████████| 153kB 1.1MB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-hceoebdx/gluonnlp/setup.py", line 47, in <module>
        long_description=long_description,
    NameError: name 'long_description' is not defined

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-hceoebdx/gluonnlp/

API doc examples are currently not easy to copy/paste

users may want to use a snippet from example directly, so making the notebooks copy-friendly is important

currently the code blocks have python shell prefix ">>>" in them. see http://gluon-nlp.mxnet.io/api/notes/data_api.html

Want to add bi-lstm CRF model

Recently engaged in NLP work, has been concerned about mxnet, want to add bi-lstm CRF model, TF already has a CRF module, Hope Mxnet and gluon as soon as possible. Thank you

Test dataset issue when performing unitest with python3

This error caused this PR CI failed: #55

The detailed error msg is as below:

tests/unittest/test_datasets.py::test_men Downloading tests/data/men/MEN.tar.gz from http://clic.cimec.unitn.it/~elia.bruni/resources/MEN.tar.gz...

FAILED

=================================== FAILURES ===================================

___________________________________ test_men ___________________________________
def test_men():

    for segment, length in [("full", 3000), ("dev", 2000), ("test", 1000)]:

        data = nlp.data.MEN(
          root=os.path.join('tests', 'data', 'men'), segment=segment)
tests/unittest/test_datasets.py:146:

gluonnlp/data/word_embedding_evaluation.py:307: in init
super(MEN, self).__init__(root=root)
gluonnlp/data/word_embedding_evaluation.py:163: in init
super(WordSimilarityEvaluationDataset, self).__init__(root=root)
gluonnlp/data/word_embedding_evaluation.py:120: in init
self._download_data()
gluonnlp/data/word_embedding_evaluation.py:131: in _download_data
verify=self._verify_ssl)

Better tutorial on how to use the attention cell

User has asked for a better tutorial on how to use the AttentionCell https://discuss.mxnet.io/t/general-purpose-attention-layer/1066 . We could have a simple example about attention + multi-head attention.

word2vec pre-trained embeddings

It would be nice to have w2v pre-trained embeddings

Move custom cython operator to MXNet

The log uniform sampler used in https://github.com/dmlc/gluon-nlp/blob/master/scripts/language_model/large_word_language_model.py needs to be built with cython, making it harder to use. Ideally this operator should be supported in MXNet.

test datasets issue with unitest

This cause the PR failed: #53

It might be the same issue with #56

The error msg:

tests/unittest/test_datasets.py::test_rare_words Downloading tests/data/rarewords/rw.zip from http://www-nlp.stanford.edu/~lmthang/morphoNLM/rw.zip...

FAILED

=================================== FAILURES ===================================

_______________________________ test_rare_words ________________________________
def test_rare_words():
  data = nlp.data.RareWords(root=os.path.join('tests', 'data', 'rarewords'))
tests/unittest/test_datasets.py:172:

gluonnlp/data/word_embedding_evaluation.py:379: in init
super(RareWords, self).__init__(root=root)
gluonnlp/data/word_embedding_evaluation.py:163: in init
super(WordSimilarityEvaluationDataset, self).__init__(root=root)
gluonnlp/data/word_embedding_evaluation.py:120: in init
self._download_data()
gluonnlp/data/word_embedding_evaluation.py:131: in _download_data
verify=self._verify_ssl)
url = 'http://www-nlp.stanford.edu/~lmthang/morphoNLM/rw.zip'

path = 'tests/data/rarewords', overwrite = False

sha1_hash = 'bf9c5959a0a2d7ed8e51d91433ac5ebf366d4fb9', verify = True
def download(url, path=None, overwrite=False, sha1_hash=None, verify=True):

    """Download an given URL



    Parameters

    ----------

    url : str

        URL to download

    path : str, optional

        Destination path to store downloaded file. By default stores to the

        current directory with same name as in url.

    overwrite : bool, optional

        Whether to overwrite destination file if already exists.

    sha1_hash : str, optional

        Expected sha1 hash in hexadecimal digits. Will ignore existing file when hash is specified

        but doesn't match.

    verify : bool

        Toggle verification of SSL certificates.



    Returns

    -------

    str

        The file path of the downloaded file.

    """

    if path is None:

        fname = url.split('/')[-1]

    else:

        path = os.path.expanduser(path)

        if os.path.isdir(path):

            fname = os.path.join(path, url.split('/')[-1])

        else:

            fname = path



    if overwrite or not os.path.exists(fname) or (

            sha1_hash and not check_sha1(fname, sha1_hash)):

        dirname = os.path.dirname(os.path.abspath(os.path.expanduser(fname)))

        if not os.path.exists(dirname):

            os.makedirs(dirname)



        print('Downloading %s from %s...' % (fname, url))

        r = requests.get(url, stream=True, verify=verify)

        if r.status_code != 200:
          raise RuntimeError('Failed downloading url %s' % url)
E RuntimeError: Failed downloading url http://www-nlp.stanford.edu/~lmthang/morphoNLM/rw.zip

Position encoding bug in transformer?

The way the position encoding in scripts/nmt/transformer.py is written computes sin on 2i and cos on 2i+1. This is opposed to computing sin and cos on 2i at positions 2i, 2i+1 (@szhengac perhaps I'm misunderstanding? Please check).

Here is my proposed fix:

def _position_encoding_init(max_length, dim):
    """ Init the sinusoid position encoding table """
    assert dim % 2 == 0

    # time-wise list [0, ..., max_length] of shape (max_length, 1)
    position_enc_time = np.arange(max_length).reshape((-1, 1))

    # hidden-wise list [0, 0, 1, 1, ..., dim/2, dim/2] of shape (1, dim)
    position_enc_hidden = np.repeat(np.arange(int(dim/2)), 2).reshape((1, -1))

    # combine by broadcasting in both dimensions
    position_enc = position_enc_time / (np.power(10000, (2. / dim) * position_enc_hidden))

    # Apply the cosine to even columns and sin to odds.
    position_enc[:, 0::2] = np.sin(position_enc[:, 0::2])  # dim 2i
    position_enc[:, 1::2] = np.cos(position_enc[:, 1::2])  # dim 2i+1

    return position_enc

Vocab is not picklable

from gluon.vocab import Vocab
import pickle
pickle.dump(Vocab(), open('tt.pkl', 'w'))

Out:

AttributeError: Can't pickle local object 'Vocab.__init__.<locals>.<lambda>'

Notebooks should have download link on the page

Beam search tests are flaky

Once in a while the tests introduced in #20 will fail:

      # Begin Testing

       for vocab_num in [4, 8]:

           for decoder_fn in [RNNDecoder,

                              functools.partial(RNNDecoder2, use_tuple=False),

                              functools.partial(RNNDecoder2, use_tuple=True)]:

               decoder = decoder_fn(vocab_num=vocab_num, hidden_size=HIDDEN_SIZE)

               decoder.hybridize()

               decoder.initialize()

               for beam_size, bos_id, eos_id, alpha, K in [(2, 1, 3, 0, 1.0),  (4, 2, 3, 1.0, 5.0)]:

                   scorer = BeamSearchScorer(alpha=alpha, K=K)

                   for max_length in [10, 20]:

                       sampler = BeamSearchSampler(beam_size=beam_size, decoder=decoder, eos_id=eos_id,

                                                   scorer=scorer, max_length=max_length)

                       for batch_size in [1, 2, 5]:

                           states = decoder.begin_state(batch_size)

                           inputs = mx.nd.full(shape=(batch_size,), val=bos_id)

                           samples, scores, valid_length = sampler(inputs, states)

                           samples = samples.asnumpy()

                           scores = scores.asnumpy()

                           valid_length = valid_length.asnumpy()

                           for i in range(batch_size):

                               max_beam_valid_length = int(np.round(valid_length[i].max()))

                               step_states = _fetch_step_states(states, i, beam_size)

                               step_input = bos_id

                               npy_samples = _npy_beam_search(decoder, scorer, step_input, step_states,

                                                              eos_id, beam_size, max_length)

>                               assert_allclose(npy_samples, samples[i, :, :max_beam_valid_length])

E                               AssertionError: 

E                               Not equal to tolerance rtol=1e-07, atol=0

E                               

E                               (mismatch 20.8333333333%)

E                                x: array([[1., 2., 0., 2., 0., 2., 0., 2., 2., 0., 2., 3.],

E                                      [1., 2., 0., 2., 0., 2., 0., 2., 2., 2., 2., 3.]])

E                                y: array([[1, 2, 0, 2, 0, 2, 2, 0, 2, 0, 2, 3],

E                                      [1, 2, 0, 2, 0, 2, 2, 0, 2, 2, 0, 3]], dtype=int32)


tests/unittest/test_beam_search.py:184: AssertionError

Should we fix the seed for these tests? If so we can simply add the @pytest.mark.seed(1) decorator.

support vocab with indices assigned based on frequencies

Currently, the indices in gluonnlp.Vocab are assigned in the following order:

assign indices to special tokens, including: <unk>, <pad>, <bos>, <eos> and reserved ones
assign indices to normal tokens in the counter.

What I see missing for the language model in https://arxiv.org/pdf/1602.02410.pdf is the capability to do the following. For large scale language models that require sampling during training, it's important to build a vocab with indices generated based on word frequencies (including <unk>, <eos>, etc):

assign indices to special & normal tokens based their frequency.
If a normal token's frequency is less than min_freq, its frequency should contribute to the frequency of <unk>.

I propose we extend gluonnlp.data.Counter and introduce gluonnlp.SortedVocab:

Counter

__init__(iterable, unknown_token=None, min_freq=None)

unknown_token: if specified, tokens whose freq is under min_freq contribute to the frequency of provided unknown_token.

min_freq: if specified, tokens whose freq is under min_freq will be removed from the counter.

SortedVocab

__init__(counter=None, max_size=None, min_freq=1, unknown_token=C.UNK_TOKEN,
padding_token=C.PAD_TOKEN, bos_token=C.BOS_TOKEN, eos_token=C.EOS_TOKEN,
reserved_tokens=None)

Indexing and embedding attachment for text tokens. The indices in the vocabulary is generated based on frequencies of the tokens, including special tokens (unk, pad, eos, bos and reserved tokens). If max_size is specified, special tokens are still guaranteed to be in the vocabulary. The provided counter instance is the source of all tokens' frequencies.

@leezu @sxjscience @szha

Word Language Model script page needs more info

The scripts page should reference state of the art papers, and identify any gap we have.

Sentiment analysis script page needs more info

The scripts page should reference state of the art papers, and identify any gap we have. Also, any difference in training methodology should (such as using additional resources) should be clarified.

Implement Squad dataset

Implement the SQuad QA dataset

when moving files around, we should keep git history from the origin file

figure out how this can be accomplished
explore ways to automate such process (hook?)

Notebooks should have consistent styles

import gluonnlp as nlp and use nlp.data, instead of from gluonnlp import data

Unify bptt_batchify interface between LanguageModelStream and LanguageModelDataset

Currently there are at least the following discrepancies:

LanguageModelStream.bptt_batchify() returns a mask
- The mask can be recovered by mask = data != vocab[vocab.padding_token]
- We can either change the LanguageModelDataset.bptt_batchify() interface to also return a mask or use above formula to recover there mask where neede.
- I vote for above formula, as it explicitly defines the mask and may be clearer to users. Users anyways need a line mask = mask.as_in_context(ctx) which could then just be replaced by mask = data != vocab[vocab.padding_token]. However, I'd also be ok with chaning the LanguageModelDataset.bptt_batchify().
- LanguageModelStream.bptt_batchify() is smart and never includes <eos> in the data batch but only in the target batch.
- LanguageModelDataset.bptt_batchify() instead assumes the data doesn't contain <bos> but only eos.
- See https://github.com/dmlc/gluon-nlp/blob/master/gluonnlp/data/stream.py#L415

@cgraywang @szha @eric-haibin-lin

gluonnlp.data.transforms.NLTKMosesTokenizer broken with nltk 3.3

>>> import nltk
>>> print(nltk)
<module 'nltk' from '/usr/local/lib/python2.7/dist-packages/nltk/__init__.pyc'>
>>> print(nltk.__version__)
3.3
>>>
>>> import gluonnlp as nlp
>>> nlp.data.NLTKMosesTokenizer()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gluonnlp/data/transforms.py", line 175, in __init__
    raise ImportError('NLTK or relevant packages are not installed. You must install NLTK '
ImportError: NLTK or relevant packages are not installed. You must install NLTK in order to use the NLTKMosesTokenizer. You can refer to the official installation guide in https://www.nltk.org/install.html .

If I install nltk 3.2.5, it works. Does it mean nltk is broken ...?

website should be served with proper https setup

embedding API should allow reading directly from zip/gz/tgz

currently embedding API still requires unzipping and keeping the text in original form on disk, which can be huge.

Better tokenizer for Chinese

Currently, we support the MosesTokenizer in NLTK and the spaCy tokenizer. Both are not very good for tokenizing Chinese sentences. We could consider adding some Chinese tokenizers.

I've searched online and found these two candidates (There would be more):

Standford segmenter: https://nlp.stanford.edu/software/segmenter.html , http://www.nltk.org/_modules/nltk/tokenize/stanford_segmenter.html
Jieba tokenizer: https://github.com/fxsjy/jieba

We can add them in transforms.py.

Setup fails because of UnicodeDecodeError

This line readme = open('README.rst').read() in setup.py can cause an decoding error on windows. For example my system uses cp950 as default code page.

please specify an explicit encoding (I guess utf-8 would be correct)

from io import open # for python2.7 compatibility
readme = open('README.rst', encoding="utf-8").read()

OSError in examples/word_embedding_evaluation/word_embedding_evaluation.ipynb

The detail of the error:

Notebook error:

OSError in examples/word_embedding_evaluation/word_embedding_evaluation.ipynb:

[Errno 14] Bad address: 'pandoc'

Host files in multiple S3 regions

Downloading files from S3 hosted in the US can be painfully slow from Hong Kong (and likely mainland). @szha can we easily host them in a second region?
In my experience the bandwidth to Europe is good, in case that the Hong Kong region is not yet ready.

gluonnlp.batchify.Stack() does not accept input parameter

According to the code base in gluonnlp.data.batchify , Stack() should be able to take dtype as input if run on a jupyter notebook

import gluonnlp 
import numpy as mp
gluonnlp.data.batchify.Stack(dtype=np.float32)

Running the above code in notebook results in:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-fbef5d961433> in <module>()
      1 import gluonnlp
      2 import numpy as mp
----> 3 gluonnlp.data.batchify.Stack(dtype=np.float32)

TypeError: object() takes no parameters`

However, the same code works fine on PyCharm

result of the same code on PyCharm, remotely connected to the same EC2 instance is:

<gluonnlp.data.batchify.Stack object at 0x7febbacade80>

environment

mxnet: 1.3.0
gluonnlp:   INSTALLED: 0.3.2 (latest)

Jupyter version:

packages in environment at /home/ubuntu/anaconda3/envs/mxnet_p36:
Name                    Version                   Build  Channel
ipython                   6.4.0                    py36_0    conda-forge
ipython_genutils          0.2.0                    py36_0    conda-forge

Run inline example code on CI

Run inline example code on CI to prevent bugs like #239 . This requires some configuration changes, but should be straightforward.

gluon-nlp should provide citation information

@mli suggested to provide reference information on github readme for software.

[Discussion] Use control-flow operators (e.g. foreach) to unroll dynamic graph models

apache/mxnet#11948 introduces foreach to the contrib.unroll method which makes RNNs hybridizable and easy to export to json for deployment. In theory it introduces extra overhead such as calling nd.where. We need to evaluate the performance impact (I assume it's relatively small) and change gluon nlp language models to hybrid blocks for ease of deployment if possible. It should not affect existing pre-trained models since the parameter structure won't change.

@zheng-da @szha

Add flake8 to CI

@cclauss has pointed out several issues discoverd by running flake8. We may want to run it on CI too.

Currently the following issues need to be addressed before flake8 will run without errors on CI (or selectively disabled):

./setup.py:5:1: F401 'shutil' imported but unused
./setup.py:6:1: F401 'sys' imported but unused
./docs/conf.py:15:1: F401 'subprocess' imported but unused
./docs/conf.py:15:10: E401 multiple imports on one line
./docs/conf.py:16:1: F401 'shlex' imported but unused
./docs/conf.py:17:1: F401 'recommonmark' imported but unused
./docs/conf.py:18:1: F401 'sphinx_gallery' imported but unused
./docs/conf.py:31:1: E402 module level import not at top of file
./docs/conf.py:84:1: E265 block comment should start with '# '
./docs/conf.py:101:1: E265 block comment should start with '# '
./docs/conf.py:103:1: E265 block comment should start with '# '
./docs/conf.py:120:1: E265 block comment should start with '# '
./docs/conf.py:123:1: E265 block comment should start with '# '
./docs/conf.py:127:1: E265 block comment should start with '# '
./docs/conf.py:131:1: E265 block comment should start with '# '
./docs/conf.py:137:1: E265 block comment should start with '# '
./docs/conf.py:140:1: E265 block comment should start with '# '
./docs/conf.py:187:1: E402 module level import not at top of file
./docs/conf.py:194:1: E302 expected 2 blank lines, found 1
./docs/conf.py:207:1: E122 continuation line missing indentation or outdented
./docs/conf.py:220:1: W391 blank line at end of file
./docs/md2ipynb.py:26:46: E228 missing whitespace around modulo operator
./gluonnlp/__init__.py:27:1: F403 'from .vocab import *' used; unable to detect undefined names
./gluonnlp/_constants.py:34:15: E114 indentation is not a multiple of four (comment)
./gluonnlp/_constants.py:34:15: E116 unexpected indentation (comment)
./gluonnlp/_constants.py:35:15: E114 indentation is not a multiple of four (comment)
./gluonnlp/_constants.py:35:15: E116 unexpected indentation (comment)
./gluonnlp/base.py:30:24: F821 undefined name 'unicode'
./gluonnlp/data/__init__.py:23:1: F403 'from .utils import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:25:1: F403 'from .registry import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:27:1: F403 'from .transforms import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:29:1: F403 'from .sampler import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:31:1: F403 'from .candidate_sampler import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:33:1: F403 'from .dataset import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:35:1: F403 'from .stream import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:37:1: F403 'from .language_model import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:39:1: F403 'from .sentiment import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:41:1: F403 'from .word_embedding_evaluation import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:43:1: F403 'from .word_embedding_training import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:45:1: F403 'from .conll import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:47:1: F403 'from .translation import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:49:1: F401 '.batchify' imported but unused
./gluonnlp/data/__init__.py:51:1: F403 'from .question_answering import *' used; unable to detect undefined names
./gluonnlp/data/__init__.py:53:12: F405 'utils' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:53:28: F405 'transforms' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:53:49: F405 'sampler' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:54:12: F405 'dataset' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:54:30: F405 'language_model' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:54:55: F405 'sentiment' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:55:12: F405 'word_embedding_evaluation' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:55:48: F405 'stream' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:56:12: F405 'word_embedding_training' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:56:46: F405 'conll' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:57:12: F405 'translation' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:57:34: F405 'registry' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/__init__.py:57:53: F405 'question_answering' may be undefined, or defined from star imports: .candidate_sampler, .conll, .dataset, .language_model, .question_answering, .registry, .sampler, .sentiment, .stream, .transforms, .translation, .utils, .word_embedding_evaluation, .word_embedding_training
./gluonnlp/data/candidate_sampler.py:71:13: E741 ambiguous variable name 'l'
./gluonnlp/data/conll.py:183:17: E127 continuation line over-indented for visual indent
./gluonnlp/data/conll.py:236:17: E127 continuation line over-indented for visual indent
./gluonnlp/data/conll.py:297:17: E127 continuation line over-indented for visual indent
./gluonnlp/data/conll.py:354:17: E127 continuation line over-indented for visual indent
./gluonnlp/data/dataset.py:95:9: E306 expected 1 blank line before a nested definition, found 0
./gluonnlp/data/language_model.py:258:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/language_model.py:305:18: F821 undefined name 'tarfile'
./gluonnlp/data/language_model.py:308:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/registry.py:76:16: E713 test for membership should be 'not in'
./gluonnlp/data/sampler.py:64:79: E502 the backslash is redundant between brackets
./gluonnlp/data/sampler.py:390:25: E127 continuation line over-indented for visual indent
./gluonnlp/data/sampler.py:391:25: E127 continuation line over-indented for visual indent
./gluonnlp/data/stream.py:34:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:48:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:56:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:66:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:111:67: E228 missing whitespace around modulo operator
./gluonnlp/data/stream.py:124:84: E228 missing whitespace around modulo operator
./gluonnlp/data/stream.py:129:87: E228 missing whitespace around modulo operator
./gluonnlp/data/stream.py:137:67: E228 missing whitespace around modulo operator
./gluonnlp/data/stream.py:149:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:271:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/stream.py:291:90: E502 the backslash is redundant between brackets
./gluonnlp/data/translation.py:175:35: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:177:32: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:191:43: E203 whitespace before ':'
./gluonnlp/data/translation.py:197:77: E502 the backslash is redundant between brackets
./gluonnlp/data/translation.py:222:35: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:225:32: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:269:77: E502 the backslash is redundant between brackets
./gluonnlp/data/translation.py:297:35: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:300:32: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:350:77: E502 the backslash is redundant between brackets
./gluonnlp/data/translation.py:372:35: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:375:32: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:404:77: E502 the backslash is redundant between brackets
./gluonnlp/data/translation.py:429:35: E127 continuation line over-indented for visual indent
./gluonnlp/data/translation.py:432:32: E127 continuation line over-indented for visual indent
./gluonnlp/data/utils.py:38:36: E261 at least two spaces before inline comment
./gluonnlp/data/utils.py:72:1: E302 expected 2 blank lines, found 1
./gluonnlp/data/word_embedding_evaluation.py:104:93: E502 the backslash is redundant between brackets
./gluonnlp/data/word_embedding_evaluation.py:105:90: E502 the backslash is redundant between brackets
./gluonnlp/data/word_embedding_evaluation.py:106:89: E502 the backslash is redundant between brackets
./gluonnlp/embedding/__init__.py:23:1: F403 'from .token_embedding import *' used; unable to detect undefined names
./gluonnlp/embedding/__init__.py:25:1: F401 '.evaluation' imported but unused
./gluonnlp/embedding/__init__.py:27:12: F405 'token_embedding' may be undefined, or defined from star imports: .token_embedding
./gluonnlp/embedding/evaluation.py:39:101: E501 line too long (107 > 100 characters)
./gluonnlp/embedding/evaluation.py:44:101: E501 line too long (104 > 100 characters)
./gluonnlp/embedding/evaluation.py:130:8: E713 test for membership should be 'not in'
./gluonnlp/embedding/evaluation.py:380:67: F821 undefined name 'name'
./gluonnlp/embedding/evaluation.py:439:67: F821 undefined name 'name'
./gluonnlp/embedding/token_embedding.py:459:5: E303 too many blank lines (2)
./gluonnlp/initializer/__init__.py:23:1: F403 'from .initializer import *' used; unable to detect undefined names
./gluonnlp/initializer/__init__.py:25:11: F405 'initializer' may be undefined, or defined from star imports: .initializer
./gluonnlp/loss/__init__.py:23:1: F403 'from .activation_regularizer import *' used; unable to detect undefined names
./gluonnlp/loss/__init__.py:25:11: F405 'activation_regularizer' may be undefined, or defined from star imports: .activation_regularizer
./gluonnlp/loss/activation_regularizer.py:66:42: E261 at least two spaces before inline comment
./gluonnlp/loss/activation_regularizer.py:128:42: E261 at least two spaces before inline comment
./gluonnlp/model/__init__.py:52:1: F403 'from .language_model import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:54:1: F403 'from .beam_search import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:56:1: F403 'from .attention_cell import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:58:1: F403 'from .utils import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:60:1: F403 'from .parameter import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:62:1: F403 'from .block import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:64:1: F403 'from .highway import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:66:1: F403 'from .convolutional_encoder import *' used; unable to detect undefined names
./gluonnlp/model/__init__.py:68:1: F401 '.train' imported but unused
./gluonnlp/model/__init__.py:70:11: F405 'language_model' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:70:36: F405 'beam_search' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:70:58: F405 'attention_cell' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:71:11: F405 'utils' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:71:27: F405 'parameter' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:71:47: F405 'block' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:71:63: F405 'highway' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:72:11: F405 'convolutional_encoder' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:103:39: F405 'standard_lstm_lm_200' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:104:39: F405 'standard_lstm_lm_650' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:105:40: F405 'standard_lstm_lm_1500' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:106:35: F405 'awd_lstm_lm_1150' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:107:34: F405 'awd_lstm_lm_600' may be undefined, or defined from star imports: .attention_cell, .beam_search, .block, .convolutional_encoder, .highway, .language_model, .parameter, .utils
./gluonnlp/model/__init__.py:111:69: E228 missing whitespace around modulo operator
./gluonnlp/model/attention_cell.py:32:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/beam_search.py:50:49: E261 at least two spaces before inline comment
./gluonnlp/model/beam_search.py:99:13: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:155:13: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:194:101: E501 line too long (133 > 100 characters)
./gluonnlp/model/beam_search.py:245:28: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:269:28: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:275:31: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:278:16: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:385:16: E127 continuation line over-indented for visual indent
./gluonnlp/model/beam_search.py:386:16: E127 continuation line over-indented for visual indent
./gluonnlp/model/block.py:41:65: E228 missing whitespace around modulo operator
./gluonnlp/model/block.py:46:44: E261 at least two spaces before inline comment
./gluonnlp/model/block.py:58:75: E228 missing whitespace around modulo operator
./gluonnlp/model/convolutional_encoder.py:136:52: E261 at least two spaces before inline comment
./gluonnlp/model/highway.py:125:29: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:73:49: E261 at least two spaces before inline comment
./gluonnlp/model/language_model.py:109:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/language_model.py:139:49: E261 at least two spaces before inline comment
./gluonnlp/model/language_model.py:170:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/language_model.py:236:12: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:283:12: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:288:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/language_model.py:325:12: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:368:12: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:411:12: E127 continuation line over-indented for visual indent
./gluonnlp/model/language_model.py:416:1: E305 expected 2 blank lines after class or function definition, found 1
./gluonnlp/model/utils.py:27:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/utils.py:102:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/train/__init__.py:26:1: F403 'from .language_model import *' used; unable to detect undefined names
./gluonnlp/model/train/__init__.py:27:1: F403 'from .cache import *' used; unable to detect undefined names
./gluonnlp/model/train/__init__.py:28:1: F403 'from .embedding import *' used; unable to detect undefined names
./gluonnlp/model/train/__init__.py:30:11: F405 'language_model' may be undefined, or defined from star imports: .cache, .embedding, .language_model
./gluonnlp/model/train/__init__.py:30:36: F405 'cache' may be undefined, or defined from star imports: .cache, .embedding, .language_model
./gluonnlp/model/train/__init__.py:30:52: F405 'embedding' may be undefined, or defined from star imports: .cache, .embedding, .language_model
./gluonnlp/model/train/__init__.py:95:18: F405 'CacheCell' may be undefined, or defined from star imports: .cache, .embedding, .language_model
./gluonnlp/model/train/cache.py:27:1: E302 expected 2 blank lines, found 1
./gluonnlp/model/train/cache.py:84:51: E261 at least two spaces before inline comment
./gluonnlp/model/train/cache.py:100:5: E303 too many blank lines (2)
./gluonnlp/model/train/cache.py:100:91: E261 at least two spaces before inline comment
./gluonnlp/model/train/cache.py:100:101: E501 line too long (125 > 100 characters)
./gluonnlp/model/train/cache.py:130:17: E127 continuation line over-indented for visual indent
./gluonnlp/model/train/embedding.py:145:9: E265 block comment should start with '# '
./gluonnlp/model/train/embedding.py:204:9: E265 block comment should start with '# '
./gluonnlp/model/train/embedding.py:485:9: E265 block comment should start with '# '
./gluonnlp/model/train/language_model.py:98:79: E741 ambiguous variable name 'l'
./gluonnlp/model/train/language_model.py:100:44: E741 ambiguous variable name 'l'
./gluonnlp/model/train/language_model.py:120:49: E261 at least two spaces before inline comment
./gluonnlp/model/train/language_model.py:239:49: E261 at least two spaces before inline comment
./gluonnlp/vocab/__init__.py:23:1: F403 'from .vocab import *' used; unable to detect undefined names
./gluonnlp/vocab/__init__.py:24:1: F403 'from .subwords import *' used; unable to detect undefined names
./gluonnlp/vocab/__init__.py:26:11: F405 'vocab' may be undefined, or defined from star imports: .subwords, .vocab
./gluonnlp/vocab/__init__.py:26:27: F405 'subwords' may be undefined, or defined from star imports: .subwords, .vocab
./scripts/language_model/cache_language_model.py:47:46: E251 unexpected spaces around keyword / parameter equals
./scripts/language_model/cache_language_model.py:189:40: E225 missing whitespace around operator
./scripts/language_model/cache_language_model.py:208:52: E228 missing whitespace around modulo operator
./scripts/language_model/cache_language_model.py:209:47: E228 missing whitespace around modulo operator
./scripts/language_model/cache_language_model.py:210:34: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:54:46: E251 unexpected spaces around keyword / parameter equals
./scripts/language_model/word_language_model.py:237:70: E261 at least two spaces before inline comment
./scripts/language_model/word_language_model.py:237:101: E501 line too long (104 > 100 characters)
./scripts/language_model/word_language_model.py:239:9: E741 ambiguous variable name 'l'
./scripts/language_model/word_language_model.py:240:9: E741 ambiguous variable name 'l'
./scripts/language_model/word_language_model.py:241:9: E741 ambiguous variable name 'l'
./scripts/language_model/word_language_model.py:251:1: E302 expected 2 blank lines, found 1
./scripts/language_model/word_language_model.py:362:21: E741 ambiguous variable name 'l'
./scripts/language_model/word_language_model.py:378:24: E225 missing whitespace around operator
./scripts/language_model/word_language_model.py:388:53: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:392:76: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:400:50: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:405:63: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:410:12: E225 missing whitespace around operator
./scripts/language_model/word_language_model.py:419:52: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:420:47: E228 missing whitespace around modulo operator
./scripts/language_model/word_language_model.py:421:34: E228 missing whitespace around modulo operator
./scripts/nmt/bleu.py:204:13: E131 continuation line unaligned for hanging indent
./scripts/nmt/bleu.py:209:13: E131 continuation line unaligned for hanging indent
./scripts/nmt/encoder_decoder.py:115:17: E131 continuation line unaligned for hanging indent
./scripts/nmt/encoder_decoder.py:151:66: E262 inline comment should start with '# '
./scripts/nmt/encoder_decoder.py:171:65: E262 inline comment should start with '# '
./scripts/nmt/encoder_decoder.py:219:46: E262 inline comment should start with '# '
./scripts/nmt/encoder_decoder.py:238:45: E262 inline comment should start with '# '
./scripts/nmt/gnmt.py:140:65: E262 inline comment should start with '# '
./scripts/nmt/gnmt.py:140:101: E501 line too long (116 > 100 characters)
./scripts/nmt/gnmt.py:310:44: E261 at least two spaces before inline comment
./scripts/nmt/gnmt.py:310:45: E262 inline comment should start with '# '
./scripts/nmt/gnmt.py:337:45: E262 inline comment should start with '# '
./scripts/nmt/gnmt.py:349:55: E262 inline comment should start with '# '
./scripts/nmt/loss.py:31:60: E261 at least two spaces before inline comment
./scripts/nmt/loss.py:59:1: E302 expected 2 blank lines, found 1
./scripts/nmt/loss.py:129:53: E261 at least two spaces before inline comment
./scripts/nmt/train_gnmt.py:218:5: E731 do not assign a lambda expression, use a def
./scripts/nmt/train_gnmt.py:222:12: E127 continuation line over-indented for visual indent
./scripts/nmt/train_transformer.py:266:9: E731 do not assign a lambda expression, use a def
./scripts/nmt/train_transformer.py:270:9: E731 do not assign a lambda expression, use a def
./scripts/nmt/train_transformer.py:285:12: E127 continuation line over-indented for visual indent
./scripts/nmt/train_transformer.py:508:26: E127 continuation line over-indented for visual indent
./scripts/nmt/transformer.py:36:20: E127 continuation line over-indented for visual indent
./scripts/nmt/transformer.py:279:80: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:279:101: E501 line too long (111 > 100 characters)
./scripts/nmt/transformer.py:402:64: E261 at least two spaces before inline comment
./scripts/nmt/transformer.py:402:65: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:425:75: E261 at least two spaces before inline comment
./scripts/nmt/transformer.py:425:101: E501 line too long (109 > 100 characters)
./scripts/nmt/transformer.py:471:95: E261 at least two spaces before inline comment
./scripts/nmt/transformer.py:471:101: E501 line too long (129 > 100 characters)
./scripts/nmt/transformer.py:668:44: E261 at least two spaces before inline comment
./scripts/nmt/transformer.py:668:45: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:696:56: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:696:101: E501 line too long (107 > 100 characters)
./scripts/nmt/transformer.py:732:100: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:732:101: E501 line too long (138 > 100 characters)
./scripts/nmt/transformer.py:743:88: E262 inline comment should start with '# '
./scripts/nmt/transformer.py:743:101: E501 line too long (120 > 100 characters)
./scripts/nmt/translation.py:190:90: E262 inline comment should start with '# '
./scripts/nmt/translation.py:190:101: E501 line too long (122 > 100 characters)
./scripts/nmt/translation.py:211:89: E262 inline comment should start with '# '
./scripts/nmt/translation.py:211:101: E501 line too long (121 > 100 characters)
./scripts/question_answering/data_processing.py:89:16: E127 continuation line over-indented for visual indent
./scripts/sentiment_analysis/sentiment_analysis.py:99:1: E302 expected 2 blank lines, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:105:53: E261 at least two spaces before inline comment
./scripts/sentiment_analysis/sentiment_analysis.py:135:43: E261 at least two spaces before inline comment
./scripts/sentiment_analysis/sentiment_analysis.py:141:1: E305 expected 2 blank lines after class or function definition, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:160:1: E302 expected 2 blank lines, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:164:1: E305 expected 2 blank lines after class or function definition, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:169:1: E302 expected 2 blank lines, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:179:1: E305 expected 2 blank lines after class or function definition, found 1
./scripts/sentiment_analysis/sentiment_analysis.py:337:57: E228 missing whitespace around modulo operator
./scripts/sentiment_analysis/sentiment_analysis.py:338:45: E228 missing whitespace around modulo operator
./scripts/sentiment_analysis/sentiment_analysis.py:339:34: E228 missing whitespace around modulo operator
./scripts/tests/test_bleu.py:145:101: E501 line too long (104 > 100 characters)
./scripts/tests/test_bleu.py:147:101: E501 line too long (111 > 100 characters)
./scripts/tests/test_bleu.py:151:1: W391 blank line at end of file
./scripts/tests/test_encoder_decoder.py:4:1: F403 'from ..nmt.gnmt import *' used; unable to detect undefined names
./scripts/tests/test_encoder_decoder.py:5:1: F403 'from ..nmt.transformer import *' used; unable to detect undefined names
./scripts/tests/test_encoder_decoder.py:13:27: F405 'GNMTEncoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:21:101: E501 line too long (105 > 100 characters)
./scripts/tests/test_encoder_decoder.py:23:101: E501 line too long (101 > 100 characters)
./scripts/tests/test_encoder_decoder.py:30:101: E501 line too long (102 > 100 characters)
./scripts/tests/test_encoder_decoder.py:38:15: F405 'GNMTEncoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:44:23: F405 'GNMTDecoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:44:101: E501 line too long (102 > 100 characters)
./scripts/tests/test_encoder_decoder.py:45:101: E501 line too long (119 > 100 characters)
./scripts/tests/test_encoder_decoder.py:50:101: E501 line too long (106 > 100 characters)
./scripts/tests/test_encoder_decoder.py:51:101: E501 line too long (106 > 100 characters)
./scripts/tests/test_encoder_decoder.py:52:101: E501 line too long (120 > 100 characters)
./scripts/tests/test_encoder_decoder.py:53:101: E501 line too long (120 > 100 characters)
./scripts/tests/test_encoder_decoder.py:57:101: E501 line too long (106 > 100 characters)
./scripts/tests/test_encoder_decoder.py:83:1: E302 expected 2 blank lines, found 1
./scripts/tests/test_encoder_decoder.py:88:27: F405 'TransformerEncoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:91:101: E501 line too long (110 > 100 characters)
./scripts/tests/test_encoder_decoder.py:96:101: E501 line too long (106 > 100 characters)
./scripts/tests/test_encoder_decoder.py:98:101: E501 line too long (101 > 100 characters)
./scripts/tests/test_encoder_decoder.py:99:101: E501 line too long (110 > 100 characters)
./scripts/tests/test_encoder_decoder.py:105:101: E501 line too long (102 > 100 characters)
./scripts/tests/test_encoder_decoder.py:117:55: E128 continuation line under-indented for visual indent
./scripts/tests/test_encoder_decoder.py:118:30: W291 trailing whitespace
./scripts/tests/test_encoder_decoder.py:121:1: E302 expected 2 blank lines, found 1
./scripts/tests/test_encoder_decoder.py:124:15: F405 'TransformerEncoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:124:101: E501 line too long (103 > 100 characters)
./scripts/tests/test_encoder_decoder.py:130:23: F405 'TransformerDecoder' may be undefined, or defined from star imports: ..nmt.gnmt, ..nmt.transformer
./scripts/tests/test_encoder_decoder.py:130:101: E501 line too long (124 > 100 characters)
./scripts/tests/test_encoder_decoder.py:131:101: E501 line too long (133 > 100 characters)
./scripts/tests/test_encoder_decoder.py:136:101: E501 line too long (110 > 100 characters)
./scripts/tests/test_encoder_decoder.py:137:101: E501 line too long (110 > 100 characters)
./scripts/tests/test_encoder_decoder.py:138:101: E501 line too long (120 > 100 characters)
./scripts/tests/test_encoder_decoder.py:139:101: E501 line too long (120 > 100 characters)
./scripts/tests/test_encoder_decoder.py:143:101: E501 line too long (106 > 100 characters)
./scripts/tests/test_encoder_decoder.py:158:101: E501 line too long (102 > 100 characters)
./scripts/tests/test_encoder_decoder.py:168:1: W391 blank line at end of file
./scripts/tests/test_question_answering.py:19:1: F401 'os' imported but unused
./scripts/word_embeddings/evaluation.py:65:5: E266 too many leading '#' for block comment
./scripts/word_embeddings/train_fasttext.py:44:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:45:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:46:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:47:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:48:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:49:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:50:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:51:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:53:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:54:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:56:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:57:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:58:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:59:1: E402 module level import not at top of file
./scripts/word_embeddings/train_fasttext.py:179:76: E502 the backslash is redundant between brackets
./scripts/word_embeddings/train_fasttext.py:349:77: E251 unexpected spaces around keyword / parameter equals
./tests/unittest/test_attention_cell.py:2:1: F403 'from gluonnlp.model.attention_cell import *' used; unable to detect undefined names
./tests/unittest/test_attention_cell.py:58:20: F405 'MLPAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:60:20: F405 'MLPAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:69:28: F405 'DotProductAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:76:28: F405 'DotProductAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:78:28: F405 'DotProductAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:89:28: F405 'MultiHeadAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_attention_cell.py:90:35: F405 'DotProductAttentionCell' may be undefined, or defined from star imports: gluonnlp.model.attention_cell
./tests/unittest/test_batchify.py:6:1: F401 'pytest' imported but unused
./tests/unittest/test_batchify.py:8:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_batchify.py:9:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_batchify.py:12:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_batchify.py:47:101: E501 line too long (136 > 100 characters)
./tests/unittest/test_batchify.py:57:101: E501 line too long (115 > 100 characters)
./tests/unittest/test_batchify.py:66:101: E501 line too long (103 > 100 characters)
./tests/unittest/test_batchify.py:67:101: E501 line too long (111 > 100 characters)
./tests/unittest/test_batchify.py:71:101: E501 line too long (113 > 100 characters)
./tests/unittest/test_batchify.py:76:101: E501 line too long (152 > 100 characters)
./tests/unittest/test_batchify.py:79:101: E501 line too long (101 > 100 characters)
./tests/unittest/test_batchify.py:80:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_batchify.py:88:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_batchify.py:98:101: E501 line too long (126 > 100 characters)
./tests/unittest/test_beam_search.py:139:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_beam_search.py:145:5: E306 expected 1 blank line before a nested definition, found 0
./tests/unittest/test_beam_search.py:160:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_beam_search.py:176:44: E127 continuation line over-indented for visual indent
./tests/unittest/test_beam_search.py:176:101: E501 line too long (101 > 100 characters)
./tests/unittest/test_beam_search.py:177:21: E127 continuation line over-indented for visual indent
./tests/unittest/test_beam_search.py:178:101: E501 line too long (101 > 100 characters)
./tests/unittest/test_beam_search.py:191:101: E501 line too long (105 > 100 characters)
./tests/unittest/test_beam_search.py:238:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_datasets.py:38:24: F821 undefined name 'unicode'
./tests/unittest/test_datasets.py:74:5: E303 too many blank lines (2)
./tests/unittest/test_datasets.py:79:5: F841 local variable 'my_dataset' is assigned to but never used
./tests/unittest/test_datasets.py:92:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_datasets.py:120:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_datasets.py:463:57: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:468:66: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:476:60: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:481:69: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:482:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_datasets.py:492:57: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:497:66: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:505:101: E501 line too long (101 > 100 characters)
./tests/unittest/test_datasets.py:513:60: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:518:69: E225 missing whitespace around operator
./tests/unittest/test_datasets.py:519:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_datasets.py:529:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_datasets.py:536:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_datasets.py:540:5: F841 local variable 'train' is assigned to but never used
./tests/unittest/test_datasets.py:541:5: F841 local variable 'val' is assigned to but never used
./tests/unittest/test_datasets.py:542:5: F841 local variable 'test' is assigned to but never used
./tests/unittest/test_datasets.py:555:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_datasets.py:559:5: F841 local variable 'train' is assigned to but never used
./tests/unittest/test_datasets.py:560:5: F841 local variable 'val' is assigned to but never used
./tests/unittest/test_datasets.py:561:5: F841 local variable 'test' is assigned to but never used
./tests/unittest/test_datasets.py:568:5: F841 local variable 'padding_idx' is assigned to but never used
./tests/unittest/test_datasets.py:579:5: F841 local variable 'num_batches' is assigned to but never used
./tests/unittest/test_datasets.py:584:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_datasets.py:588:5: F841 local variable 'test' is assigned to but never used
./tests/unittest/test_datasets.py:598:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_datasets.py:611:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_datasets.py:618:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_loss.py:21:1: F401 'mxnet.gluon' imported but unused
./tests/unittest/test_loss.py:24:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_loss.py:30:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_models.py:30:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_models.py:37:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_models.py:41:101: E501 line too long (130 > 100 characters)
./tests/unittest/test_models.py:42:101: E501 line too long (207 > 100 characters)
./tests/unittest/test_models.py:48:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_models.py:56:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_models.py:58:20: E128 continuation line under-indented for visual indent
./tests/unittest/test_models.py:65:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_models.py:78:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_models.py:79:101: E501 line too long (104 > 100 characters)
./tests/unittest/test_models.py:80:101: E501 line too long (106 > 100 characters)
./tests/unittest/test_models.py:97:101: E501 line too long (106 > 100 characters)
./tests/unittest/test_models.py:99:101: E501 line too long (106 > 100 characters)
./tests/unittest/test_models.py:109:20: E128 continuation line under-indented for visual indent
./tests/unittest/test_models.py:114:44: E127 continuation line over-indented for visual indent
./tests/unittest/test_sampler.py:5:1: F401 'gluonnlp as nlp' imported but unused
./tests/unittest/test_sampler.py:21:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_sampler.py:25:101: E501 line too long (101 > 100 characters)
./tests/unittest/test_sampler.py:47:101: E501 line too long (102 > 100 characters)
./tests/unittest/test_sampler.py:48:101: E501 line too long (127 > 100 characters)
./tests/unittest/test_transforms.py:22:1: F403 'from gluonnlp.data.transforms import *' used; unable to detect undefined names
./tests/unittest/test_transforms.py:31:20: F405 'ClipSequence' may be undefined, or defined from star imports: gluonnlp.data.transforms
./tests/unittest/test_transforms.py:57:27: F405 'PadSequence' may be undefined, or defined from star imports: gluonnlp.data.transforms
./tests/unittest/test_transforms.py:72:17: F405 'NLTKMosesTokenizer' may be undefined, or defined from star imports: gluonnlp.data.transforms
./tests/unittest/test_transforms.py:84:17: F405 'SpacyTokenizer' may be undefined, or defined from star imports: gluonnlp.data.transforms
./tests/unittest/test_transforms.py:96:19: F405 'NLTKMosesDetokenizer' may be undefined, or defined from star imports: gluonnlp.data.transforms
./tests/unittest/test_vocab_embed.py:32:1: F403 'from mxnet.test_utils import *' used; unable to detect undefined names
./tests/unittest/test_vocab_embed.py:38:24: F405 'unicode' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:48:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_vocab_embed.py:51:1: E302 expected 2 blank lines, found 1
./tests/unittest/test_vocab_embed.py:67:36: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:73:30: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:121:57: E231 missing whitespace after ','
./tests/unittest/test_vocab_embed.py:135:26: E231 missing whitespace after ','
./tests/unittest/test_vocab_embed.py:218:15: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:221:15: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:224:15: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:394:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:400:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:401:24: E128 continuation line under-indented for visual indent
./tests/unittest/test_vocab_embed.py:404:24: E124 closing bracket does not match visual indentation
./tests/unittest/test_vocab_embed.py:408:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:411:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:416:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:419:9: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:429:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:435:101: E501 line too long (106 > 100 characters)
./tests/unittest/test_vocab_embed.py:437:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:439:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:441:101: E501 line too long (107 > 100 characters)
./tests/unittest/test_vocab_embed.py:443:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:445:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:507:101: E501 line too long (105 > 100 characters)
./tests/unittest/test_vocab_embed.py:513:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:521:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:529:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:532:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:536:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:539:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:543:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:550:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:554:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:559:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:571:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:580:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:589:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:601:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:646:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:655:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:659:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:667:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:694:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:712:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:730:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:750:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:783:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:784:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils
./tests/unittest/test_vocab_embed.py:785:5: F405 'assert_almost_equal' may be undefined, or defined from star imports: mxnet.test_utils

reproducing AWD model

The training script that reproduces the AWD results in https://github.com/dmlc/gluon-nlp/tree/master/scripts/language_model is using SGD. But actually the AWD model stand for ASGD Weight-Dropped LSTM, in which ASGD is the average sgd.

Environment variable for configuring the default downloading location

Currently all models/datasets will be downloaded to ~/.mxnet/... by default. We should add an environment variable to configure the constant. Sometimes the user may not have enough space under ~ and will need to download the dataset to another location.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.