keon / awesome-nlp Goto Github PK

Develop models for Hindi from the IIT Bombay English Hindi Parallel Corpus using Cython/spaCy-multilingual.

This is a blogpost and code from DennyBritz's project. Should I add it?

As you mention here the video are not available and I go through the link it's not there so it would be better to update with this link

I think one should add "fasttext" to list.
It is new library developed by FAIR team together with Thomas Mikolov.
It can do everything word2vec can, but faster. It also provides more features: model compression, supervised classification, character n-grams.

workin on it

We need lists for books

The paper link is here.

P.S. There might arise the issue of finding a suitable Hindi dataset.

German NLP is a curated list of German NLP resources including datasets.

P.S. This paper is one of the first to work on Embed, encode, attend, predict.

@keon Thank you.

Hi @keon ,This is an awesome project.I hope more people can see .
I have completed the translation of Traditional Chinese and I have made a PR.
May you add the Traditional Chinese version on this repo?

We can set up a website at something like www.keon.github.io/awesome-nlp

Reference Demo: www.nirantk.github.io/awesome-project-ideas
which auto-updates from the Github repository at https://github.com/NirantK/awesome-project-ideas

The website would automatically pick up content and get up updated from the Markdown on our master/README.md.

This should make it easier for search engines to rank and find us. It would also make it easier for some people e.g. academics, beginners like college students etc.

@keon thoughts?

Create new category Text Summarization in Techniques

• Add at least 2 relevant papers such as those based on attention
• Add at least 2 relevant code implementations such as textsum and PyTextRank
• Add at least 1 tutorial and review blog in relevant sections from gensim and others

Can we add research topic specific section? For example - What are the latest research in Question Answering?

https://paperswithcode.com/area/natural-language-processing

Awesome.re is a meta-list of awesome repositories.

It has a few guidelines to be included there.
Maybe we can use that as our checklist to improve this. And then raise a PR when we are done?

What do you suggest @keon ?

Machine Translation has explored CNNs, made encode-decoder architectures a norm and other changes since the last update. Please add recent papers, demos, tutorials and 1-2 line explanations for each link.

If you know of better content, please add to this space.

To become the go-to resource, we should be able to curate good tools and datasets in at least the following languages (in addition to English):

• Chinese
• Korean
• Japanese

Of course, we are willing to accept PRs from all other languages.

Please feel free to raise a PR or simply comment on this issue itself and we will add it on your behalf.

@keon

1. Thoughts? Is this a good direction to take?

2. Can you take up any of the languages? Sorry for assuming your Asian heritage, the South Korean flag is on your Github account

I found a whole section for Thai resources in NLP. Here's the link
@NirantK shall I work on it and open a PR?

@NirantK I thought I would check before submitting another PR – would the following NLP tool fit the list?

https://github.com/amir-zeldes/RFTokenizer

It is a trainable subword tokenizer for morphologically rich languages, such as Afro-Asiatic languages. It comes with pre-trained models for Arabic, Hebrew and Coptic, supports Python 2-3 and is installable from PyPI. Current performance is SOA on this task at least for Hebrew and Coptic (not sure about Arabic, since different papers seem to use different targets and metrics).

Deep Natural Language Processing

I would like to propose a channel like Slack or Gitter for chat for the following reasons:

• Issue discussion can be done more easily
• Enhancement suggestions can be discussed more openly

Would like to hear about it from other maintainers- @keon, @outpark and @NirantK

Indic

• Hindi - To be done later
• Gujarati - To be done later
• Tamil
• Telegu
• Bengali

Asian

• Chinese
• Korean
• Japanese

We should be able to add content regarding Indian/Indic languages as well, keeping in mind the growth of India Stack and need for Indic tools.

Do you think we should add a MIT or CC0 license to the repository?

This would

• make it explicit that we (the maintainers) are not liable
• allow developers working for corporations to use the listing here more freely
  (not specifying a license makes corporate legal itchy)

awesome meta-list recommends CC0 probably because it is more permissive than MIT, AGPL and the like.

What do you suggest?

Hello @keon,

Just wanted to check if we could add @the-ethan-hunt as a collaborator?

It will allow faster contributions.

He has made ample contributions imho, refer author contributions

https://github.com/stanfordnlp/stanfordnlp

We need to add Tensorflow and Torch implementations of various models.
P.S. As suggested by Sebastian Ruder in his tweet.

I will be working on this, but first-time contributors are welcome to contribute too!

Considering the recent surge in NLP research, I propose the listing of some of the top labs that are carrying cutting-edge research in NLP (no endorsements, here) like Stanford NLP group, University of Edinburgh NLP group etc.

@NirantK how does this sound?

The Single Exchange Dialogs section is ambiguous, too broad and out of date. Here is how you can help us improve this:

1. Removing links which you think do not fit in the section. Don't worry about damaging this repository, we can have a discussion on the PR you raise.
2. Consider adding 2-3 code examples and datasets
3. Consider adding 2-3 examples from slot filling (sequence mining for text is welcome too) and other approaches in chatbots

Here is the academic work I could find:

1. Clustering Algorithm for Gujarati words, 2012
2. Gujarati-Hindi Machine Translation, 2013
3. Lightweight Stemmer for Gujarati, 2013

None of the above work is inspiring but this is what we have.

Hi @keon

I'm the author of Curated Papers a website solely designed for managing, curating and interacting with curated lists of academic papers, projects, links, etc. I saw your list and thought it could be a great addition to the website, are you willing to give it a try? Currently CP is pre-launched but it is available online and I'm working on adding high quality content.

Word Forms: Accurately generate all possible forms of an English word

• Python Library
• Github Link - https://github.com/gutfeeling/word_forms

some links can be found here
https://github.com/niderhoff/nlp-datasets

There exists a similar task that is named text classification.

But I want to find a kind of model that the inputs are keyword set. And the keyword set is not from a sentence.

Thank you.

Since the languages section is a nested list, the references to individual languages are currently useless. AFAIF only headers can be linked to.

Great resource, thanks for creating this! I noticed that you don't have any Ruby libraries listed. I maintain a list of Ruby NLP libraries that you could link to if you like: https://github.com/diasks2/ruby-nlp

A Neural Network Approach to Context-Sensitive Generation of Conversational Responses Sordoni 2015. Generates responses to tweets. Uses Recurrent Neural Network Language Model (RLM) architecture of (Mikolov et al., 2010). source code: RNNLM Toolkit

The toolkit source code link is broken

I think it is a good idea to add more maintainers so we can keep up to date.
We are also going to polish the list. Please let me know if anyone is interested.

@NirantK - Are you interested?

Hello @keon,

Github topics allow Github beginners to "discover" us. It also makes it possible for us to feature in the Discover Dashboard.

Since only owners can add Github topics, can you please add some topics? Here are a few suggestions:

1. awesome-list
2. awesome
3. natural-language-processing
4. text-mining
5. deep-learning or machine-learning

Adding a quick screenshot to help you find where to add Github topics:

I haven't tried it.

https://github.com/chakki-works/chazutsu

http://www.cs.sfu.ca/~anoop/teaching/CMPT-413-Spring-2014/

Standardize all links into some schematic/syntax. Here are few suggestions:

• The link and description are separated by a dash.
  Example: [AVA](…) - JavaScript test runner
• Section description starts with an uppercase character and ends with a period.
• Drop all the A / An prefixes in the descriptions. Don't begin sentences with This, here or that
• Consistent and correct naming. For example, Node.js, not NodeJS or node.js.
• Do not end bullet points of links with . or full stops, this makes it easier to read

We can then consider setting up a CI like Circle CI or Travis and run this linter on every PR automatically.

This linter would be incredibly useful beyond awesome-nlp to several other awesome-* repositories. The value created from your effort must be seen in that context.

Link for the University of Pennsylvania research group under prominent NLP research groups seems to be broken

I found a huge collection for NLP resources in Hebrew at here and here.

@NirantK , should I submit a PR for the section?

These days, a lot of programs have been initiated by technical organizations to get students acquainted with open source. One such program is the Kharagpur Winter of Code held by KOSS, IIT Kharagpur in the month of December since last two years. I understand that awesome-NLP is a curated list but I believe we can benefit greatly from this program. I would like to hear thoughts about this from @NirantK and @keon . In case there is no available time, I can act as a mentor. 😄

We need to find and add a banner image similar to awesome-qauntified-self or awesome-electron

This is towards the broader goal of making this list more polished. And raise a PR to awesome.re

Requests to anyone sending a PR:

• Either fullwidth or placed at the top-right of the readme. Example here
• Image should link to the Github URL of the project
• Image should be high-DPI. Set it to maximum half the width of the original image.

Any support for Gujarati? I think you are from Surat hence thought some support might be there for Gujarati as well

papers should be unified in one format with author names included.