Code Monkey home page Code Monkey logo

nlp_overview's Introduction

Modern Deep Learning Techniques Applied to Natural Language Processing

This project contains an overview of recent trends in deep learning based natural language processing (NLP). It covers the theoretical descriptions and implementation details behind deep learning models, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and reinforcement learning, used to solve various NLP tasks and applications. The overview also contains a summary of state of the art results for NLP tasks such as machine translation, question answering, and dialogue systems. You can find the learning resource at the following address: https://nlpoverview.com/. A snapshot of the website is provided below:

alt txt

About this project

The main motivations for this project are as follows:

  • Maintain an up-to-date learning resource that integrates important information related to NLP research, such as:
    • state of the art results
    • emerging concepts and applications
    • new benchmark datasets
    • code/dataset releases
    • etc.
  • Create a friendly and open resource to help guide researchers and anyone interested to learn about modern techniques applied to NLP
  • A collaborative project where expert researchers can suggest changes (e.g., incorporate SOTA results) based on their recent findings and experimental results

Table of Contents

How to Contribute?

There are various ways to contribute to this project.

  • The quickest way to propose an edit or add text is as follows: fork the repo, browse to the corresponding chapter, and then click on edit button to add your info. The image below shows the last two steps after you have forked the repo. You can then submit a pull request and we will approve accordingly. If you would like to change a huge portion of the project or even add a chapter, then we recommend looking at the "Build site locally" section below.

alt txt

  • You can also propose text additions in this public shared document if you are not familiar with git. We will help edit and revise the content and then further assist you to incorporate the contributions to the project.
  • Refer to the issue section to learn more about other ways you can help.
  • Or you can make suggestions by submitting a new issue. More detailed instructions coming soon.

Build site locally

If you are planning to change some aspect of the site (e.g., adding section or style) and want to preview it locally on your machine, we suggest you to build and run the site locally using jekyll. Here are the instructions:

  • First, check that Ruby 2.1.0 or higher is installed on your computer. You can check using the ruby --version command. If not, please install it using the instructions provided here.
  • After ensuring that Ruby is installed, install Bundler using gem install bundler.
  • Clone this repo locally: git clone https://github.com/omarsar/nlp_overview.git
  • Navigate to the repo folder with cd nlp_overview
  • Install Jekyll: bundle install
  • Run the Jekyll site locally: bundle exec jekyll serve
  • Preview site on the browser at http://localhost:4000

Maintenance

This project is maintained by Elvis Saravia and Soujanya Poria. You can also find me on Twitter if you have any direct comments or questions. A major part of this project have been directly borrowed from the work of Young et al. (2017). We are thankful to the authors.

nlp_overview's People

Contributors

ar7z1 avatar bayethiernodiop avatar dmelancon avatar jabalazs avatar lxdatgithub avatar omarsar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nlp_overview's Issues

Include papers on bias in word embeddings

Word embeddings contain bias. Ethics in general is an active area of research and is worth mentioning.

One of the seminal papers:

  • Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
    (Bolukbasi et al., NIPS 2016) (pdf)

Other interesting papers on the subject:

  • Text Embedding Models Contain Bias. Here's Why That Matters. (Packer et al., Google 2018)
  • Measuring and Mitigating Unintended Bias in Text Classification (Dixon et al., AIES 2018)

PR Ideas!!!

Finding ideas on how to contribute to this repository? Consider the following:

  • Propose and design better segmentation of chapters and subsections (currently we only have chapters)
  • Convert the tables into markdown (currently they are in html format)
  • Help with the website design
  • Help to check for bugs or grammatical errors, missing links, etc.
  • Translate into your language
  • Make suggestions on how to integrate with other useful resources such as Sebastian Ruder's NLP Progress repo

Fancy PRs:

  • Include code blocks using any of of the common deep learning frameworks
  • Re-write some sections, suggest new sections, include implementation details, SOTA results, etc.
  • Incorporate descriptions and results of recent NLP models such as FastText, BERT, ULMFit, ELMo, etc.

Can't find the equation labels + typos

For example, in the 'D. Contextualized Word Embeddings' session, you wrote 'In Eq. 8, s_{j}^{task} ...'. Although we could figure it out from the context. :)

Printable version

This repo is a great work!

The current version is almost impossible to print with a decent quality.

If Ctrl+P from the HTML version the output PDF gets messed up.

Is it possible to have a decent printable version? Is it planned?

Thanks!

Small translation error for "Bank" in "B. Word2Vec"

In section B. Word2Vec it says "...when translated to French provides two different words: banc and banque representing financial and geographical meanings, respectively": the two french words are reversed. Banque is financial and banc geographical.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.