• 列表汇总

激活函数

• 1606.08415 Gaussian Error Linear Units (GELUs) 高斯误差线性单位(GELUs)

正则化

损失函数

• [1911.02855 Dice Loss for Data-imbalanced NLP Tasks]

• 201801 AMSoftmax: Additive Margin Softmax for Face Verification

• 201708 Focal Loss for Dense Object Detection

• 列表汇总

• 列表汇总

• 2002.12620 TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing

• 202003 Natural Language Processing Advancements By Deep Learning: A Survey
  • https://arxiv.org/abs/2003.01200

• 列表汇总

• 201911 Generalization through Memorization: Nearest Neighbor Language Models

• 列表汇总

• 2103.12407 Detecting Hate Speech with GPT-3 用 GPT-3检测仇恨言论

• 202004 Deep Learning Based Text Classification: A Comprehensive Review

• 2014 Convolutional Neural Networks for Sentence Classification

• 201412 Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

• 2016 Hierarchical Attention Networks for Document Classification

• 2016 Recurrent Neural Network for Text Classification with Multi-Task Learning

• 201607 Bag of Tricks for Efficient Text Classification

• 2017ACL Deep Pyramid Convolutional Neural Networks for Text Categorization

• 2018 Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN

• Baselines and Bigrams: Simple, Good Sentiment and Topic Classification

• 列表汇总

序列标注

• 201908 - LAN - Hierarchically-Refined Label Attention Network for Sequence Labeling

分词

• 201704 Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF
• 2019ACL Is Word Segmentation Necessary for Deep Learning of Chinese Representations?

命名实体识别

• 2101.00396 Lex-BERT: Enhancing BERT based NER with lexicons Lex-BERT: 用词典增强基于 BERT 的 NER
• 2101.11420 Recent Trends in Named Entity Recognition (NER) 命名实体识别(NER)的最新发展趋势
• 202002 Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
• 202002 Zero-Resource Cross-Domain Named Entity Recognition
• 201812 A Survey on Deep Learning for Named Entity Recognition

指代消解

语义角色

依存分析

NLU

• 2012.15639 TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis 一个用于细粒度 NER 和增强语义分析的文本理解系统

• 列表汇总

词向量

• 2014 Glove: Global Vectors for Word Representation
• 2013 Efficient Estimation of Word Representations in Vector Space
• 2013 Distributed Representations of Words and Phrases and their Compositionality

句向量

• 2015 From Word Embeddings To Document Distances
• 2014 Doc2Vec Distributed Representations of Sentences and Documents

• 列表汇总

• 201908 Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

• 2019 Keyword-Attentive Deep Semantic Matching

• 201908 ACL RE2: Simple and Effective Text Matching with Richer Alignment Features

• 201905 FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance

• 201904 Understanding the Behaviors of BERT in Ranking

• 201804 SAN - Stochastic Answer Networks for Natural Language Inference

• 201609 ESIM Enhanced LSTM for Natural Language Inference

• 2013 DSSM Learning Deep Structured Semantic Models for Web Search using Clickthrough Data

• 201606 A Decomposable Attention Model for Natural Language Inference

• 列表汇总

• 202004 When does data augmentation help generalization in NLP?

• 1904.12848 Unsupervised Data Augmentation 无监督数据增强

• 201901 EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification

• 列表汇总

• 202005 Cross-lingual Transfer of Twitter Sentiment Models Using a Common Vector Space

• 列表汇总

Review

• 2009.10229 An Empirical Study on Neural Keyphrase Generation 神经关键词生成的实证研究

• 201905 A review of keyphrase extraction

Unsupervised

• 2004.13639 Joint Keyphrase Chunking and Salience Ranking with BERT 基于BERT联合关键词分组和显著性排序

• 202002 SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-Trained Language Model 基于预训练语言模型的无监督关键词提取基线 SIFRank

• 2018 ACL EmbedRank++ Simple Unsupervised Keyphrase Extraction using Sentence Embeddings

Supervised

• 列表汇总

General

• 201907 Simple Unsupervised Summarization by Contextual Matching

Extractive

• 201903 Fine-tune BERT for Extractive Summarization
• 2019ACL Single Document Summarization as Tree Induction

Abstractive

• 列表汇总

Task-oriented

• 202003 Recent Advances and Challenges in Task-oriented Dialog System

Dialogue

• 201911 DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

意图识别与槽抽取

• 列表

• 2103.15871 Industry Scale Semi-Supervised Learning for Natural Language Understanding 自然语言理解的工业规模半监督学习

• 201902 BERT for Joint Intent Classification and Slot Filling

• 列表汇总

• 列表汇总

Entity Linking

Relation Extraction

• 2102.01373 An Improved Baseline for Sentence-level Relation Extraction 一种改进的句级关系抽取基线

• 2102.01156 Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings 基于 bert 标记和实例嵌入的远监督关系抽取

• 2010.12812 A Frustratingly Easy Approach for Entity and Relation Extraction 一种简单得令人沮丧的实体和关系抽取方法

• 202004 Review and Outlook for Relation Extraction

• 1711.07010 A Discourse-Level Named Entity Recognition and Relation Extraction Dataset for Chinese Literature Text 中文文学作品命名实体识别与关系抽取数据集

Event Extraction

• 列表汇总

• 201409 Neural Machine Translation by Jointly Learning to Align and Translate

• 201409 Sequence to Sequence Learning with Neural Networks

• 列表汇总

Chatbot

• 202004 Few-Shot Natural Language Generation by Rewriting Templates

诗歌

• 2019 基于神经网络的集句诗自动生成

新闻评论

• 201909 Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation
• 201906 Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
• 201809 Unsupervised Machine Commenting with Neural Variational Topic Model
• 201805 Automatic Article Commenting: the Task and Dataset

评论评价

Pretrain ML

• 201909 A Conditional Transformer Language Model for Controllable Generation

• 列表汇总

• 列表汇总

• 2103.06874 CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation

• 2103.04350 Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

模型

• 2103.10360 All NLP Tasks Are Generation Tasks: A General Pretraining Framework 所有 NLP 任务都是生成任务: 一个通用的预训练框架

• 2103.04350 Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

• 2103.06874 CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation

• 202004 DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

• 202004 Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

• 202003 Pre-trained Models for Natural Language Processing: A Survey

• 2020 ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

• 202001 ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

• 201909 ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

• 201909 NEZHA: NEURAL CONTEXTUALIZED REPRESENTATION FOR CHINESE LANGUAGE UNDERSTANDING

• 201907 ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding

• 201906 RoBERTa: A Robustly Optimized BERT Pretraining Approach

• 201906 ERNIE: Enhanced Language Representation with Informative Entities（THU/ACL2019）

• 1905.03197 Unified Language Model Pre-training for Natural Language Understanding and Generation

• 201904 ERNIE: Enhanced Representation through Knowledge Integration

• 2018 ACL Deep contextualized word representations: ELMO

• 2018 GPT Improving Language Understanding by Generative Pre-Training

• 201801 Universal Language Model Fine-tuning for Text Classification

• 201706 attention is all you need

蒸馏

• 1909.10351 TinyBERT: Distilling BERT for Natural Language Understanding
• 1910.01108 DistilBERT,adistilledversionofBERT:smaller, faster,cheaperandlighter
• 1908.08962 Well-Read Students Learn Better: On the Importance of Pre-training Compact Models

应用

• 2101.10642 Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks BERT 和 ALBERT 句子嵌入对下游自然语言处理任务的影响
• 2010.05522 Pre-trained Language Model Based Active Learning for Sentence Matching 基于预训练语言模型的主动学习句子匹配
• 2103.10385 GPT Understands, Too GPT 也能理解
• 2103.10673 Cost-effective Deployment of BERT Models in Serverless Environment 无服务环境下 BERT 模型的性价比部署
• 2004.02288 Continual Domain-Tuning for Pretrained Language Models 预训练语言模型的持续域微调
• 202004 Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning
• 202002 REALM: Retrieval-Augmented Language Model Pre-Training

情感分类

• 2103.07098 A Weakly Supervised Approach for Classifying Stance in Twitter Replies

• 202005 SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis

原理

• 202003 What the MASK? Making Sense of Language-Specific BERT Models
• 202002 A Primer in BERTology: What we know about how BERT works
• 201909 How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
• 201910 T5 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

• 列表汇总

综述

开放领域

KBQA

• 2104.00218 Integrating Subgraph-aware Relation and Direction Reasoning for Question Answering 基于子图感知关系和方向推理的问答系统

• 列表汇总

概念挖掘

模型

• 202011 Graph Neural Networks in Recommender Systems: A Survey

• 1703.04247 DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

• 201606 Wide & Deep Learning for Recommender Systems

• 1601.02376 Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

• 2010 Factorization Machines

新闻推荐

• 1801.08284 DKN: Deep Knowledge-Aware Network for News Recommendation

• KDD2017 Embedding-based News Recommendation for Millions of Users