Yonsei NLP Study Season 2

[About Yonsei NLP Study]
Yonsei NLP Study is consisted of 4 Students who are very passionate in the field of Natural Language Processing!
Every week, each student read papers regarding a certain subject.
Then, each student gets to present about the topic he prepared for.
After the presentation, all the students discuss about that paper and share ideas.

[About Season 2]
Yonsei NLP Study Season 2 (2021.09.07 ~ 2021.11.17)
Main Topic : Question Answering and Various Techniques needed to solve QA Tasks
Among the 4 members, Seungone Kim and Sejune Joo participated in this topic!

[About this repository]
This repository contains presentation materials, links to presentation videos, and a summary of all the papers we have studied in Yonsei NLP Study Season2(2021.09.08~2021.11.17). Our Main Topic was Question Answering and the various techniques needed to solve QA. We have covered 55 papers in total!

About the Members

• Seungone Kim(김승원) - Leader!
• Sejune Joo(주세준)

Link to Presentation(Youtube)

• Taxonomy of QA Datasets and Tasks (Seungone Kim)
• Information Retrieval (Seungone Kim)
• Knowledge Graph & Commonsense Knowledge Graph (Seungone Kim)
• Knowledge Injection & Domain Adaptation (Seungone Kim)

List of Papers we covered

The following are the list of 55 papers

TALK 1 : Question Answering Tasks & Datasets (Seungone Kim)

• Wikiqa : A challenge dataset for open-domain question answering
• SQuAD : 100,000+ Questions for Machine Comprehension of Text
• MS MARCO : A Human Generated MAchine Reading COmprehension Dataset
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• Do We Know What We Don’t Know? Studying Unanswerable Questions beyond SQuAD 2.0
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• The NarrativeQA Reading Comprehension Challenge
• Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
• Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• ELI5: Long Form Question Answering
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• CoQA: A Conversational Question Answering Challenge
• PubMedQA: A Dataset for Biomedical Research Question Answering
• ANTIQUE: A Non-Factoid Question Answering Benchmark
• ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
• WinoGrande: An Adversarial Winograd Schema Challenge at Scale
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning

TALK 1 : Machine Reading Comprehension (Seungone Kim & Sejune Joo)

• Bidirectional Attention Flow for Machine Comprehension
• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
• SG-Net: Syntax-Guided Machine Reading Comprehension
• Retrospective Reader for Machine Reading Comprehension

TALK 2 : Information Retrieval (Seungone Kim)

• ColBERT : Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• RepBERT : Contextualized Text Embeddings for First-Stage Retrieval
• Dense Passage Retrieval for Open-Domain Question Answering
• COIL : Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List

TALK 2 : Information Retrieval & Open Domain QA (Sejune Joo)

• Reading Wikipedia to Answer Open-Domain Questions
• Latent Retrieval for Weakly Supervised Open Domain Question Answering
• REALM: Retrieval-Augmented Language Model Pre-Training
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

TALK 3 : Knowledge Graph Completion (Seungone Kim)

• Translating embeddings for modeling multi-relational data
• Knowledge Graph Embedding by Translating on Hyperplanes
• Learning Entity and Relation Embedding for Knowledge Graph Completion
• Embedding Entities and Relations for Learning and Inference in Knowledge Bases
• Complex Embeddings for Simple Link Prediction
• Open-World Knowledge Graph Completion
• Commonsense Knowledge Base Completion

TALK 3 : Commonsense Knowledge Graph Construction (Seungone Kim)

• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
• COMET: Commonsense Transformers for Automatic Knowledge Graph Construction

TALK 3 : Commonsense Knowledge Graph & QA (Sejune Joo)

• Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

TALK 4 : Internal Knowledge Base(LM) vs External Knowledge Base Analysis (Seungone Kim)

• Language Models as Knowledge Bases?
• How Much Knowledge Can You Pack Into the Parameters of a Language Model?
• Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
• Improving Question Answering with External Knowledge
• Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings

TALK 4 : Domain Adaptation (Seungone Kim)

• SciBERT: A Pretrained Language Model for Scientific Text
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• UDALM: Unsupervised Domain Adaptation through Language Modeling

TALK 4 : Knowledge Injection (Seungone Kim)

• ERNIE: Enhanced Language Representation with Informative Entities
• Knowledge Enhanced Contextual Word Representations
• K-BERT: Enabling Language Representation with Knowledge Graph
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
• KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation