Goal to solve several Facebook AI Research(FAIR)'s babi QA tasks using a novel approach based on temporal graph traversal.
The tasks are described in detail in the paper : Towards AI Complete Question Answering: A Set of Prerequisite Toy Tasks Artificial Tasks for AI Each task checks one skill that a reasoning system should have. Performing well on all of them is a pre-requisite for any system aiming at understanding language and able to reason.
To solve 3 of 20 QA tasks (https://github.com/facebook/bAbI-tasks)
# | Task | Class name |
---|---|---|
1 | Basic factoid QA with single supporting fact | WhereIsActor |
2 | Factoid QA with two supporting facts | WhereIsObject |
6 | Yes/No questions | IsActorThere |
- Construction of temporal graph to represent the activities of actors in the story
- Parse the input dataset-(facts of a task) to extract (timestamp, verb, pos, lemma) using Stanford CoreNLP Parser
- Build the graph where Nodes represent subject & objects, Edges represent verb actions associating subject to object using python networkx module
- Visualize graph using matplotlib.pyplot
- Answering questions by traversing graph.
- Parse the questions to identify subject/object and locate the corresponding node in graph
- Use Word Embedding and K-Means clustering on input dataset, to provide semantic heuristic to graph traversal algorithm in order to distingish verb actions which associate subject-object using gensim Clustered verb actions annotated into classes: attach, detach, transport
- Traverse the graph to find the answers
- Interactive mode to provide text2speech feedback while parsing stories using espeak accompained with intuitive graphs which present the state of graph model
Project Demo : https://www.youtube.com/watch?v=WnN6i4ODGHM
Project Pipeline : How To Guide
src\
: source codetasks_1-20_v1-2\
: set of 20 tasks for testing text understanding and reasoning in the bAbI project
input\
- *_train.txt - tasks_1-20_v1-2/en-10k/ 10,000 training examples
parsedInput\
- *jl - Part of Speech and Facts/Question-Answer(Supporting Facts) identified
models\
: word2vec binary model files using gensimclusters\
: clusters computed using k-means algorithm with best silhouette_score.annotatedClusters\
: annotated clusters with appropriate labels identifying (attach/detach/transport/person/location/misc)output\
:
- *jl - predicted and actual answer with supportingFacts.
- Setup Stanford CoreNLP server
- Download and extract Core NLP Server http://stanfordnlp.github.io/CoreNLP/#download
- Start the server on Port 9000
java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000
http://localhost:9000/ - Test Sample
- Python wrapper for Stanford CoreNLP : https://github.com/smilli/py-corenlp
- Espeak https://ports.macports.org/port/espeak-ng/, Python Espeak https://github.com/relsi/python-espeak
- GenSim : http://radimrehurek.com/gensim/install.html
- K-Means Clustering : http://scikit-learn.org/stable/install.html
Module | Description |
---|---|
babiparser.py | Parse the dataset to annotate POS using StanfordCoreNLP, identifying Facts/Question-Answer(Supporting Facts) |
get_clusters.py | Leverage parsed input to obtain word2vec embedding for facts, cluster the vectors using k-means clustering with k (having best silhouette_score) |
annotate_clusters.py | Annotate clusters with appropriate labels identifying (attach/detach/transport/person/location/misc) |
babigraph.py | Represents facts of story in temporal graph with Actors, Objects/Location as Nodes and relationship between them as edges labelled with timestamps using networkx. Parse the questions to identify the corresponding actor/object node in the graph. Leverage annotated cluster labels to filter candidate edges associated with the node and use the latest edge to answer the questions. |
evaluate.py | Computes No of Correct and Incorrect Predictions |