asknowqa / lc-quad Goto Github PK

Hey guys,

Is there a plan to get ride of the DBpedia property and shift it to the ontology space anytime soon?

Cheers,
Hamid

There's a single QA pair in the dataset where the corrected_question is entirely different from what it's supposed to be (i.e. given the SPARQL and auto-verbalized question):

The SPARQL is wrong too, so this makes it the only question with a nil result set.

[{'_id': '3631',
  'corrected_question': 'List down the schools whose mascot is an animal from the order of Even toed Ungulates?',
  'intermediary_question': "What are the <companies> whose <programming language>'s <designer> is <Bjarne Stroustrup>?",
  'results': [],
  'sparql_query': 'SELECT DISTINCT ?uri WHERE { ?x <http://dbpedia.org/property/designer> <http://dbpedia.org/resource/Bjarne_Stroustrup> . ?uri <http://dbpedia.org/property/programmingLanguage> ?x  . ?uri <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/ontology/School>}',
  'sparql_template_id': 306}]

Fix template 9 from
{ <%(e_in_in)s> <%(e_in_in_to_e_in)s> ?x . ?x <%(e_in_to_e)s> ?uri }
to
{ <%(e_in_in)s> <%(e_in_in_to_e_in)s> ?x . ?x <%(e_in_in_to_e_in)s> ?uri }

Also interchange 3 and 303

Describe the bug
There are some funny things in DBpedia.
For instance, dbr:Nicaragua rdf:Type dbo:MusicalArtist

Template ID: Any

Expected behaviour
Somehow, iron these kinks out. Ignore this triple, or at least don't make questions out of it.

Actual behaviour
No such filters. You'd end up with a question containing this :/

Describe the bug
http://lc-quad.sda.tech/ says 5042 entities covered while only 1369 in entities.txt

The bracket of the where-clause ends too early in templates: 302, 306, 315, 316, 402, 405, 407 and 408.
The templates 303, 307 and 403 don't fit with the corresponding questions. They need one more triple:
303: ?x rdf:type class
307: ?uri rdf:type class
403: ?x rdf:type class
In the dataset templates are referenced that don't exist: 11, 601, 605, 906.

I think it maybe a good idea to include the Wikidata/DBpedia answer to these questions for KBQA training/testing purposes

Is the code used in the paper http://lc-quad.sda.tech/static/ISWC2017_paper_152.pdf available? Where? I would like to reproduce the steps to create a dataset for a specific domain...

hi ,
i think you should put the number of the total resource in the lc_quad dataset , to get sure about the number when we calculate precision and recall in entity linking , and i whould ask if the number of resource use in the sparql_queries is 6621 resource ??

Hey guys,
There are couple of pairs in which the question doesn't make sense w.r.t its query:

{
"_id": "4121",
"corrected_question": "What organisation regulates and controls the New Sanno Hotel?",
"intermediary_question": "What is the of New Sanno Hotel ?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/New_Sanno_Hotel http://dbpedia.org/ontology/tenant ?uri } ",
"sparql_template_id": 2
},
{
"_id": "3557",
"corrected_question": "What are the awrds won by Laemmle Theatres ?",
"intermediary_question": "What is the of Laemmle Theatres ?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/Laemmle_Theatres http://dbpedia.org/ontology/service ?uri } ",
"sparql_template_id": 2
},
{
"_id": "4917",
"corrected_question": "Who owns Chelsea F.C.?",
"intermediary_question": "Who is the whose is ?",
"sparql_query": "SELECT DISTINCT ?uri WHERE {?uri http://dbpedia.org/ontology/occupation http://dbpedia.org/resource/Chelsea_F.C. . }",
"sparql_template_id": 1
},
{
"_id": "4932",
"corrected_question": "What is the location of Sam Sen Railway Station ?",
"intermediary_question": "What is the of Sam Sen Railway Station ?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/Sam_Sen_Railway_Station http://dbpedia.org/property/other ?uri } ",
"sparql_template_id": 2
},
{
"_id": "4225",
"corrected_question": "Where did Aghasalim Childagh die?",
"intermediary_question": "What are the of Aghasalim Childagh?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/Aghasalim_Childagh http://dbpedia.org/property/deathDate ?uri } ",
"sparql_template_id": 2
},
{
"_id": "4103",
"corrected_question": "What is the political party to which Purnima Banerjee is a member of?",
"intermediary_question": "What is the of Robert Nutting ?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/Robert_Nutting http://dbpedia.org/ontology/knownFor ?uri } ",
"sparql_template_id": 2
},
{
"_id": "3390",
"corrected_question": "Where is Lao Plaza Hotel located?",
"intermediary_question": "What is the of Lao Plaza Hotel ?",
"sparql_query": " SELECT DISTINCT ?uri WHERE { http://dbpedia.org/resource/Lao_Plaza_Hotel http://dbpedia.org/property/developer ?uri } ",
"sparql_template_id": 2
},

Is your feature request related to a problem? Please describe.
Need a script to seamlessly use a QA solution, and benchmark its results over LC-QuAD

Describe the solution you'd like

• Require them to open an API which returns answers or SPARQLS given a question.
• Use over-arching platforms like Project Hobbit, but this requires dockerized systems and other added constraints.
• What else?

Running LC-QuAD locally would require a local DBpedia instance.
We should provide a simple guide to do that.

There are 190 unique predicates (occurring a total of 1931 times) that I found in the dataset that are not listed in resources/predicates.txt. See not_in_predicates.txt.

Also, 21 predicates listed in resources/predicates.txt do not occur in the dataset at all. See not_in_dataset.txt.

I’m working on a KBQA system on LC-QuAD 1.0. I would like to ask some questions about the evaluation.
Should I include the literal in golden answers when I execute the golden SPARQL against DBpedia 1604? I noticed that all lambda variables in golden SPARQLs are ‘?uri’. Since there’s no FILTER in SPARQL for literals, does it mean answers are just URIs?
Thanks.

Describe the bug
The README on the repository has a link to LCQUAD 2.0, http://lc-quad.sda.tech/. Using that link however does not seem to load.

Expected behaviour
Redirected to LCQUAD 2.0

Actual behaviour
I get a site unreachable error.