A PEG-based query parser for INSPIRE.
inspirehep / inspire-query-parser Goto Github PK
View Code? Open in Web Editor NEWA PEG-based query parser for INSPIRE.
License: GNU General Public License v3.0
A PEG-based query parser for INSPIRE.
License: GNU General Public License v3.0
Search should support also report number queries with * eg. r atlas-conf-*
. It used to support it in a previous version
recid
is currently mapped as recid
in keywords mapping.
We need to map it to control_number
, while also adding a keyword mapping for control_number
.
Deleted.
So, it seems that date queries in ES are not working, as some records are missing some of the date fields that we are querying on.
"range": {
"date_field1": {
"gt": "2017"
},
"date_field2": {
"gt": "2017"
},
...
}
Thus we need to generate a chain of bool.should [ range1, range2, ... ]
kind of query.
This is for >, >=, <, <=
operators.
The tricky part would be the "equality", which comes from queries d 2016
or d = 2017-08
.
These have to be expressed in range queries, as well.
For this, it will be helpful to look into visitor_utils
.
Currently, we support queries like:
with variations of (-1, -2, ...).
Last month and this month queries should respect the current day. Currently they "reset" to the first day of the month.
For this badly formatted query: t: t: /electroweak bosons/
, we need to do some stripping, i.e. removing the :
character before directing it to ElasticSearch otherwise it's a baddly formatted query for its parser.
The query that's being generated is:
{
"query": {
"bool": {
"filter": [
{
"match": {
"_collections": "literature"
}
}
],
"minimum_should_match": "0<1",
"must": [
{
"query_string": {
"query": "t: t: /electroweak bosons/",
"default_field": "_all"
}
}
]
}
},
"from": 0,
"size": 25
}
That's a simple fix.
We could also introduce error handling during our parsing step.
@michamos informed me of a new keyword we need to support, that is irn
.
An example query is find irn 5988462
, this operator adds support for searching SPIRES
identifiers.
These are stored inside external_system_identifiers
when schema = SPIRES
.
Type code should support the following codes:
tc b
: should map to book
tc bookchapter
: should map to book chapter
tc c
: should map to article
tc note
: should map to note
tc proceedings
: should map to proceedings
tc t
: should map to thesis
@michamos can you pls check whether the above is correct?
Do we also need to support the following?
tc core
: When the schema contains core=true
tc p
: When the schema contains refereed= true
Is there a mapping for the existing:
i | Introductory and l | Lectures ?
Is there a mapping for the new doc types:
Given this query foo and "bar"
the parser crashes with aforementionened error.
Output is:
...
File "/Users/chris/development/inspire/inspire-query-parser/inspire_query_parser/parser.py", line 439, in parse
t, right_operand = parser.parse(text_after_bool_op, cls.grammar[2])
File "build/bdist.macosx-10.12-x86_64/egg/pypeg2/__init__.py", line 792, in parse
File "build/bdist.macosx-10.12-x86_64/egg/pypeg2/__init__.py", line 1102, in _parse
File "build/bdist.macosx-10.12-x86_64/egg/pypeg2/__init__.py", line 885, in _parse
...
which is repeated unitl it crashes.
We need support for exact-author as they were implemented in legacy.
The keywords bb
and bull
should link to eprint. Currently, they are linked to report number
The analyze_wildcard
flag parameter in _generate_query_string_query was introduced in case we wanted to disable wildcard analysis.
It seems that everywhere that is being used in the codebase is set to True
, which is actually the default behaviour of ElasticSearch (see docs).
For now it seems better to make the parameter True
by default and if that's the case, don't generate the "analyze_wildcard": True
in that method.
We need to exclude supervisor
role from authors search, as for the query a arkani-hamed
we got a match on this 1426739 records, while he is a supervisor and not an author.
To be done after merging the 1st version of author search and merged after the demo.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.