xiphereal / ieiwarehouse Goto Github PK

It seems to be a bug in relation with Publication being mixed/duplicated with the other types: If there's a Book: {"Title1", 2018, "https://url.com"}, then our Warehouse also retrieves and Article and CongressCommunication with that same data.

With the Warehouse containing all the data extracted from all the 3 data sources and for the REST request:
localhost:8080/getData?author=Anna Krizanova&searchCongressCommunications=false,

the following response is received:

If copy-pasted to the MySQL Workbench, the SQL query made for that request in the BookDAO retrieves the same results from the database. That might means that the problem lies in the extractors or in the persistence logic for some entities.

This tentative bug has been detected in #12.

Create a standalone UI project (maybe using Spring too) with the load and search forms. Also, it must connect to the backend endpoints for the corresponding load and search APIs.

Publications from Google Scholar doesn't contain the 'URL' attribute, so it must be obtained with Selenium.

Related to #6.

Implement the data scraping from Google Scholar using Selenium.

The data can be filtered within this Selenium extraction if the related Selenium scripting doesn't present any major problem or before the returned response of the REST API of #6.

Implement wrapper for the IEEE data source, with a REST API for requesting year-filtered data.

This wrapper should be a standalone module (see #16 as reference, specially 25b8610).

It seems that the demo account we should use for the connection to the IEEE API only allows for a limited number of results before asking for a captcha. That limit of number of result has to be taken into account when performing the queries.

The returned response must be in JSON format, only containing results in the requested year range.

I've seen some empty 'resolveEntitiesRelationships' methods in the GoogleScholar extractor, and I don't really know if those are as they should, meaning no relationship must be resolved hence the methods should be removed, or if it's an error and the relationships are not resolve.

The given methods are:

 private static void resolveEntitiesRelationshipsBook(Book book, List<Person> authors) {
        return;
    }

    private static void resolveEntitiesRelationshipsCommunication(CongressCommunication congressCommunication, List<Person> authors) {
        return;
    }

Could anyone take a better look at it? Thanks! :)

Implement a REST API for the Google Scholar wrapper that allows for year-filtered requests to the data source.

Note that this REST API must be in the GoogleScholarWrapper module, integrated alongside #15. The reason of not creating a single issue for the whole wrapper and splitting the contents in two, is because the data extraction is performed with Selenium and it seems to be a tiresome and hard work just on its own.

The returned response must be in JSON format (transformed from BibTex), only containing results in the requested year range.

Implementation of a REST API that lives in the IeiDataWarehousePopulator project, allowing requests for filtered queries to our data warehouse.

The returned response must be filtered by the options specified in the form (see poliformat lab files for more information)

Implement a wrapper for the DBLP data source, with a REST API for requesting year-filtered data.

This wrapper should be a standalone module (see #16 as reference, specially 25b8610).

There will be no real connection to the DBLP website or API, so the data source will be the XML file in Poliformat.

The returned response must be in JSON format, translated from that XML file (in Poliformat there's a code snippet for that XML-JSON mapping), and only containing results in the requested year range.