Tool to consolidate and identify correlation in literature searches
The intention of this tool is to perform literature search across various catalogues, and consolidate results in a de-duplicated list.
The tool performs a correlation analysis between different search terms.
It is WiP ๐ง and requires manual steps.
- Define search strategy in
SearchTable.xlsx
and export as csv - Perform manual searches and save results
- Adjust
LitResearch.ipynb
as needed - Run
LitResearch.ipynb
- Review Correlation Analysis
- Find consolidated results in
consolidated_results.csv
- Each search consists of a combination of two search terms (tuples)
- Each search can be performed across one or multiple source catalogues
- Each search requires a unique ID
- Each search is assigned to exactly one group (groups are e.g. synonyms of the same term)
Populate the Excel as follows:
ID
: Unique ID of the searchsource_<name>
: one column for each source catalogue; additional columns can be added as needed;TRUE
if the respective search shall be performed in that catalog,FALSE
otherwiseTerm1
andTerm2
: the respective search term tupleGroup
: Group assignment of the searchcount_<name>
: per each catalogue, the number of expected search results (if known, e.g. for manual searches; this is only a countercheck/quality criterion)YearFrom
: start year of the searchURL_<name>
: Excel formula, URL leading to the search results online; this is for user convenience only
Save the file as csv file.
For all searches, which do not offer a search API (e.g. Scopus, IEEE Xplore), perform the search manually and export the search results to csv.
Create a subfolder named exactly as the catalogue name, and save the csv according to the respective search ID, e.g.: /Scopus/4.csv
.
The following can be adjusted without changing the code:
- Target Columns
- Mapping of search result columns to Target Columns
- Adding API calls for further sources (currently, only arXiv API is implemented)
The correlation analysis shows:
- off-diagonal values: per each row, how many of the search results have also been found in another group
- on-diagonal values: how many of the search results of the row have exclusively been found in the respective group
Results are saved including row totals in group_correlation.csv
Results are saved in consolidated_results.csv