Code Monkey home page Code Monkey logo

neurocurator's Introduction

Getting Started | Upgrade | Releases | Roadmap | Status

For the Annotation Viewer or the integration with the OpenMinTeD platform, please refer to this branch.

NeuroCurator

Desktop application to perform systematic and collaborative curation of neuroscientific literature.

This is a Graphical User Interface (GUI) for the Python module NeuroAnnotation Toolbox (NAT).

This framework has been described in details in the following open-access paper: https://doi.org/10.3389/fninf.2017.00027.

With NeuroCurator, annotations are:

  • traceable,
  • reusable across projects,
  • structured with controlled vocabularies.

Getting Started

Installation

After having installed the requirements:

pip3 install neurocurator

You might want to install NeuroCurator in a dedicated conda environment:

conda create --name neurocurator_env python=3.7
conda activate neurocurator_env
pip install neurocurator

To launch NeuroCurator:

neurocurator

Requirements

Python dependencies

Upgrade

pip3 install --upgrade neurocurator

If you have used conda, activate the environment before:

conda activate neurocurator_env

Releases

Versions and their notable changes are listed in the releases section.

Roadmap

Ongoing

  1. Stabilize NeuroCurator and NAT (Software Architecture).
  2. Split NAT into nat-core, nat-analytics and nat-server (separation of scope).

TODO

  1. Integrate fully the Annotation Viewer into NeuroCurator.
  2. Semi-automate the Knowledge Extraction process.

Done (latest first)

  • Make the annotations publishable into a Blue Brain Nexus instance.
  • Remove legacy dependencies in NeuroCurator (Qt 4 and Python 3.4).
  • Integrate OpenMinTeD annotations into the literature curation framework.
  • Visualize NeuroCurator and OpenMinTeD annotations directly on PDFs.
  • Make NeuroCurator easily installable, especially by scientists.

Updated on 04.10.18.

Status

Created during 2016.

Ongoing stabilization and reengineering in the branch refactor-architecture.

The branch refactor-architecture is not intended to be used by end-users.

New features, bug fixes and improvements are done on the reengineered code sections.

When a reengineered code section is stable, it's merged into the branch master and a release is published.

neurocurator's People

Contributors

christian-oreilly avatar pafonta avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

mfsy

neurocurator's Issues

Annotations: ignoring annotation modification doesn't work

Steps:

  1. Click on an annotation of type 'table' (for example) in the Listing of existing annotations table of the Annotations tab.
  2. Delete a character in the Row no. form or a tag in the Annotation tags list (for example).
  3. Click on a different annotation.
  4. A dialog asks There are unsaved changes in the currently loaded annotation. Do you want to save modifications?.
  5. Click on No.
  6. The modification is not reverted.

Stack trace:

None

Focus issue when creating a new annotation

A code regression caused an "Invalid localizer" error showing 3 times when a user want to create a new annotation. This is likely a problem of focus causing and early call to the function used to localize the string of annotated text in the PDF.

Better support for local projects

Projects that are not linked to a remote GIT project can already be setup by creating a local GIT repository. However, the user will always be asked if he want to push modifications to a remote. The user should be able to specify the remote (or absence of remote) in the preference dialog and the program should not ask for pushing if there is no remote associated with the project.

[References] No legend for the colors used

There is no explanation in the UI for the signification of the colors - white, grey, green - of the rows.

References in:

  • white have no PDF attached,
  • grey have a distant PDF attached,
  • green have a local PDF attached.

When a user attaches locally a PDF, the reference is considered to have a distant PDF attached. Having a distant PDF attached is the same as having a user who attached a PDF at least one time.

#42 should be fixed before.

The Qt binding mess

  • Support for PySide ended with Python 3.4, which blocks the use of the NeuroCurator with newer version of Python. This will become an increasingly important matter. Already, up-to-date Ubuntu systems comes with Python 3.5 and previous versions of Python has been removed from the apt directories (see https://askubuntu.com/a/802300 for a get-around), which complicates the use of the NeuroCurator on these systems.
  • PyQt4 has similar limitations and the installation process is complicated.
  • PyQt5 is compatible with 3.5, 3.6, and 3.7 and is pipable. However, this would mean porting the code from Qt4 to Qt5 and loosing the compatibility with prior Python versions.
  • PySide2 is not stable yet.

There is no clear path to offers easy-to-install, flexible (i.e., compatible with a large range of Python versions). Maybe adding compatibility with various (e.g., PySide and PyQt5) bindings would be the best solution.

Annotation review GUI

For collaborative annotation workflow, it is generally a good idea to have an annotation review process. Although this process could be done using the GIT diff of JSON annotations, this is obviously far from ideal. We should provide a GUI to support annotation revision. Correspondingly, we should add a field to the annotation structure for "reviewers" (i.e., idem as for the "authors" field, but to capture who reviewed the annotation, if anyone did).

[References] Context menu (right click) to edit references

On operating systems with the menu bar at the top of the screen, we have to click on the reference to edit and then to go to the top of the screen to open the Zotero menu. It would be quicker to be able to launch the edition of a reference through a context menu appearing after a right click on it.

Ontology service REST access point URL

The nat ontoServ.py script:

  1. hard code the URL for the ontology service points
  2. provide a set of functions.

This code should probably be re-factored as a proper OntoServ class and the URL should be passed at the initialization (maybe using the presently hard coded values as default values for backward compatibility). The URL should be modifiable in the preference dialog of the NeuroCurator.

Annotations > Parameters: 'SyntaxError: unexpected EOF while parsing'

Steps:

  1. Double-click on an empty cell in the column 'Values' in the Parameter details panel.
  2. Click elsewhere.

Stack trace:

Traceback (most recent call last):
  File "/Users/fonta/repos/neurocurator/neurocurator/paramValueWgt.py", line 229, in setData
    return self.setDataByIndex(index.row(), index.column(), value)
  File "/Users/fonta/repos/neurocurator/neurocurator/paramValueWgt.py", line 200, in setDataByIndex
    values = eval(data)
  File "<string>", line 0
    
    ^
SyntaxError: unexpected EOF while parsing

Command > 'Add a term to ontologies' is not implemented

Currently, to add new local ontological terms, we used to add them manually to the additionsToOntologies.csv file. An entry "Add a term to ontologies" was added in the "command" menu of the NeuroCurator, but is currently not functional. We need to complete the implementation of this functionality so that the users can easily add new ontological terms for tagging. Further, in the current way to do things, adding a term requires modifying the code of NAT (additionsToOntologies.csv) and pushing the modifications to the server. This behavior needs to be modified because it is not practical for users of NeuroCurator to have to push GIT modifications to NAT. Further, modifications cannot be only local since these terms must be standardized across users. Maybe the functionality could be implemented through the REST server.

Annotations > Parameters: doesn't (ask to) save modifications when switching context

Modifications are not saved when:

  • we switch to an other parameter (a)
  • we switch to an other annotation (b)
  • we switch to an other paper (c)
  • we close the application (d)

Modifications seem to have been saved visually (ie. the new value is displayed) for parameters of type 'point value' when switching to an other parameter or annotation.

The modifications are visually reverted (ie. the previous value is displayed) for parameters of type 'numerical trace' and 'function' when switching (a, b, c) or closing and for 'point value' when switching to an other paper or closing.

Steps:

  1. Double-click on a cell with a value in the Parameter details panel.
  2. Change the value.

(a)
3. Click on an other parameter
4. Click on the modified parameter

(b)
3. Click on an other annotation
4. Click on the modified annotation
5. Click on the modified parameter

(c)
3. Click on the Paper Zotero database tab.
4. Double-click on a grey entry.
5. Click on the Paper Zotero database tab.
6. Double-click on the previous paper.
7. Click on the modified annotation.
8. Click on the modified parameter.

(d)
3. Obvious.

The modified value hasn't been saved.

NB: Tested with publications of ID 10.1016/j.neuroscience.2014.02.026 (point value and numerical trace parameter types) and PMID_9457638 (function parameter type).

Stack trace:

None

Function annotation

The annotation of functions is currently buggy. It need to be thoroughly re-tested.

Relative paths

In the configuration file, when relative paths are used, these end-up being relative to the path from which the application has been started and not the path from which the application is such that

  1. python3.4 neurocurator/main.py
  2. python3.4 main.py

will not use the same path for this configuration element. This behavior should obviously be avoided.

Selecting experimental properties

Clicking on the checkbox to select an experimental property in the "Relevant experimental properties" table does not work on Mac.

Annotations: 'Open PDF' doesn't consider the first attempt

Steps:

  1. Double-click on a grey entry in the Paper Zotero database tab.
  2. Click on 'Open PDF' and confirm by hitting 'Yes'.
  3. Select a PDF file.
  4. Click on 'Open'.
  5. The file selection window closes and reopens.

Steps 3 and 4 need to be done a second time. It shouldn't be the case.

Won't fix. Here just to document this behaviour until #24 is done.

Stack trace:

None

Switching between corpus

Add an easy way to switch between corpus. Adding in the preference dialog buttons to save/load project configurations should do the trick.

[Notebooks] /notebooks should be in the NAT repository

Check if the folder notebooks can safely be deleted from the NeuroCurator repository.

These notebooks could have been modified in the NeuroCurator repository after the project splitting and the creation of the NAT repository.

Annotations: 'IndexError: list index out of range'

Step:

  1. Click on an annotation of type 'position' in the Annotations tab.

Tested with the publication of ID (DOI) 10.1038/367069a0. The PDF, which is a scan, was attached before the above step.

Stack trace:

Traceback (most recent call last):
  File "/Users/fonta/repos/neurocurator/neurocurator/annotWidgets.py", line 531, in annotationSelectionChanged
    self.loadThumbnail()
  File "/Users/fonta/repos/neurocurator/neurocurator/annotWidgets.py", line 584, in loadThumbnail
    self.container.currentAnnotation.localizer.height)
  File "/Users/fonta/repos/neurocurator/neurocurator/areaSelector.py", line 403, in loadImage
    image = QtGui.QImage.fromData(areaSelector.pages[pageNo-1],"PNG");
IndexError: list index out of range

Paper Zotero database: selection is not kept when sorting

Steps:

  1. Click on a paper in the Paper Zotero database tab.
  2. Sort by an attribute (ex: year) except 'Annotations' (see #18).

The displayed* selection is on a different paper.

*in the Annotations tab, the annotations are still the ones from the paper selected at step 1.

Stack trace:

None

Zotero > 'Add reference' doesn't work

Steps:

  1. Click on Zotero in the menu bar then on Add reference.
  2. Nothing happens (on the application side).

Stack trace:

Traceback (most recent call last):
  File "/Users/fonta/repos/neurocurator/neurocurator/mainWin.py", line 378, in addToZotLib
    addToZoteroDlg = AddToZoteroDlg(self.zoteroTableModel, self)
  File "/Users/fonta/repos/neurocurator/neurocurator/addToZoteroDlg.py", line 62, in __init__
    self.loadReference(row)            
  File "/Users/fonta/repos/neurocurator/neurocurator/addToZoteroDlg.py", line 69, in loadReference
    item = self.zotWrapTable.zotWrap.refList[row]
TypeError: list indices must be integers, not Window

[Packaging] Packaged executable build the 'ontological trees ' at each startup

When not packaged, NeuroCurator build the 'ontological trees' only at the first start.

Doing it at each startup slow down the starting of NeuroCurator.

Terminal output:

Building ontological tree for  PATO:0000261 ... 
Building ontological tree for  NIFORG:birnlex_569 ... 
Building ontological tree for  CHEBI:23367 ... 
Building ontological tree for  GO:0097458 ... 
Building ontological tree for  NIFGA:birnlex_1167 ... 
Building ontological tree for  NIFMOL:nifext_2506 ... 
Building ontological tree for  NIFSUB:sao914572699 ... 
Building ontological tree for  nifext_8054 ... 
Building ontological tree for  sao1813327414 ... 

Annotations: 'Localize' has an unexpected behaviour

Steps:

  1. Click on an annotation of type 'text' in the Annotations tab (ex: [1]).
  2. Click on Correct.
  3. Delete the last character in the Annotated text form (ex: the trailing period in [1]).
  4. Click on Localize.
  5. The value of the Starting character form changes (ex: from 16442 to 16486 for [1]).

It is unexpected. The starting character value shouldn't change in this case.

[1]
Tested with the annotation of ID c77e7b60-02da-11e6-93a1-3417ebb8f5ca.
It can be find by using the Search > Annotations tab and the Annotation ID criterion with this value. Clicking on Search and then double-clicking on the annotation in the result list brings you to the situation just after step 1. You need to have the PDF of the article to use Localize: https://doi.org/10.1113/jphysiol.2004.070888.

Stack trace:

None

Annotations: 'Select area' doesn't work

Steps:

  1. Click on an annotation of type 'position' in the Annotations tab.
  2. Click on Select area in the Annotation details panel.

Tested with the publication of ID (DOI) 10.1038/367069a0. The PDF, which is a scan, was attached before the above steps.

Probably linked to #36.

Stack trace:

Traceback (most recent call last):
  File "/Users/fonta/repos/neurocurator/neurocurator/annotWidgets.py", line 562, in selectArea
    self.selectAreaDlg.exec_()
  File "/Users/fonta/repos/neurocurator/neurocurator/areaSelector.py", line 47, in exec_
    self.open()
  File "/Users/fonta/repos/neurocurator/neurocurator/areaSelector.py", line 65, in open
    self.selectDlg = PDFAreaSelectorDlg(self)
  File "/Users/fonta/repos/neurocurator/neurocurator/areaSelector.py", line 172, in __init__
    self.loadImage()
  File "/Users/fonta/repos/neurocurator/neurocurator/areaSelector.py", line 191, in loadImage
    image = QtGui.QImage.fromData(self.parent.pages[self.parent.currentPageInd],"PNG");
IndexError: list index out of range

how to install and use ?

Hello,
I find the idea of this project great but how can I actually use it ?
I think it would be very helpful if there was an "Install" and a "Getting started" section in the readme because right now I have no idea how to use it.

Annotations > Parameters: 'Parameter details' panel is not properly cleared or initialized

Steps:

  1. Click on an annotation with a parameter.
  2. Click on a parameter of this annotation.
  3. Click on an other annotation.

Although there is no parameter selected, the Parameter details panel show the form for a parameter of type 'text', with 'point value' as Result type selected, or the type of the last seen parameter.

The panel should be initialized and cleared without setting a form for a specific parameter type.

Tested with the publication of ID (DOI) 10.1016/j.neuroscience.2014.02.026. This publication has annotations of type 'text' and 'figure'.

Stack trace:

None

[Annotations] Empty PCR file created automatically after adding a new reference

Steps:

  1. Click on the Zotero menu, then on Add reference.
  2. Create a new reference (tested with one with an UNPUBLISHED ID).
  3. An empty PCR file is created in the local repository folder.

This is an implementation logic, neither a bug nor a feature. However, this could lead to misunderstanding for the user of NeuroCurator (UI) and NAT (annotations).

NeuroCurator:
Why nobody in my curation team has the PDF of a reference in grey (ie. no PDF has been submitted to the localization server)?

NAT:
Why is there empty PCR files in the corpus of annotations?

The goal would be to have an other implementation logic which doesn't rely on these empty PCR files.

NB: The row added in the References (Zotero) tab has therefore the color (grey) of references having been added to the "curator database".

ValueError: ufunc <ufunc 'positive'> not supported by quantities

antonkulaga@YOGA ~/neurocurator $ python3 neurocurator/main.py
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/quantities/quantity.py", line 231, in array_prepare
res._dimensionality = p_dictuf
KeyError: <ufunc 'positive'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "neurocurator/main.py", line 8, in
from neurocurator.mainWin import Window
File "/usr/local/lib/python3.5/dist-packages/neurocurator/mainWin.py", line 21, in
from nat.utils import Id2FileName #, fileName2Id
File "/usr/local/lib/python3.5/dist-packages/nat/init.py", line 4, in
from .annotation import Annotation
File "/usr/local/lib/python3.5/dist-packages/nat/annotation.py", line 10, in
from .modelingParameter import ParameterInstance, ParamRef
File "/usr/local/lib/python3.5/dist-packages/nat/modelingParameter.py", line 6, in
import quantities as pq
File "/usr/local/lib/python3.5/dist-packages/quantities/init.py", line 285, in
from .units import *
File "/usr/local/lib/python3.5/dist-packages/quantities/units/init.py", line 9, in
from . import acceleration
File "/usr/local/lib/python3.5/dist-packages/quantities/units/acceleration.py", line 7, in
from .time import s
File "/usr/local/lib/python3.5/dist-packages/quantities/units/time.py", line 18, in
aliases=['milliseconds']
File "/usr/local/lib/python3.5/dist-packages/quantities/unitquantity.py", line 62, in new
ret._conv_ref = definition._reference
File "/usr/local/lib/python3.5/dist-packages/quantities/quantity.py", line 139, in _reference
rq = rq * u._reference**d
File "/usr/local/lib/python3.5/dist-packages/quantities/unitquantity.py", line 230, in pow
return self.view(Quantity).pow(other)
File "/usr/local/lib/python3.5/dist-packages/quantities/quantity.py", line 88, in g
return f(self, other, *args)
File "/usr/local/lib/python3.5/dist-packages/quantities/quantity.py", line 321, in pow
return super(Quantity, self).pow(other)
File "/usr/local/lib/python3.5/dist-packages/quantities/quantity.py", line 236, in array_prepare
""" % uf
ValueError: ufunc <ufunc 'positive'> not supported by quantities
please file a bug report at https://github.com/python-quantities

[References] Preset 'creatorType' to 'author' in the creation/edition forms

'author' is the most common 'creatorType'. It could be great to preset the value of 'creatorType' to 'author' for new creators.

However, with the current UI/UX design (automatic addition of an empty row), doing this could lead to misunderstanding: will this row with the 'creatorType' preset be added to the reference?

Paper Zotero database: sorting by 'Annotations' doesn't work

Steps:

  1. Click on the column header Annotations in the Paper Zotero database tab.
  2. Nothing happens (the rows are not sorted).

Stack trace:

Traceback (most recent call last):
  File "/Users/fonta/repos/neurocurator/neurocurator/zoteroWrap.py", line 168, in sort
    self.zotWrap.refList = sorted(self.zotWrap.refList, key=lambda x: self.getByIndex(x, col).lower(), reverse = reverse)
  File "/Users/fonta/repos/neurocurator/neurocurator/zoteroWrap.py", line 168, in <lambda>
    self.zotWrap.refList = sorted(self.zotWrap.refList, key=lambda x: self.getByIndex(x, col).lower(), reverse = reverse)
AttributeError: 'int' object has no attribute 'lower'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.