pomodoren / comorbid-graphs Goto Github PK

Explore the best text metrics for creation of landscape.

• sklearn som
• text vectors
• spacy kgs

Recursive extraction of wikipedia categories together with lists.
Store the info as:

name: page_name
url: wikipedia.org/page_name

Introduction of multiparents should allow the graph properties.
Closed dependencies should allow for checks in this tree or not when loading from tree.

# closed-dependencies: checks whether all parents are in the hierarchy
# if not throws error
ComorbidGraph.load_yaml('graph.yaml', closed_dependencies=True)

To be able to do this kind of change, I should allow multiparent actions for each main functionality.

• load
  • from_tree
  • from_ontology
• export
• search
  • select
  • simple search
  • advanced search

For this we can use the existing scoring procedure at comorbid-labs which was relatively successful.

• update filter-mixin with scoring more than true/false
• update order-by in mix
• test

• create new nodes on analysis
• analyse_from_bow - from bag of words
• analyse_from_graph - from organized structured information
• analyse_from_spacy - from well known models

Most sources of dependencies.

• Pubmed
• Psycharxiv
• BioArxiv
• Arxiv
• Others ?

Allow different lambda's for updating order of parents for each node.

# sample.yaml
name: parent
children:
- name: node-1
  parents:
  - name: parent-1
  - name: parent-2
- name: node-2
  parents: 
  - name: parent-2 
  - name: parent-3 
name: parent-1 
name: parent-2
name: parent-3

Allow sorting by score.

ComorbidGraph
.load_yaml('sample.yaml')
.sort_parents(by_score=['score'])
.to_yaml('sample_ordered.yaml')

# sample.yaml
name: parent
children:
- name: node-1
  score: 1
  parents:
  - name: parent-1
    score: 0.5
  - name: parent-2
    score: 0.2
- name: node-2
  score: 0.8
  parents: 
  - name: parent-2 
    score: 0.3
  - name: parent-3 
    score: 0.1
name: parent-1 
name: parent-2
name: parent-3

ComorbidGraph
.load_yaml('sample.yaml')
.sort_parents(by_name=['parent-1', 'parent-3'])
.to_yaml('sample_ordered.yaml')

# sample_ordered.yaml
name: parent
name: parent-1 
children: 
- name: node-1
  parents:
  - name: parent
  - name: parent-2
name: parent-2
name: parent-3
children: 
  name: node-2
  parents:
  - name: parent
  - name: parent-2 

Closed dependencies when merging many graphs.

# closed_dependencies: check for at parent match in the hierarchy
# if not, release 
ComorbidGraph.merge_trees(
  list_of_graphs,
  closed_dependencies=True
)

• nodes involved
• main node metrics
• body length
• most mentioned word

Advanced extraction together with extraction of sections for pages that are not Category or List or some other special word.

Because of the problem of pointers of objects, we are using deep copy for now.
Still the problem increases in scale if we want to use the results of the search in an analysis sense.
In this case the problem will become more problematic once increase in scale.

Assume that you have two graphs with shared children-names.
How do you compare these?

• check where they share names of children - if not compare on bodies
• if they share names, combine names, create a matrix of comparison

• Define

anxiety, panic attack

inc_parent:DSM-V
inc_title:Disorder
inc_text_longer:300

exc_ancestor:Neurodevelopmental Disorders

• Code
• Test

Clear pdf structures for each.