ismms-himc / clustergrammer-gl Goto Github PK

Allow users to selection section of the heatmap to zoom into.

Do not actually crop into this section, but just zoom in. Need to be careful about tracking zoom with this function. Also, improve camera resetting, ideally reset camera after zooming/panning event is over.

I'm going to probably just make this work through cropping. I'll build a general method to remake the entire viz based on filtering for an arbitrary set of rows/columns (dendrogram will not be shown, rearrange buttons will either be deactivated or reflect on-the-fly calculations). This general filter network method will be built off this method (https://github.com/MaayanLab/clustergrammer/blob/master/src/network/filter_network_using_new_nodes.js) and will be generalized to support dendrogram cropping, crop to zoomed area, and zoom into selection.

• get arbitrary network row/col filtering working
• set up SVG brush crop similar to brush_crop_matrix
  • need to do the algebra to find selected rows/cols on brushing

Make a method for plotting single cell spatial data using Voronoi plots.
Look into this (regl webgl dots as points) as method to draw circles in the center of the Voronoi plots: https://gist.github.com/vlandham/0415aa443992f71288e86137208fa690

Have the front-end generate histograms for a single row/column and have it broken into separate histograms for each category.

For instance, a user can quickly see the expression of a gene across different cell types.

https://github.com/rivulet-zhang/vis-utils/tree/master/hdbscanjs

Use the custom version of clusterfck (https://harthur.github.io/clusterfck/) to implement front-end clustering (e.g. load csv of data and cluster) and re-clustering (e.g. after performing arbitrary filtering). This will be useful for simplifying the deployment of Clustergrammer-gl (no need for clustering on python), Clustergrammer2 arbitrary re-clustering, and deploying Clustergrammer-gl as a standalone library on observable.hq.

Sometimes the camera and zoom object can get out of sync. They are reset when the user zooms all the way out (as a fail safe). We should ultimately prevent this mis-syncing from happening or add an additional fail safe that resets the camera when the user has finished interacting with the visualization.

In addition to showing the Harmonizome Refseq name and description add a clickable link to the Harmonizome and/or Uniprot gene pages (clicking on name will make the permanent).

First and foremost this is an amazing library. Thank you for your work.

I am trying to implement clustergrammer-gl in a Vue application but I am running into errors.

clustergrammer-gl.node.js?3773:42556 Uncaught TypeError: Cannot read property '0' of undefined
    at clustergrammer_gl (clustergrammer-gl.node.js?3773:42556)
    at VueComponent.renderHeatmap (BxHeatmap.vue?de86:45)
    at <anonymous>:1:6

coming from this line:
var canvas_container = d3.select(container).select('.canvas-container')[0][0];

import * as clustergrammer from 'clustergrammer-gl';
this.heatmapInstance = clustergrammer({
        container: '#heatmap',
        network: this.heatmapJSON,
        viz_height: 1000,
        viz_width: 1000,
      });

where this.heatmapJSON is a viz_json generated from clustergrammer-py.

It works with the clustergrammer-js but not the gl lib.

Please let me know if theres something I am not doing correctly.

Is your feature request related to a problem? Please describe.
I want to be able to change the color scheme - currently it is only red/blue for positive/negative values, respectively

Describe the solution you'd like
I would like to be able to pass in two colors in the arguments JSON (e.g. named colors or hexcode) and have those override the positive/negative colors.

Additional context
This issue is necessary for resolving this issue in Clustergrammer2 ismms-himc/clustergrammer2#39

Some users want to visualize the data in its original order. This may be some special configuration of rows/columns that is not obtainable with other ordering mechanisms (other than value-based-categories).

We will likely replace the alphabetical ordering button, since this button is really only used as a temporary replacement for row/column searching, which will be addressed with this issue #23

We received the following recommendations from the Observable Discourse page (https://talk.observablehq.com/t/help-with-reactive-updates-from-custom-library/2907):

• Clustergrammer-GL requires that the container be inserted into the DOM before initializing, and requires a globally-unique identifier for that container. If it didn’t have these requirements, it would be easier to initialize in Observable (where cells normally run with detached DOM elements, and only insert them into the DOM by returning a value at the end).

• The most natural way to make Clustergrammer-GL reactive would be for it to emit custom events when things change. I did that manually by intercepting when some internal parameters are set, and by listening to browser native events (mousemove). But if Clustergrammer-GL emitted these events itself, it’d be more straightforward to expose those as reactive values in Observable using Generators.observe.

• It’s best to avoid bundle.run in notebooks, as it isn’t very reliable. I re-attached the generated bundle to your notebook, but if you published an AMD- or UMD-compatible bundle to npm, then folks could require("clustergrammer-gl") and it would work out of the box.

My matrix has very long sample names as row names and GO/KEGG annotations as column names. These are mostly hidden from view in the Web API and jupyter widget version. Also, mousing over a cell to see its annotations is not feasible when the hovertext only lasts 2 seconds. A feature to accommodate longer names or infinite hovertext time would make my experience using the tool flow better.

In the shorter term if you had any hack-way of going about getting these row/column names to display fully I would be keen to try it.

Is your feature request related to a problem? Please describe.
We would like to be able to zoom into the heatmap and export the visible rows and columns.

Describe the solution you'd like
Clicking on the heatmap arena should bring up a dendrogram like tooltip that shows the rows and columns and makes them available in different formats.

Describe alternatives you've considered
None

Additional context
None

Use simple input HTML element to run row/column (e.g. gene/cell) search.

• set up search autocomplete using some light library (not bootstrap)

• highlight found row/column with arrow (WebGL) or tooltip (SVG)

• be able to clear search result

• possibly look into SVG based input box: https://stackoverflow.com/questions/4176146/svg-based-text-input-field

• Add support for searching more than one row/col.

In order to speed up the calculation of text triangles, we could try to pre-calculate all letters and numbers and then construct labels etc. from the single letter position arrays. We would have to keep track of spacing between letters so this might be tedious.

Also, in terms of reducing the size of the JavaScript, we could try making a smaller text2vect that only produces text triangles.

Is your feature request related to a problem? Please describe.
We want to be able to reorder on one category and have the previous category ordering be maintained

see MaayanLab/clustergrammer#49

Use the mean and variance values (that are shown as value based categories) to display both the Z-scored and original values when mousing over a matrix cell.

Users should be able to pass a row/column dictionary with additional information (e.g. longer names or other information) as an argument. This will display when mousing over rows/columns instead of making a GET request to the Harmonizome.

For instance

• Row mouseover should give the full definition of these NBA stats:
  https://www.kaggle.com/cornhundred/heatmap-view-of-modern-era-player-stats?scriptVersionId=15969060
• Custom surface marker information (e.g. in house surface marker information) can be given as a mouseover and this small dictionary can be loaded into the notebook without having to make GET requests to some server.

Is your feature request related to a problem? Please describe.
We want to be able to shut off the gene lookup feature using the Harmonizome for datasets where it is not appropriate.

Describe the solution you'd like
We should have an option passed to Clustergrammer-GL (and Clustergrammer2 should pass to Clustergrammer-GL) that says no Hzome lookup.

Try and make a react component for Clustergrammer-GL following the example from

https://github.com/deargen/react-clustergrammer

Describe the bug
The blue rounded squares of the control panel are not being rendered correctly in the Firefox browser. We think this is an issue with defining the SVG rendering since we have seen that Chrome seems to be more lenient about syntax rules.

To Reproduce

1. Open heatmap in Mozilla and see no blue button outlines.

Expected behavior
We expect to see the selected button have a blue rounded square.

Desktop (please complete the following information):

• OS: MacOS
• Browser: Chrome and Firefox

Is your feature request related to a problem? Please describe.
We want to be able to visualize the expression of a gene across different categories of cells.

Describe the solution you'd like
Clicking on a gene should show a box plot with the expression of that gene across each category.

Describe alternatives you've considered
We could have a heatmap that displays the average expression of all genes across categories, but that does not generalize well to multiple types of categories.

Additional context
We have a similar modular approach to enrichment analysis we're working on using Observable where we show the enrichment results as separate D3 barplots

https://observablehq.com/@ismms-himc/clustergrammer-gl-and-enrichr

We want to show the box plots as separate plots in this same manner.

Animate transition to and from UMAP for rows and columns when pressing appropriate reordering buttons.

Is your feature request related to a problem? Please describe.
I want to reduce the speed of scrolling.

Describe the solution you'd like
Scroll zooming should happen more slowly.

Describe alternatives you've considered
None

Probably just use this url http://amp.pharm.mssm.edu/clustergrammer/static/icons/graham_cracker_70.png

Add gene name lookup using the Harmonizome and Enrichrgram support to Clustergrammer2 widget (see https://clustergrammer.readthedocs.io/biology_specific_features.html)

Describe the bug
Clustergrammer heatmap doesn't render well using clustergrammer2 generated network and clustergrammer-gl for rendering. Here is the piece of code in charge of rendering:

import cgm from 'clustergrammer-gl'

const ReactClustergrammer = ({ root, height, width, maxWidth, className, onColumnLabelClick, ...args }) => {
  const draw = () => {
    let network = args.network;
    cgm({
      network,
      viz_width: 900,
      viz_height: 900,
      use_hzome: true,
      root: `#${root}`,
    });
  };

  React.useEffect(() => {
    if (typeof window !== 'undefined' && args.network !== null) {
      window.d3 = d3;

      draw();
    }
  }, [args.network]);

  return <div id={root} className={className} style={{ height, width: width, maxWidth: maxWidth }}></div>;
};

Depending on the parameters it raises different problems:

• with parameter root: '#${root}' it doesn't render at all but without any error
• with parameter container: '#${root}' it raises the following error TypeError: targetel.getScreenCTM is not a function that seems to be linked to d3-tip .

Dependencies

clustergrammer-gl@^0.23.0:
  version "0.23.0"
  resolved "https://registry.yarnpkg.com/clustergrammer-gl/-/clustergrammer-gl-0.23.0.tgz#0321df460419221d2d2ccda9b2a54384b10d8007"
  integrity sha512-bAhd9EzyNRUspBKEaflo/XTPhkiD3zro+OK/jzYfkjijQ3iRNF1R6qirq863PEFtwKCFpVCzjfdRnyqUpz1WDA==
  dependencies:
    axios "0.19.0"
    d3 "5.15.0"
    d3-tip v0.9.1
    eases "^1.0.8"
    gl-mat4 "^1.1.4"
    gl-vec4 "^1.0.1"
    ify-loader "^1.1.0"
    mathjs "3.17.0"
    mouse-change "^1.4.0"
    mouse-event-offset "3.0.2"
    normalized-interaction-events "2.0.1"
    pako "1.0.6"
    regl "^1.3.7"
    underscore "1.8.3"
    util-extend "1.0.3"
    vectorize-text "^3.2.1"
    xtend "^4.0.1"

Desktop:

• macOS monterey 12.4

Hello, I would like to try clustergrammer-gl in my application - may I ask you have a plan to publish this to npm registry soon?

Variable Size and Full Screen

Have the heatmap be variable width/height (or at least variable width and square). This functionality will allow a variable width Jupyter Widget ismms-himc/clustergrammer2#2 (comment)

Thoughts on Implementing Full Screen Mode

Add button to toggle between full screen and normal view. Have control panel float above the visualization. Maybe make a new WebGL context that is positioned on top of everything else (or maybe make a modal type full screen).

After issue #1 is finished, work on some Observable-hq examples.

• Les Miserables observable notebook

• Dashboard notebook for exploration of single cell RNA-seq data

make this button after support is added to the back-end, see ismms-himc/clustergrammer2#16

Describe the bug
when tip hide ,tooltip may be overflow

Is your feature request related to a problem? Please describe.
We want to be able to see the distribution of value-based-categories in a selected cluster (e.g. hovering over a dendrogram).

Describe the solution you'd like
Show a histogram of the values on mouseover, etc.

Describe alternatives you've considered
Alternatively, we could just show a box plot.

Additional context
This issue was originally raised here MaayanLab/clustergrammer#14, but will be addressed in Clustergrammer-GL first.

https://github.com/MaayanLab/clustergrammer-docs

Is your feature request related to a problem? Please describe.
Clustergrammer-GL is being used by Clustergrammer2 and information needs to be passed from Clustergrammer-GL to the Clustergrammer2 widget. We need to enable callback functions to be passed to Clustergrammer-GL. These can be used for instance to: update the widget state on mouseover.

Describe the solution you'd like
Have callback functions as arguments to Clustergrammer-GL. Have these callback functions run on specific interaction events (e.g. click, mouseover).

Describe alternatives you've considered
I have not considered alternatives.

Additional context
This is going to be useful for Clustergrammer2 widget functionality.

Clicking the dendrogram should bring up the dendrogram tooltip, but allow the user to interact with it, e.g. stay visible permanently until closed again by clicking the dendrogram (or an x in the tooltip). I will probably need to create a separate class for this tooltip to prevent it from being hidden and to prevent additional tooltips from popping up while it is visible.

The tooltip needs:

• text selection box for rows/columns (e.g. comma separated)
• wrap around text for row and column cluster selection will instead have external widget view of selected terms see Observable examples
• run enrichment analysis via Enrichr
  • https://observablehq.com/@ismms-himc/clustergrammer-gl-and-enrichr
• export genes to Enrichr
• crop button to zoom into this section
  • will be useful for zooming into clusters of genes
  • will be useful for running front-end enrichment analysis on a cluster of genes

Setting up re-initialization (as well as destroying previous contexts) of visualization. Need to re-create network filtering as seen here

https://github.com/MaayanLab/clustergrammer/blob/master/src/network/filter_network_using_new_nodes.js

Hi,
I'm impressed by clustergrammer-gl using the toy data.
I would like to try it with my large matrix (40000 X 100) on my machine. It seems there is no info about how to install it on a local machine.
Sorry for the trivial question, I'm pretty new to jupyter.
Thanks!

Describe the bug
Zooming is breaking when the matrix is 200x200 in size.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.com/ismms-himc/clustergrammer2-notebooks
2. Run the first example notebook
3. Make the random matrix 200x200
4. Zooming does not work correctly (matrix is not hiding correctly)

Expected behavior
Zooming should work.

Desktop (please complete the following information):

• OS: MacOS
• Browser Chrome

Is your feature request related to a problem? Please describe.
It can be useful to download the data and/or metadata/categories that is being visualized in a heatmap.

Describe the solution you'd like
We want to implement a button that triggers a download of the data matrix. This can also be very helpful for saving manual categories on a static HTML and then downloading the CSV in order to save manual categorization.

Describe alternatives you've considered
We may want to have separate downloads for the data and row/col metadata.

Additional context
None

Is your feature request related to a problem? Please describe.
We would like to have improved granular control of where slices are taken in the dendrogram.

Describe the solution you'd like

• We would like to take slices of the dendrogram at any level.
• we want to visualize the entire visualization tree (perhaps as stacked triangles)

Describe alternatives you've considered
We could take more slices evenly along the dendrogram.

Additional context
This has been a feature-request for Clustergrammer-JS for a while. It will be implemented here first. See MaayanLab/clustergrammer#8

Fix bleedthrough of heatmap through low opacity non-highlighted categories.

https://github.com/ismms-himc/clustergrammer-gl/blob/master/src/tooltip/display_and_position_tooltip.js#L36

Is your feature request related to a problem? Please describe.
We want to be able to run and undo normalization procedures on data (e.g. undo Z-score, run Z-score, divide columns by sum, etc).

Describe the solution you'd like
We currently have a simply toggle button called Z-scored that shows up when a dataset has been initially Z-scored. We want to replace this with a more complicated normalization menu (similar to the reorder and recluster menus). This will allow users to run/undo normalization procedures on rows/columns (e.g. undo row Z-score then undo column divide by sum (UMI norm for scRNA-seq data)).

Describe alternatives you've considered
N.A.

Additional context
Depending on the state of incoming data we will need to allow certain calculations to be run and others not to be which depends on the normalization steps that the data has undergone.

To Do

• fix row reordering with initialized z-scored data