The biojupies from maayanlab

Lines 89 to 153 in 31046dc

    
           function upload_reads() { 
        
           	// Get credentials 
        
           	username = 'biojupies'; 
        
           	passwd = 'sequencing'; 
        
           	// Get files 
        
           	var files = $('#fileinput').prop('files'); 
        
           	// Loop through files 
        
           	$.each(files, function(index, file) { 
        
           		// Add progress bar 
        
           		progress_bar(file['name']); 
        
           		// Get Sign Policy 
        
           		$.getJSON("https://amp.pharm.mssm.edu/charon/signpolicy?username=" + username + "&password=" + passwd, function (data) { 
        
           			// Create Form 
        
           			var FD = new FormData(); 
        
           			FD.append('key', data['uid'] + "/{{upload_uid}}-${filename}"); 
        
           			FD.append('AWSAccessKeyId', data['cid']); 
        
           			FD.append('acl', 'private'); 
        
           			FD.append('success_action_redirect', 'success.html'); 
        
           			FD.append('policy', data['policy']); 
        
           			FD.append('signature', data['signature']); 
        
           			FD.append('Content-Type', 'application/octet-stream'); 
        
           			FD.append('file', file); 
        
           			// Get filename 
        
           			var filename = file['name']; 
        
           			// Create Request 
        
           			var xhr = new XMLHttpRequest(); 
        
           			xhr.open("POST", "https://" + data["bucket"] + ".s3.amazonaws.com/", true); 
        
           			xhr.onloadstart = function (e) { 
        
           				console.log("start") 
        
           			} 
        
           			xhr.onloadend = function (e) { 
        
           				// Set complete status on progress bar 
        
           				$('[data-filename="' + filename + '"] .progress-bar').attr('data-status', 'complete'); 
        
           				$('[data-filename="' + filename + '"] .progress-bar-text span:first-child').html('Successfully uploaded '); 
        
           				$('[data-filename="' + filename + '"] .progress-bar-text span:last-child').html(''); 
        
           				// Activate button if all progress bars are complete 
        
           				if ($('.progress-bar').length === $('.progress-bar[data-status="complete"]').length) { 
        
           					$('[form="read-upload-form"]').prop('disabled', false).toggleClass('black white bg-white bg-blue'); 
        
           				} 
        
           				console.log("end") 
        
           				// Call FASTQ Upload Status API 
        
           			} 
        
           			xhr.upload.addEventListener('progress', function (evt) { 
        
           				if (evt.lengthComputable) { 
        
           					var progress = Math.ceil((evt.loaded / evt.total) * 100)+'%'; 
        
           					$('[data-filename="' + filename + '"] .progress-bar').html(progress); 
        
           					$('[data-filename="' + filename + '"] .progress-bar').css('width', progress); 
        
           				} 
        
           			}, false); 
        
           			xhr.send(FD); 
        
           		}); 
        
           	}) 
        
           }

Add link back to BioJupies at the top of the Notebook

Users that receive a link to the notebook would probably what to see how it was created and what is biojupies exactly...

Provide figure for $0.01 alignment cost in Elysium

Look into CWL and WDL as formats to make the processing pipeline more standard

The BioJupies pipeline can be encoded into a pipeline that is described by the high-level pipeline specification languages CWL and/or WDL.

https://www.commonwl.org/

https://software.broadinstitute.org/wdl/

Allow users to assign custom names to uploaded datasets

Add a dataset name column to user dataset database table. Users should be able to rename them from the dashboard UI, potentially adding a field during the upload phase if the user is authenticated.

Publish notebook in JOIN by assigning it a DOI and submitting it for internal review

Add minimum number of samples to GTEx group selection

log10P in the volcano plot instead of -log10P

Hello,

I was using biojupies and noticed that the volcano plot y-axis has 'log10P' while it should be - log10P, so all the values are positive.

It doesn't interefere with the analysis, but might be confusing for some people.

(great tool, nevertheless)

Text edits to this page http://amp.pharm.mssm.edu/biojupies-dev/upload

Raw FASTQ sample sequencing files generated from an RNA-seq profiling study

Update data to ARCHS v6

Add user FK to notebook table and user dataset table

Users should be allowed to access their own notebooks and uploaded datasets from the dashboard.

GTEx analysis seems to be broken

Selected 20-29 old and not exposed skin in group 1 and 20-29 exposed skin in group 2, click continue... ended back in the starting page.

Add progress bar for the alignment process

User is left waiting for a long time during the alignment process. It would be great to have a progress bar that keeps increasing, perhaps based on an estimate of the time it should take based on the file size and number of samples.

Allow users to delete datasets, notebooks and alignment jobs from dashboard

Files not uploading

Tried to upload 6 files... No progress... See screenshot. Using Windows and Chrome.

Fix spacing when multiple normalization methods are selected

biojupies/server/app/static/py/NotebookGenerator.py

Line 198 in b963ed5

    
           normalization_methods_section += '##### {}\n'.format(annotations['core_options'][normalization_method]['option_name'])+annotations['core_options'][normalization_method]['methods']

Unable to upload an Expression Table File.

Hi!

I am trying to upload an Expression Table File (.xlsx), but I get the following error:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500 Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
I tried several times with Firefox 57 and also with Chromium 67 using Linux.

Thank you!

Add Google OAuth

Prevent selection of same GTEx sample in two groups

biojupies/website/app/templates/analyze/gtex.html

Lines 171 to 303 in 31046dc

    
           $(document).ready(function () { 
        
               // Load GTEx metadata 
        
               $.ajax({ 
        
                   url: "{{ url_for('gtex_api') }}", 
        
                   dataType: "json", 
        
                   method: "POST", 
        
                   success: function(json) { 
        
                       // Loop through tables 
        
                       $('.gtex-table').each(function(index, elem) { 
        
                           // Create table 
        
                           var table = $(elem).DataTable({ 
        
                               data: json, 
        
                               aaSorting: [], 
        
                               pagingType: 'simple', 
        
                               columns: [ 
        
                                   {data: 'checkbox', orderable: false, className: 'select-checkbox'}, 
        
                                   {data: 'Age', orderable: false}, 
        
                                   {data: 'Gender', orderable: false}, 
        
                                   {data: 'Tissue', orderable: false} 
        
                               ], 
        
                               select: { 
        
                                   style: 'multi' 
        
                               }, 
        
                               fnCreatedRow: function (nRow, aData, iDataIndex) { 
        
                                   $(nRow).attr('data-sampleid', aData['id']); // or whatever you choose to set as the id 
        
                               }, 
        
                               initComplete: function () { 
        
                                   this.api().columns().every(function () { 
        
                                       var column = this; 
        
                                       if (column.index() > 0) 
        
                                       { 
        
                                           var select = $('<select class="gtex-sample-filter"><option value=""></option></select>') 
        
                                               .appendTo($(column.header()).empty()) 
        
                                               .on('change', function () { 
        
                                                   var val = $.fn.dataTable.util.escapeRegex( 
        
                                                       $(this).val() 
        
                                                   ); 
        
                                                   column 
        
                                                       .search(val ? '^' + val + '$' : '', true, false) 
        
                                                       .draw(); 
        
                                               }); 
        
                                           column.data().unique().sort().each(function (d, j) { 
        
                                               select.append('<option value="' + d + '">' + d + '</option>') 
        
                                           }); 
        
                                       } 
        
                                   }); 
        
                               } 
        
                           }) 
        
                           // Add listener for checkbox 
        
                           table.on('click', '.select-all', function(evt) { 
        
                               // Get button 
        
                               var $button = $(this); 
        
                               // Deselect and select 
        
                               if (table.rows({ selected: true }).count() > 0) { 
        
                                   table.rows().deselect(); 
        
                                   $button.removeClass('active'); 
        
                               } else if (table.rows({ filter: 'applied' }).count() == table.rows().count()) { 
        
                                   alert('Please select at least one filter in the table such as age, gender or tissue.') 
        
                               } else { 
        
                                   table.rows({ filter: 'applied' }).select(); 
        
                                   $button.addClass('active'); 
        
                               } 
        
                           }) 
        
                           // Add update number of samples 
        
                           table.on('select deselect', function(evt) { 
        
                               var $col = $(evt.target).parents('.gtex-table-col'), 
        
                                   selected_rows = table.rows({ selected: true }), 
        
                                   ids = Array.from(selected_rows.data().pluck('id')); 
        
                               // Update number 
        
                               $col.find('.gtex-table-header .nr-samples').html(selected_rows.count()); 
        
                               // Add samples to HTML 
        
                               // $col.find('.selected-samples').data('id', ids); 
        
                               $col.find('.selected-samples').val(ids); 
        
                           }) 
        
                           // Change group names 
        
                           table.on('change', '.gtex-sample-filter', function (evt) { 
        
                               var $table = $(this).parents('table'), 
        
                                   group_name = $table.find('.gtex-sample-filter').map(function () { if ($(this).val().length) {return $(this).val()} }).get().reverse().join(', '); 
        
                                   $table.parents('.gtex-table-col').find('.gtex-group-name').first().val(group_name); 
        
                                   console.log(group_name); 
        
                           }) 
        
                       }) 
        
                       // Show table 
        
                       $('.gtex-table').removeClass('d-none'); 
        
                       $('.gtex-table-header').removeClass('d-none'); 
        
                       $('.gtex-loader').addClass('d-none'); 
        
                   } 
        
               }) 
        
               // Check sample size 
        
               $('button[form="select-samples-form"]').click(function(evt) { 
        
                   // Prevent submission 
        
                   evt.preventDefault(); 
        
                   // Get group size 
        
                   group_1_size = $('#DataTables_Table_0_wrapper tr.selected').length; 
        
                   group_2_size = $('#DataTables_Table_1_wrapper tr.selected').length; 
        
                   // Check sizes 
        
                   if ((group_1_size < 2) || (group_2_size < 2)) { 
        
                       // Alert 
        
                       alert('Please select at least two samples for each group using the provided checkboxes.') 
        
                   } else { 
        
                       // Submit 
        
                       $('#select-samples-form').submit(); 
        
                   } 
        
               }) 
        
           })

Fix pair-end analysis

Allow users to specify private datasets

If toggled, the dataset will be made private on Google Cloud. This happens directly from the server. The notebook generator server will automatically set the dataset to public prior to notebook generation.

Add a private column to the dataset table.
Upon setting a dataset to private, toggle privacy on Gcloud.
Whenever a notebook is submitted through the website, toggle privacy on and then off.

Low priority.

Support upload of larger files

Note: The form currently supports uploading of files in .fastq.gz format of less than 1GB in size.

Clustergrammer and L1000FWD Query not loading

Tried to upload notebook from BioJupies to JShelf... Clustergrammer and L1000FWD did not load in the notebook.

Dump from Inspect is pasted below:

Failed to load resource: the server responded with a status of 404 (NOT FOUND)
login.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
upload.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
launch.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
css.css:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 This page includes a password or credit card input in a non-secure context. A warning has been added to the URL bar. For more information, see https://goo.gl/zmWq3m.
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
BioJupies.ipynb:740 Mixed Content: The page at 'https://nbviewer.jupyter.org/urls/s3.amazonaws.com/biodos/e62583354f9bb743b10359c2913aa82d/BioJupies.ipynb' was loaded over HTTPS, but requested an insecure resource 'http://amp.pharm.mssm.edu/clustergrammer/viz/5b4395231576c511ba09e19c/tmpqvbmaeb_.txt'. This request has been blocked; the content must be served over HTTPS.
BioJupies.ipynb:40453 Mixed Content: The page at 'https://nbviewer.jupyter.org/urls/s3.amazonaws.com/biodos/e62583354f9bb743b10359c2913aa82d/BioJupies.ipynb' was loaded over HTTPS, but requested an insecure resource 'http://amp.pharm.mssm.edu/l1000fwd/vanilla/result/5b43958f5c8056001639e5fe'. This request has been blocked; the content must be served over HTTPS.
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
css.css:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)

Troubleshooting section does not mention the option for static images

All three issues under troubleshooting should provide optional instructions on how to select static rendering of figures in PNG format.

Add metadata entry form and instruction on how to make uploaded datasets FAIR

Integrate CEDAR for annotation of generated notebooks

https://metadatacenter.org/tools-training/cedar-metadata-tools

Figure S1 in the revised manuscript has text appearing twice... This looks like a mistake

ubuntu Docker image
Marathon/Mesos
orchestrator
Flask, uWSGI, nginx
back-end

Improve error message for failed signature generation

Landing page text edits

Biojupies Automatically Generates
RNA-seq Data Analysis Notebooks
With BioJupies you can produce in seconds a customized, reusable, and interactive report from your own raw or processed RNA-seq data through a simple user interface

Pass alignment info to expression tables

Save alignment info in dataset HDF5
Add alignment FK to user dataset table

Buttons to download the CSV files of the processed data and results

Ability to download the raw reads per gene counts
Ability to download the normalized reads per gene
Ability to download the signature values for all genes
Ability to download the enrichment results
Ability to download the small molecule query results

Validate plugin parameters in notebook generation step

Jupyter Notebook generation using Jinja2

Add Dropzone accepted files option

biojupies/website/app/templates/upload/upload_table.html

Lines 86 to 98 in 31046dc

    
           dropzone = $("#dropzone").dropzone({  
        
           	dictDefaultMessage: "Drag and drop your file here, or click to browse and upload it.", 
        
           	url: "{{ url_for('upload_dataframe_api') }}", 
        
               success: function(file, response){ 
        
               	// Parse response 
        
               	response = JSON.parse(response); 
        
               	// Add dataset 
        
               	addTable(response); 
        
               } 
        
           });

Elysium API middleware

Edit to text for the alignment step https://amp.pharm.mssm.edu/biojupies-dev/upload/reads?alignment=AJo8zVxvtho

The RNA-seq alignment jobs have been successfully submitted to the Cloud. The table below displays the status of each job, including the names of the FASTQ files, the reference genome used for alignment, and the creation date. Once the alignment jobs have completed, the Continue button will become active.

Add example CSV download file in Help section

Tool selection card interface

Develop user dashboard

Require users to select at least one tool before continuing

Automatically assign minids to newly generated notebooks

http://minid.bd2k.org/
https://github.com/fair-research/minid

Allow users to specify private notebooks

Add a private column to the table. If it's true, it will only display if the user is logged in.

A tutorial for BioJupies Generator

How to trigger the BioJupies Generator at GEO after I installed? So confused! A tutorial will be helpful. Thank you.

Fix span highlighting in search page

biojupies/website/app/app.py

Line 208 in c709c10

h = lambda x: '<span class="highlight">{}</span>'.format(x)

Capture methods for uploaded FASTQ files

Update video tutorial links

Only two classes in many GSEs - suggestion of enhancement

Hello,

I have been using biojupies satisfactorily for a while now, and I notice that for many studies, there are only two groups: one control and one perturbation.

Automatic labeling of classes is hard, and the underlying code does a nice job of identifying controls.

For binary cases (only one perturbation), it would be practical to have a "Tag all non-assigned samples to Perturbation" button.

It is minor, but might save many clicks for many users.

Thanks,

Text edits to this page http://amp.pharm.mssm.edu/biojupies-dev/upload/reads

Use the form below to upload raw RNA-seq data in FASTQ format. The files will be aligned to the reference genome to generate the quantified reads expression table at the gene level. The table will be subsequently made available for you to download and to generate the interactive notebook. Once you have uploaded the desired files, click Continue to proceed.

Upload Read Files
To begin, select the files you wish to upload using the form below, then click on Upload Files to proceed.

Add progress bar to notebook generation stage

If you select a lot of tools, the notebook generation takes a long time. It would be great to see a progress bar that keeps increasing as time passes so users don't leave the site.

	function upload_reads() {

	// Get credentials
	username = 'biojupies';
	passwd = 'sequencing';

	// Get files
	var files = $('#fileinput').prop('files');

	// Loop through files
	$.each(files, function(index, file) {

	// Add progress bar
	progress_bar(file['name']);

	// Get Sign Policy
	$.getJSON("https://amp.pharm.mssm.edu/charon/signpolicy?username=" + username + "&password=" + passwd, function (data) {

	// Create Form
	var FD = new FormData();
	FD.append('key', data['uid'] + "/{{upload_uid}}-${filename}");
	FD.append('AWSAccessKeyId', data['cid']);
	FD.append('acl', 'private');
	FD.append('success_action_redirect', 'success.html');
	FD.append('policy', data['policy']);
	FD.append('signature', data['signature']);
	FD.append('Content-Type', 'application/octet-stream');
	FD.append('file', file);

	// Get filename
	var filename = file['name'];

	// Create Request
	var xhr = new XMLHttpRequest();
	xhr.open("POST", "https://" + data["bucket"] + ".s3.amazonaws.com/", true);
	xhr.onloadstart = function (e) {
	console.log("start")
	}
	xhr.onloadend = function (e) {
	// Set complete status on progress bar
	$('[data-filename="' + filename + '"] .progress-bar').attr('data-status', 'complete');
	$('[data-filename="' + filename + '"] .progress-bar-text span:first-child').html('Successfully uploaded ');
	$('[data-filename="' + filename + '"] .progress-bar-text span:last-child').html('');

	// Activate button if all progress bars are complete
	if ($('.progress-bar').length === $('.progress-bar[data-status="complete"]').length) {
	$('[form="read-upload-form"]').prop('disabled', false).toggleClass('black white bg-white bg-blue');
	}
	console.log("end")

	// Call FASTQ Upload Status API
	}
	xhr.upload.addEventListener('progress', function (evt) {
	if (evt.lengthComputable) {
	var progress = Math.ceil((evt.loaded / evt.total) * 100)+'%';
	$('[data-filename="' + filename + '"] .progress-bar').html(progress);
	$('[data-filename="' + filename + '"] .progress-bar').css('width', progress);
	}
	}, false);
	xhr.send(FD);

	});

	})
	}

	$(document).ready(function () {

	// Load GTEx metadata
	$.ajax({
	url: "{{ url_for('gtex_api') }}",
	dataType: "json",
	method: "POST",
	success: function(json) {

	// Loop through tables
	$('.gtex-table').each(function(index, elem) {

	// Create table
	var table = $(elem).DataTable({
	data: json,
	aaSorting: [],
	pagingType: 'simple',
	columns: [
	{data: 'checkbox', orderable: false, className: 'select-checkbox'},
	{data: 'Age', orderable: false},
	{data: 'Gender', orderable: false},
	{data: 'Tissue', orderable: false}
	],
	select: {
	style: 'multi'
	},
	fnCreatedRow: function (nRow, aData, iDataIndex) {
	$(nRow).attr('data-sampleid', aData['id']); // or whatever you choose to set as the id
	},
	initComplete: function () {
	this.api().columns().every(function () {
	var column = this;
	if (column.index() > 0)
	{
	var select = $('<select class="gtex-sample-filter"><option value=""></option></select>')
	.appendTo($(column.header()).empty())
	.on('change', function () {
	var val = $.fn.dataTable.util.escapeRegex(
	$(this).val()
	);

	column
	.search(val ? '^' + val + '$' : '', true, false)
	.draw();
	});

	column.data().unique().sort().each(function (d, j) {
	select.append('<option value="' + d + '">' + d + '</option>')
	});
	}
	});
	}
	})

	// Add listener for checkbox
	table.on('click', '.select-all', function(evt) {

	// Get button
	var $button = $(this);
	// Deselect and select
	if (table.rows({ selected: true }).count() > 0) {
	table.rows().deselect();
	$button.removeClass('active');
	} else if (table.rows({ filter: 'applied' }).count() == table.rows().count()) {
	alert('Please select at least one filter in the table such as age, gender or tissue.')
	} else {
	table.rows({ filter: 'applied' }).select();
	$button.addClass('active');
	}

	})

	// Add update number of samples
	table.on('select deselect', function(evt) {
	var $col = $(evt.target).parents('.gtex-table-col'),
	selected_rows = table.rows({ selected: true }),
	ids = Array.from(selected_rows.data().pluck('id'));

	// Update number
	$col.find('.gtex-table-header .nr-samples').html(selected_rows.count());

	// Add samples to HTML
	// $col.find('.selected-samples').data('id', ids);
	$col.find('.selected-samples').val(ids);

	})


	// Change group names
	table.on('change', '.gtex-sample-filter', function (evt) {
	var $table = $(this).parents('table'),
	group_name = $table.find('.gtex-sample-filter').map(function () { if ($(this).val().length) {return $(this).val()} }).get().reverse().join(', ');
	$table.parents('.gtex-table-col').find('.gtex-group-name').first().val(group_name);
	console.log(group_name);

	})

	})

	// Show table
	$('.gtex-table').removeClass('d-none');
	$('.gtex-table-header').removeClass('d-none');
	$('.gtex-loader').addClass('d-none');

	}
	})

	// Check sample size
	$('button[form="select-samples-form"]').click(function(evt) {

	// Prevent submission
	evt.preventDefault();

	// Get group size
	group_1_size = $('#DataTables_Table_0_wrapper tr.selected').length;
	group_2_size = $('#DataTables_Table_1_wrapper tr.selected').length;

	// Check sizes
	if ((group_1_size < 2) \|\| (group_2_size < 2)) {

	// Alert
	alert('Please select at least two samples for each group using the provided checkboxes.')

	} else {

	// Submit
	$('#select-samples-form').submit();

	}
	})

	})

	dropzone = $("#dropzone").dropzone({
	dictDefaultMessage: "Drag and drop your file here, or click to browse and upload it.",
	url: "{{ url_for('upload_dataframe_api') }}",
	success: function(file, response){

	// Parse response
	response = JSON.parse(response);

	// Add dataset
	addTable(response);

	}
	});

maayanlab / biojupies Goto Github PK

biojupies's People

Contributors

Stargazers

Watchers

Forkers

biojupies's Issues

Recommend Projects

Recommend Topics

Recommend Org