• Benchmark the single-cell dataset again with the Feature selection R-package (feseR) previously developed.

  • Test in a single machine.

• Benchmark of the single-cell dataset in the following infrastructures using fsspark:

  • Single machine benchmark (preferably in a user laptop).
  • Spark cluster of a single node with multiple processors, benchmark with multiple processor sizes 10, 20, 50, 100?
  • Spark cluster with multiple nodes.

• Main priority is the creation of the CPTAC dataset #2 #3 using the phosphoproteomics and acetylome data.
• Review all the algorithms and see which one are loading everything in memory and which ones are parallelizing all the compute.
• Review other libraries that provide FS methods in Spark and reuse some of the algorithms.

Small issues:

• response in the file format must be changed to label
• The example should contain letter such as A, B, C rather than binary notation.
• Make sure the example (small example) always refer to gene expression and not protein expression. It is a GEO dataset.
• Annotate for every algorithm if is provided by Spark or is implemented by us.

• Create a README file in the repository where the dataset format and structure are described.
  - The description should include the structure of the Tab-Separate Value (TSV) file as the primary dataset structure input.
  - The interface with Spark Data Frame for Feature Selection (internal data structure used by the tool).
• Add to the README dataset file, the link to the Single-cell example we have been using for the benchmark of the algorithms.
  - Single-cell dataset: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE156793
  - Include the formatted matrix used as input by the tool.

We have tested the library and algorithms using single-cell data. However, we may use other types of data to see if the algorithms perform well.

• Contact the CPTAC Team to see if we can get help from them @ypriverol . In case the CPTAC team can't help us, we should try to generate the dataset ourself.
• Explore other sources of phosphorylation information, including quantms.
• Create the file format needed to benchmark the algorithms and the FS workflows.

The spark feature selection library for bigdata multiomics in an evolution of a previous R-package developed by Enrique et. al.. Major steps to finalize the library are:

• Create a README file in the repository where the dataset format and structure are described.
• Add to the README dataset file, the link to the Single-cell example we have been using for the benchmark of the algorithms.
• Benchmark the single-cell dataset again with the Feature selection R-package previously developed.
• Benchmark the single-cell dataset in the following infrastructures:
  • Single machine benchmark (preferably in a user laptop).
  • Spark cluster of a single node with multiple processors, benchmark with multiple processor sizes 10, 20, 50, 100?
  • Spark cluster with multiple nodes.
• Contact CPTAC team to get the list of phospho-sites with spectral counting with the different cancer and tumor types. @ypriverol #3
  • Perform the same benchmarks previously done for single-cell dataset.
• Create a readthedocs for the project.
• Implement the framework of algorithms:
  • Implement the independent feature selection algorithms: RF, correlation analysis, PCA.
  • Implement different workflows combining multiple FS algorithms. Name them.
  • Provide as group of command line tools that enable access to some of the workflow for a given dataset file.
• Discuss the given results and write a publication.

• The readthedocs for the project should include:
  • General description.
  • Description of the data structures.
  • Description of the methods for data pre-processing (e.g., imputation and normalization).
  • Univariate feature selection methods supported.
  • Multivariate feature selection methods supported.
  • Machine learning algorithms available.
  • Description of predefined FS workflows.
  • Examples (HOWTO).