To reproduce, just click SUBMIT on the front page. There is nothing returned.

Update on home page

• Create a new tab in Help page for introductions of AMPSphere.
• The first column, by data type: genomes and metagenomes, AMPs and families.
• Statistics (maybe represented using a plot) and Search box (MMseqs and HMMSearch) go to home page, and two buttons to select search method.
• Create a Krona plot for AMP distribution across environments.

Potential icon resources: https://www.g2.com/products/biorender/competitors/alternatives

Update on AMP page

• Two numbers for genomes and metagenomes in General information
• Two subsections for graphs: Comparison with entire database and Features for the current AMP
• Secondary structure: pie chart (the rest is disordered)
• Sunburst plots for the distribution across habitats and hosts (all linages)
• Titles for the distribution plots.
• Download button: download as a pdf or an HTML.
• Associated metagenomes and proGenomes2 genomes
• Basic information (a table with three different columns: GMSC SAMPLE/GENOME TAXONOMIC_GENE_ORIGIN)
• Geographical distribution (colors are determined by microontology level I)
• Pep-fold for 3D structure prediction.
• Outline navigation on the left side.
• Figure captions for amp graphs.

Microontology

air	3681
anthropogenic	4055
anthropogenic:built environment	21418
anthropogenic:food	7
anthropogenic:food:fermented food	1848
anthropogenic:mine	555
anthropogenic:mock community	106
anthropogenic:sludge	33206
anthropogenic:wastewater	39952
aquatic	2093
aquatic:estuarine	4521
aquatic:freshwater	55769
aquatic:freshwater:lake	31024
aquatic:freshwater:pond	458
aquatic:freshwater:river	13313
aquatic:groundwater	7670
aquatic:ice	3214
aquatic:marine	96479
aquatic:marine:pelagic	43051
aquatic:saline	19152
aquatic:spring:hot spring	2304
aquatic:spring:hydrothermal vent	1523
host-associated:algal host	232
host-associated:animal host	19063
host-associated:animal host:coral reef	3407
host-associated:animal host:digestive tract	654
host-associated:animal host:digestive tract:intestine	218637
host-associated:animal host:digestive tract:mouth	11402
host-associated:animal host:digestive tract:mouth:saliva	7458
host-associated:animal host:digestive tract:rumen	41552
host-associated:animal host:insect host	2854
host-associated:animal host:mammalian host:human host	271
host-associated:animal host:reproductive tract	197
host-associated:animal host:respiratory tract	3377
host-associated:animal host:skin	6535
host-associated:animal host:urogenital tract	109
host-associated:plant host	6249
host-associated:plant host:leaf	2993
host-associated:plant host:phyllosphere	5264
host-associated:plant host:plant litter	9896
host-associated:plant host:rhizosphere	75929
nan	4094
ph:alkaline	1514
sediment	51020
terrestrial	4116
terrestrial:dust	50
terrestrial:halite	326
terrestrial:permafrost	9048
terrestrial:soil	240668
terrestrial:subsurface	1806
terrestrial:wetland	20207

User stories

• 1. Biochemist trying to figure out if their sequence was already observed and retrieve some info out of AMPSphere:
  • Example peptide: KRVKLLLKGYMRAIEINAALMYGYRPK
  • Search for similar peptides
  • Search for its family
• 2. Biochemist trying to produce or characterize one sequence from AMPSphere:
  • Example peptide: AMP10.000_010
  • Text search the access and retrieve the peptide information
  • Check features and peptide secondary structure
  • Check the genes
• 3. Ecologist trying to figure out the relationship between a certain bacteria in the microbiota of a given host:
  • Example host: Azolla sp. (a plant) and Nostoc azollae (a bacteria)
  • Browse filtering of the host and the original taxon, retrieve the list of AMPs
  • Entering the AMP access codes and check the location and habitat (guts, mouth, skin...)
• 4. Bioinformatician wanting to use AMPSphere as a resource:
  • Download the files
  • Retrieve the peptides and the genes
  • Retrieve families
• 5. Now imagine the case where the guy just wants to retrieve quick info
     and maybe export this to somewhere else, like your blog
  • use API
• 6. The same bioinformatician wants more info about a family of a certain AMP:
  • Example peptide: AMP10.000_010
  • Find the family, get the number of sequences, distribution in the world, the features, genes, environment, and hosts
  • Retrieve the HMM profile for the family and the logos for the sequence and the model

TODO:

+ implement a column and the filter in the browser tab for the AMP taxon origin (ex. E. coli)
+ implement file format and columns explanation in the download page 
+ add more explanation in the independent and conditional e-values in the HMMer search page (Family search)

Tips

https://github.com/BigDataBiology/AMPSphereWebsite/tree/main/frontend#performance-diagnosis-tips

Potential solutions

https://juejin.cn/post/6844903961154207757

The current layout is based on Quasar but changing all Element Plus components is too costly. Leave this here for following up when other important tasks are done.

Is your feature request related to a problem? Please describe.
Currently, the package size is too large and makes the website a bit slow.

Describe the solution you'd like
Can be improved by importing Quasar through CDN (https://quasar.dev/start/umd).

Describe alternatives you've considered
No

Additional context
No

https://ampsphere.big-data-biology.org/home

1. Click "home"
2. Click "browse"
3. Click "home"
4. ...

Everytime you cycle, a counter gets incremented. Also, clicking OK once should make the warning go away forever.

@celio

Database statistics for the home page are very slow when calculation, this should be accelerated by create a statistics table in the database. Also, API will directly retrieve the data from the table, not real-timely calculate using all data in the database.

Pages (frontend)

• AMP presentation
• family presentation
• level (selecting by each level of the microontology?)
• search result
• data browsing (selecting by species or habitat)

Backend

• demo database using SQLite
• data retrieval from the database
• posting necessary data to the frontend

https://plotly.com/python/distplot/

--- curve distplots where we have AMPSphere curve, family curve, and an array pointing to the site of the current AMP should be.

Is your feature request related to a problem? Please describe.

Right now, the Download tab is empty for families with fewer than 8 elements. At least the sequences FASTA can be provided.

• Example: https://ampsphere.big-data-biology.org/family?accession=SPHERE-III.008_711

The information is there (the sequences are displayed already), the file can even be generated in JS.

TODO

Change the behavior of get_page_info function in crud.py.

def get_page_info(kind: str, page_size: int, page: int, db: Session,
                  accession: str = None, search_text: str = None):
    if kind == 'amp':
        total_items = db.query(models.AMP.accession).count()

So far I have tried but had no success.

• Caddy to enable automatic HTTPS.

(base) huichong@ip-172-31-43-121:~/projects/caddy-learn$ sudo caddy run                                                                                                    │··································
[sudo] password for huichong:                                                                                                                                              │··································
2021/11/30 10:49:53.039 INFO    using adjacent Caddyfile                                                                                                                   │··································
2021/11/30 10:49:53.040 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127│··································
.0.0.1:2019"]}                                                                                                                                                             │··································
2021/11/30 10:49:53.040 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "ht│··································
tps_port": 443}                                                                                                                                                            │··································
2021/11/30 10:49:53.040 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}                                                            │··································
2021/11/30 10:49:53.041 INFO    http    enabling automatic TLS certificate management   {"domains": ["huichong.me"]}                                                       │··································
2021/11/30 10:49:53.057 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc000340af0"}                                          │··································
2021/11/30 10:49:53.058 INFO    tls     cleaning storage unit   {"description": "FileStorage:/root/.local/share/caddy"}                                                    │··································
2021/11/30 10:49:53.061 INFO    tls     finished cleaning storage units                                                                                                    │··································
2021/11/30 10:49:53.427 INFO    autosaved config (load with --resume flag)      {"file": "/root/.config/caddy/autosave.json"}                                              │··································
2021/11/30 10:49:53.427 INFO    serving initial configuration

• Certbot using huichong.me as a test domain

(base) huichong@ip-172-31-43-121:~/projects/caddy-learn$ sudo certbot certonly --nginx                                                                                     │··································
Saving debug log to /var/log/letsencrypt/letsencrypt.log                                                                                                                   │··································
Please enter the domain name(s) you would like on your certificate (comma and/or                                                                                           │··································
space separated) (Enter 'c' to cancel): huichong.me                                                                                                                        │··································
Requesting a certificate for huichong.me                                                                                                                                   │··································
                                                                                                                                                                           │··································
Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:                                                     │··································
  Domain: huichong.me                                                                                                                                                      │··································
  Type:   unauthorized                                                                                                                                                     │··································
  Detail: Invalid response from http://huichong.me/.well-known/acme-challenge/Z53Fx14Ma4rbAXW9VDO8FbqNehgcJOIBx5Jzus4A-I4 [185.199.111.153]: "<!DOCTYPE html>\n<html>\n  <h│··································
ead>\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\">\n    <meta http-equiv=\"Co"                                                              │··································
                                                                                                                                                                           │··································
Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that i│··································
t is accessible from the internet.                                                                                                                                         │··································
                                                                                                                                                                           │··································
Some challenges have failed.                                                                                                                                               │··································
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

This will eat a lot of time. I cannot estimate how much. Any ideas?

It could be better.

Is your feature request related to a problem? Please describe.
The current design is confusing: it is not clear to the user what these features are.

Describe the solution you'd like
A single search box, but two output tabs.

Using the command below will accelerate mmseqs search by a factor of 100, and can avoid timeout when waiting for search response.

run in advance

• createdb

mmseqs createdb AMPSphere_v.2021-03.faa AMPSphere_v.2021-03.mmseqsdb

• createindex

mmseqs createindex AMPSphere_v.2021-03.mmseqsdb tmp

run when searching

mmseqs createdb input.fasta input.mmseqsdb && \
mmseqs search input.mmseqsdb AMPSphere_v.2021-03.mmseqsdb search_results.mmseqsdb tmp && \
mmseqs createtsv input.mmseqsdb AMPSphere_v.2021-03.mmseqsdb search_results.mmseqsdb search_results.tsv

Is your feature request related to a problem? Please describe.

For example, the page
https://ampsphere.big-data-biology.org/family?accession=SPHERE-III.003_141

could have the logo

Describe the solution you'd like

See the logo.

Description
The CSV downloaded from the Browse page is unnecessarily large and the format is not CSV-friendly at all. Maybe selecting just some of the columns and the sequences per page makes a better result, or at least, provides a more readily parsed file.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Browse'
2. Click on 'Download CSV'
3. See error

Expected behavior
A better format for the CSV file, easier to parse, and virtually smaller in size.

Screenshots

Desktop (please complete the following information):

• OS: Linux
• Browser: Firefox

#!/usr/bin/env python

import sys
from Bio import AlignIO
from Bio.Align import AlignInfo

alignment = AlignIO.read(sys.argv[1], 'fasta')
summary_align = AlignInfo.SummaryInfo(alignment)
summary_align.dumb_consensus(float(sys.argv[2]))

From Biopython documentation (https://biopython.org/docs/1.76/api/Bio.SeqUtils.ProtParam.html), I noticed that the three probabilities of the helix, turn and sheet are independent. There is one amido acid 'L' that can involve in both helix and sheet. So, we cannot treat the resting probability as 'disordered'.

The secondary structure pie chart and error bars chart should be changed.

Automatically fit the content of each page to the window size.

calling it 'microbial host' seems better

Is your feature request related to a problem? Please describe.
Filter by the microbial source isn't practical enough.

Describe the solution you'd like
Change microbial sources to adopt GTDB taxonomic classification tree.

Describe alternatives you've considered
No

Additional context
Helpful resource:

• https://github.com/nick-youngblut/gtdb_to_taxdump
• https://github.com/rrwick/Metagenomics-Index-Correction

Is your feature request related to a problem? Please describe.
Yeah, to delete text search bar for a while, we need a new layout.

Describe the solution you'd like
Maybe something like

<div class="row">
  <div class="q-col-md-1 q-col-12"></div>   # for the logo
  <div class="q-col-md-11 q-col-12"></div>  # for the mavigation bar, maybe remove the bg color to make it clearer.
</div>

Describe the bug

Downloading sphere files results in a redirect to 127.0.0.1 (which then fails).

To Reproduce
Steps to reproduce the behavior:

1. Go to https://ampsphere.big-data-biology.org/family?accession=SPHERE-III.002_399
2. Click on Downloads tab
3. Click on any of the Download links

Expected behavior

Download a file

Desktop (please complete the following information):
Chrome on Linux

Use https://shields.io/ or https://badgen.net/ to generate the badges.

Reference colors:

• Gold: FFD700
• Silver: C0C0C0
• Bronze: CD7F32

Describe the bug

HMM logo images are incomplete?

To Reproduce

See
https://ampsphere-api.big-data-biology.org/v1/families/SPHERE-III.006_036/downloads/SPHERE-III.006_036.png

Desktop (please complete the following information):

Chrome on Linux, but I see the error with wget/curl too

HMM logos are currently PNGs, but they clearly should be SVGs.

This is a bug, causing empty distribution graphs on the amp page.

Example: AMP10.007_686 and AMP10.000_732

Use opacity of graph data and centered text to display empty graph.

• 4 hours

• Open graph engine as a tool.

• fix the colors in geo-distribution map (should be consistent with the habitat sunburst plot).