Code Monkey home page Code Monkey logo

cfn-ps-illumina-dragen's Introduction

quickstart-illumina-dragen

DRAGEN on the AWS Cloud

This Quick Start deploys Dynamic Read Analysis for GENomics Complete Suite (DRAGEN CS), a data analysis platform by Illumina, on the AWS Cloud in about 15 minutes.

DRAGEN CS enables ultra-rapid analysis of next-generation sequencing (NGS) data, significantly reduces the time required to analyze genomic data, and improves accuracy. It includes bioinformatics pipelines that provide highly optimized algorithms for mapping, aligning, sorting, duplicate marking, and haplotype variant calling. These pipelines include DRAGEN Germline V2, DRAGEN Somatic V2 (Tumor and Tumor/Normal), DRAGEN Virtual Long Read Detection (VLRD), DRAGEN RNA Gene Fusion, DRAGEN Joint Genotyping, and GATK Best Practices.

The Quick Start builds an AWS environment that spans two Availability Zones for high availability, and provisions two AWS Batch compute environments for Spot Instances and On-Demand Instances. These environments include DRAGEN F1 instances that are connected to field-programmable gate arrays (FPGAs) for hardware acceleration.

The Quick Start offers two deployment options:

  • Deploying DRAGEN into a new virtual private cloud (VPC) on AWS
  • Deploying DRAGEN into an existing VPC on AWS

You can also use the AWS CloudFormation templates as a starting point for your own implementation.

Quick Start architecture for DRAGEN on AWS

For architectural details, best practices, step-by-step instructions, and customization options, see the deployment guide.

To post feedback, submit feature ideas, or report bugs, use the Issues section of this GitHub repo. If you'd like to submit code for this Quick Start, please review the AWS Quick Start Contributor's Kit.

cfn-ps-illumina-dragen's People

Contributors

kuanyili avatar vsnyc avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

cfn-ps-illumina-dragen's Issues

Deployment Guide error

Hey all, I am facing some issues with dragen deployment using the guide here - https://aws-ia.github.io/cfn-ps-illumina-dragen/.
I have done this before, but its been some time I and don't remember having this problem.
I just ran with "deploy in new VPC" link, here are my parameters, as seen on CloudFormation stack.

image

But I keep getting the following error

image

It looks like it fails to copy something into the bucket (or cant find source/destination?).

Since its a relatively fresh account and I didn't specify anything extra in template creation, such as IAM (I just left it empty), its strange that it fails here.

Could it be its missing from source for copying?
What direction can I take for this?

Thanks!

single app mode waiting to run

Hey,

When attempting to run either the somatic or germline pipeline in Dragen, the program displays a screen and remains stuck on it. Do you know how I can ı resolve this?
image

DRAGEN error using Deployment Guid

Hi, I am using the DRAGEN Complete Suite and using the AWS CloudStack deployment guide. When I submit a batch job, all the pre-processing steps seems to work (i.e., reference downloaded from S3 bucket, etc...) but when DRAGEN runs, it exits with the following error message:

  2024-01-09T09:54:46.188-08:00 ERROR: The following extra command line options are not recognized
  2024-01-09T09:54:46.188-08:00Copy > /ephemeral/9518aff5-e295-4e05-802a-f4365af3b4bd/dragen_log_1704822886.txt 2>&1 > /ephemeral/9518aff5-e295-4e05-802a-f4365af3b4bd/dragen_log_1704822886.txt 2>&1

My guess is somehow the last parameters are being passed as arguments directly to DRAGEN versus being treated as I/O pipes. Here's the command being run by the DRAGEN stack:

Executing /opt/edico/bin/dragen -f -r /ephemeral/DRAGEN/hg38/ -1 s3://XXX/temp/NA24385-AJ-Son-R1-NS_S33_L001_R1_001.fastq.gz -2 s3://XXX/temp/NA24385-AJ-Son-R1-NS_S33_L001_R2_001.fastq.gz --RGID 1 --RGSM Test --enable-bam-indexing true --enable-map-align-output true --enable-sort true --output-file-prefix output --enable-map-align true --output-format BAM --output-directory /ephemeral/9518aff5-e295-4e05-802a-f4365af3b4bd --enable-variant-caller true --output_status_file /ephemeral/9518aff5-e295-4e05-802a-f4365af3b4bd/job-speedometer.log --intermediate-results-dir /ephemeral/ --lic-no-print > /ephemeral/9518aff5-e295-4e05-802a-f4365af3b4bd/dragen_log_1704822886.txt 2>&1

I am not sure how this is happening as I am using all the default settings. Any help would be appreciated.

Deploy DRAGEN error using guide

Hi,
I followed the Illumina DRAGEN on AWS Partner Solution Deployment Guide to deploy DRAGEN, but encountered a error during creating stack(DragenStack-ContainerBuild stage).

Some part of error info from codepipeline log file like below

Step 1 : FROM public.ecr.aws/docker/library/centos:centos7.9.2009
114 | centos7.9.2009: Pulling from docker/library/centos
115 | 2d473b07cdd5: Pulling fs layer
116 | 2d473b07cdd5: Verifying Checksum
117 | 2d473b07cdd5: Download complete
118 | 2d473b07cdd5: Pull complete
119 | Digest: sha256:be65f488b7764ad3638f236b7b515b3678369a5124c47b8d32916d6487418ea4
120 | Status: Downloaded newer image for public.ecr.aws/docker/library/centos:centos7.9.2009
121 | ---> eeb6ee3f44bd
122 | Step 2 : RUN rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
123 | ---> Running in a9fb22e4266c
124 | curl: (22) The requested URL returned error: 404 Not Found
125 | error: skipping https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm - transfer failed
126 | Retrieving https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
127 | The command '/bin/sh -c rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm' returned a non-zero code: 1
128 |  
129 | [Container] 2024/08/14 23:53:24.507010 Command did not exit successfully docker build --tag ${REPOSITORY_URI}:${TAG} . exit status 1
130 | [Container] 2024/08/14 23:53:24.510321 Phase complete: BUILD State: FAILED
131 | [Container] 2024/08/14 23:53:24.510339 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build --tag ${REPOSITORY_URI}:${TAG} .. Reason: exit status 1
132 | [Container] 2024/08/14 23:53:24.549467 Entering phase POST_BUILD
133 | [Container] 2024/08/14 23:53:24.550509 Running command docker push ${REPOSITORY_URI}:${TAG}
134 | The push refers to a repository [654724916652.dkr.ecr.us-east-1.amazonaws.com/dragen-new-t1-dragenstack-10ln435tg3mi-dockerbucketrepository-p2y6xi7v4qt2-dragen-c4cpmz9xwdp8]
135 | An image does not exist locally with the tag: 654724916652.dkr.ecr.us-east-1.amazonaws.com/dragen-new-t1-dragenstack-10ln435tg3mi-dockerbucketrepository-p2y6xi7v4qt2-dragen-c4cpmz9xwdp8
136


I followed the "the Deployment steps" in guide document with default setting. Could you please help to provide some suggestions on this error?
I used AWS Region us-east-1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.