genomejs / dna2json Goto Github PK

Formats your genome file as JSON

License: MIT License

JavaScript 100.00%

dna2json's Introduction

Information

Package	dna2json
Description	Formats your DNA file as JSON
Node Version	>= 0.4

Terminology

SNP

A SNP (Single-Nucleotide Polymorphism - pronounced snip) is a variation of a single nucleotide (A, G, C or T) at some location in your genome. The presense of particular variants, or groups thereof, can tell you a lot about yourself (hair color, height, muscle type, allergies, response to disease, response to pharmaceuticals, heritage, etc.).

Personal genome vendors will typically produce an analysis that covers some of these attributes - but will also provide you with 'raw' data, should you wish to investigate further.Your 'raw' data from any vendor will be a list of several hundred thousand SNPs, located across the autosomes (chromosomes 1-22), the sex chromosomes (X and Y), and possibly the mitochondrial chromosome. Given that your genome contains two copies of each autosome, and either XX (female) or XY (male), the data contains a 'genotype' composed of the two variants at each location.

Vendors

Name	Supported	Price	Sample	Autosomal SNPs	Y SNPs	X SNPs	MT SNPs	Raw Data
23andMe	Yes	199/99 USD^*	Saliva	967,000	3,089	26,087	2,737	Yes
ancestryDNA	Yes	99 USD	Saliva	682,549	885	17,604	0	Yes
FamilyTree	Yes	99 USD	Cheek Swab	708,092	0	18,091	0	Yes

^* 23andMe picing is 199 USD domestically and 99 USD abroad.

CLI Usage

Use this if you just want to convert your data to the correct format so you can start querying your genome.

$ npm install dna2json -g
$ dna2json
Usage: dna2json <input file> <output file>
$ dna2json dna.txt dna.json
This will take a while...

Module Usage

var dna = require('dna2json');
var fs = require('fs');

var txt = fs.readFileSync('dna.txt');
dna.parse(txt, function(err, snps){
  // snps = the object with your mutations
});

SNP-JSON

Every vendor has their own format for your DNA. I decided to make a standard format called SNP-JSON. This library will convert custom formats to this standard.

SNP-JSON looks like this

{
  "rs4477212": {
    "chromosome": 1,
    "genotype": "AA"
  },
  "rs3094315": {
    "chromosome": 1,
    "genotype": "AA"
  },
  "rs3131972": {
    "chromosome": 1,
    "genotype": "GG"
  },
  "rs12124819": {
    "chromosome": 1,
    "genotype": "AA"
  },
  "rs11240777": {
    "chromosome": 1,
    "genotype": "GG"
  },
  "rs6681049": {
    "chromosome": 1,
    "genotype": "CC"
  },
  ...
}

Explanation:

The key is the RSID or vendor ID.
Chromosome is which chromosome the data came from (1-22, X, Y, or MT).
Genotype is the value for the RSID.

Differences between formats

SNP-JSON, SNPedia and vendor formats use different notations to indicate indels and no-calls.

Format	Insertion	Deletion	No Call	Reference
23AndMe	`I`	`D`	`-`	https://customercare.23andme.com/hc/en-us/articles/212196888-What-does-not-determined-or-not-genotyped-mean-
AncestryDNA	`I`	`D`	`0`	https://www.ancestry.com/dna/en/legal/us/faq#raw-6
FamilyTreeDNA	`I`	`D`	`-`	https://www.familytreedna.com/learn/autosomal-ancestry/universal-dna-matching/read-family-finder-raw-data-file/
SNP-JSON	`I`	`-`	`?`
SNPedia	(the actual letters)	`-`	N/A	https://www.snpedia.com/index.php/Talk:Rs5030655

Using your SNP-JSON

Just Write Code™

In this example we will determine if you are immune to norovirus:

var dna = require('./dna.json');

if (dna.rs601338.genotype == 'AA') {
  // congrats you are immune!
}

Genosets

Once you have your data in the right format you can use the library of genosets by genomejs.

You can view a list of these by searching npm for genoset

You can also publish your own tools that analyze your DNA! Here is a tool that we recommend to get you started GQL

LICENSE

(MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

dna2json's People

Contributors

Stargazers

Watchers

dna2json's Issues

AncestryDNA only returns one allele per rs#?

Please excuse my ignorance, if this question is ill-informed.

I have a sample TXT file from AncestryDNA. Unlike the other two vendors, it looks like they list the two alleles for each rs# in separate columns. As a result, my JSON file only contains one allele per rs#. The result looks like this:

"rs4477212": { "chromosome": 1, "genotype": "T" }, "rs3131972": { "chromosome": 1, "genotype": "A" }, "rs12562034": { "chromosome": 1, "genotype": "G" }

Is this an error on my part, or a limitation with AncestryDNA's TXT file formatting?

Thanks!

Demo Genome

Is there a demo genome out there that can be used to test with the library? I think this would be super helpful to people trying to get test out this project.

Preferably a demo genome from each of the services listed in the readme (23andMe, ancestryDNA, FamilyTree).

Extremely Long Parse Time

I tried to parse my genome from the text file downloaded from 23andMe. It appears to get stuck while parsing the SNPs. I let the script run for 3 hours and it appeared to be stuck on "Parsed 960613 SNPs".

Is there an error log or debug feature that I can use to find out what is happening? The example listed in the readme says "This will take awhile..." What is the average time it takes to parse a genome?

Some other information that may be useful:
genome file downloaded from 23andMe is 24 MB
After stopping the data2json process after 3 hours, the json file was 34 MB.

Thanks for any help you can provide

How do you write genosets?

Where I can find a guide/tutorial/reference to write useful GQLs queries for genosets?

Collaboration with Biojs

Hi genomejs, hi @contra

I am one of the dudes of BioJS.
Basically we try to write reusable JavaScript for life sciences with a special focus on visualization - so I do think that some of our goals overlap and that both of us could profit beneficial from each other. Here is a raw list of ideas that come to mind just after discovering your project:

Use existing visualizations <-> or share yours with the community
List your parsers on the BioJS registry (we have built a registry where people can share and discover bio* related components)
Work on an easy-to-embed genome browser
Or maybe you have even an amazing idea for a GSoC project.

I do believe there are more even ways to work together - so we are very open for your ideas :)

Cheers,
Seb

Other BioJS dudes that are interested and would be happy to help include at least @daviddao, @manuelcorpas, @selewis, @nathundunn, @ljgarcia, @saketc.

Using JSONStream

Thanks for the incredible work on this library. Also, thanks for the great usage documentation. I'm new to coding, and was able to quickly get this working in both CLI and a node app.

Quick question. I saw one example on a blog where someone had implemented dna2json with JSONStream, like so:

var dna = require('dna2json');
var JSONStream = require('JSONStream');
var fs = require('fs');

fs.createReadStream("./data/23andme-male.txt")
.pipe(dna.createParser())
.pipe(JSONStream.stringify())
.pipe(fs.createWriteStream("./data/23andme-male.json"));

Is this necessary? What are the pros/cons?

fastq format?

does this library support the fastq format:

https://en.wikipedia.org/wiki/FASTQ_format

Genes for Good

Hey, do you have any interest in adding a parser for Genes for Good? It's a group at University of Michigan that offers free genotyping.

They provide users with text files formatted in the same manner as 23andMe. Because of that, it should be pretty straightforward to create a new parser based on the one already written for 23andMe. If you are interested, I could test it out with some Genes for Good TXT files I have from a friend.

SNP-JSON format

From what I can see in the demo genomes, all vendors provide as chromosomal location, is there any reason why this isn't included a a property in the SNP-JSON.

SAM/BAM file support

Specs are at http://samtools.sourceforge.net/

JSONStream is a production dependency as well as a development dependency

I had to manually install JSONStream, to complete the conversion instructions in the README. should move the dependency declaration outside of only development.

23andMe deletions

23andMe indicates deletions with D. Other groups use other standards.

genomejs/gql#4 (comment)
genomejs/gql#4 (comment)

Orientation issues

I'm still learning and re-learning all this sequencing stuff, so this may or may not be an issue. Has orientation been considered for the three vendors listed in the readme? I've read that 23andMe reports genotype information based on the plus strand of GRCh37, while some other companies use the minus strand (I'm not sure what FamilyTree and AncestryDNA use). If that's the case, this would need to be accounted for, since use of the opposite strand would require that all A/T, and all G/C nucleobases be 'flipped' for queries to be accurate. (This is discussed more on SNPedia.)

Is this already known and accounted for? Thanks!