Code Monkey home page Code Monkey logo

benchmarking_platform's Introduction

Benchmarking Platform

original version presented in S. Riniker, G. Landrum, J. Cheminf., 5, 26 (2013), DOI: 10.1186/1758-2946-5-26, URL: http://www.jcheminf.com/content/5/1/26

extended version presented in S. Riniker, N. Fechner, G. Landrum, J. Chem. Inf. Model., 53, 2829, (2013), DOI: 10.1021/ci400466r, URL: http://pubs.acs.org/doi/abs/10.1021/ci400466r

GENERAL USAGE NOTES

The virtual-screening process implemented by the benchmarking platform is divided into three steps:

  1. Scoring

  2. Validation

  3. Analysis

The three steps are run separately and read in the output of the previous step. In the scoring step, the data from the directories compounds and query_lists is read in.

The directory compounds contains lists of compounds for 118 targets from three public data sources: MUV, DUD and ChEMBL. The compound lists contain the external ID, the internal ID and the SMILES of each compound.

There are three subsets of targets available:

subset I: 88 targets from MUV, DUD & ChEMBL described in J. Cheminf., 5, 26 (2013)

subset I filtered: 69 targets from MUV, DUD & ChEMBL filtered for difficulty described in JCIM (2013), online

subset II: 37 targets from ChEMBL designed for a second VS use case described in JCIM (2013), online

The directory query_lists contains training lists for each target with the indices of randomly selected active and inactive molecules. Training lists with 5, 10 or 20 active molecules are available. The number of training decoys is 20 % of the decoys for subsets I and 10 % for subset II.

The scripts are written in Python and use the open-source cheminformatics library RDKit (www.rdkit.org) and machine-learning library scikit-learn (www.scikit-learn.org).

Running a script with the option [--help] gives a description of the required and optional input parameters of the script.

benchmarking_platform's People

Contributors

greglandrum avatar sriniker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

benchmarking_platform's Issues

Where the score is the same, actives will rank higher than inactives

In the course of using this benchmark, I just recently noticed a small error, regarding the line:

scores[fp].append(sorted(single_score[fp], reverse=True))

...which occurs in several similarly-named Python scripts.

Since single_score[fp] is a tuple of (simscore, id, active/inactive), it does indeed rank first by similarity, but then it ranks by Id, and the actives have Ids with 'A' in them instead of 'D' for the decoys, and so rank higher (when the similarity is the same). However, even just sorting by the similarity is not sufficient to avoid this problem, as Python sort is a stable sort, and the actives are added to the list first, and so will always occur ahead of the decoys. In other words, a random shuffle is needed first. Here is a potential fix:

# random.seed(1) at the top of the file
random.shuffle(single_score[fp])
scores[fp].append(sorted(single_score[fp], reverse=True, key=lambda x:x[0]))

Random selection missing?

I've been setting up the original benchmarking platform (from 5 months ago) in order to do some testing of a similarity measure we have here at NextMove.

At least for the code checked into github, I can't see where the random selection for the query is being made. Am I missing something, or was this never checked in?

ImportError: No module named Pharm2DFP

Hi, I get an error when trying to run the fingerprint_lib.py.

I use the following command from ipython notebook. This error is reproducible for me because I did with the Desktop and also with the laptop.

%run fingerprint_lib.py
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/Users/sawsimeon/Downloads/1758-2946-5-26-s4/scoring/fingerprint_lib.py in <module>()
     10 from rdkit.Chem.Fingerprints import FingerprintMols
     11 from rdkit.Chem.ChemicalFeatures import BuildFeatureFactory
---> 12 from rdkit.Chem.Pharm2DFP import Pharm2DFPCalculator
     13 from rdkit.Chem import rdMolDescriptors
     14 

ImportError: No module named Pharm2DFP

hello

Hi~
what is the inverse of the function GetMorganFingerprintAsBitVect()?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.