Comments (6)
Hi,
This week I'm working on improving the speed so I'm happy to have a look at your example and experiment with different settings both on the client and server side. Can you please give a list of ~5k targets that give take you 2m45s to retrive activities?
from chembl_webresource_client.
It appears that I goofed, and the slow part isn't the data retrieval, but the conversion/looping-through the list of dicts to just a list of lists of the relevant properties:
for x in chembl_ids:
if x not in existing_files:
print("Checking CHEMBL id {}".format(x))
res = activity.filter(target_chembl_id='{}'.format(x), relation='=', assay_type='B', standard_type="IC50")
print("Data retrieved")
for i,z in enumerate(res):
if z['parent_molecule_chembl_id'] and z['canonical_smiles'] and z['standard_value'] is not None:
temp.append([str(z['parent_molecule_chembl_id']), str(z['canonical_smiles']), float(z['standard_value'])])
write_csv(x, temp)
The data retrieval itself looks to take ~1-2s, but the conversion takes the remaining few minutes, at least.
from chembl_webresource_client.
OK, in that case is there anything I can help you here?
from chembl_webresource_client.
BTW: if you just rerun your script and noticed it's much faster it can because now the results are cached on the client side so you may concider clearing cache or configuring the client not to use the cache during profiling.
from chembl_webresource_client.
Ah, thanks for the clarification. Didn't know things were getting cached on our side--it still seems that that bulk of the time is in processing of the dict being returned, so I'll close this out!
from chembl_webresource_client.
No worries, I'm still optimising the API performance, next week it should work smoother but don't expect any dramatic improvements.
from chembl_webresource_client.
Related Issues (20)
- .filter for different names in "document_journal" column HOT 1
- Query hangs after reaching specific entry HOT 1
- Error for url https://www.ebi.ac.uk/chembl/api/data/activity.json, server response: <!doctype html> HOT 6
- Mechanism results do not match what is on the website HOT 3
- Isomeric SMILES string
- Problem with unichem.structure HOT 4
- Http Application error HOT 5
- Querying by inchi
- new_client is failing to import from chembl_webresource_client HOT 1
- Assay offsets and limit HOT 2
- How do I retrieve the Unichem Cross references as a dictionary/list for a CHEMBL compound?
- Fail to import new_client due to API problem HOT 5
- Fixed
- Space between words of the query
- HttpApplicationError HOT 8
- confidence score
- Error for url
- ChEMBL webresourse client issue
- status 500 with chembl_webresource_client.new_client
- Problem of parallelized big data mining via batch chunks
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chembl_webresource_client.