Code Monkey home page Code Monkey logo

Comments (1)

jkoshy avatar jkoshy commented on August 22, 2024

Python's cProfile module was used to profile an ingestion run.

A faster JSON decoder may help.

>>> p.sort_stats('time').print_stats(25)
Wed Apr 20 22:52:58 2011    prof.1

       606063949 function calls (538093040 primitive calls) in 2355.761 CPU seconds

   Ordered by: internal time
   List reduced from 569 to 25 due to restriction <25>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
2894113/2423987  293.772    0.000 1037.676    0.000 /usr/lib/python2.6/json/decoder.py:162(JSONObject)
62802266/179064  252.699    0.000 1249.725    0.007 /usr/lib/python2.6/json/scanner.py:38(iterscan)
  1841908  236.037    0.000  236.037    0.000 {built-in method acquire}
    89532  191.165    0.002  191.165    0.002 {method 'sendall' of '_socket.socket' objects}
   186671  100.853    0.001  100.853    0.001 {method 'recv' of '_socket.socket' objects}
 33943035   86.064    0.000   86.064    0.000 {_json.scanstring}
4965746/89532   84.138    0.000 1247.784    0.014 /usr/lib/python2.6/json/decoder.py:206(JSONArray)
 12299462   83.452    0.000  131.961    0.000 /usr/lib/python2.6/json/decoder.py:60(JSONNumber)
  2423987   81.286    0.000   81.286    0.000 ./apiserver/osmelement.py:138(from_mapping)
   897109   73.387    0.000  143.122    0.000 ./dbmgr/dbm_input.py:39(_make_osm_iterator)
128843635   67.806    0.000   67.806    0.000 {built-in method end}
 50502797   52.652    0.000   52.652    0.000 {built-in method scanner}
   897497   50.951    0.000   94.286    0.000 ./dbmgr/dbm_stats.py:98(increment_stats)
   153426   50.850    0.000  212.364    0.001 ./datastore/lrucache.py:258(insert_slab)
 31401133   46.769    0.000   46.769    0.000 {built-in method match}
 11241805   45.127    0.000   91.058    0.000 /usr/lib/python2.6/json/decoder.py:152(JSONString)
  3330996   32.307    0.000   76.072    0.000 ./apiserver/osmelement.py:116(__init__)
   153426   32.257    0.000   48.994    0.000 ./datastore/lrucache.py:233(_remove_slab_items)
   387550   27.715    0.000   27.742    0.000 {map}
  4443187   27.343    0.000   27.343    0.000 ./datastore/slabutil.py:35(_make_numeric_slabkey)
 43465635   24.734    0.000   24.734    0.000 {getattr}
   632818   24.690    0.000   91.443    0.000 /usr/lib/python2.6/threading.py:116(acquire)
 43700595   24.624    0.000   24.624    0.000 {built-in method span}
34895320/34895075   17.521    0.000   17.521    0.000 {len}
   897108   17.395    0.000 2285.756    0.003 ./dbmgr/dbm_ops.py:50(add_element)

from mapquest-osm-server.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.