Code Monkey home page Code Monkey logo

momoko's Introduction

Momoko

image

image

image

Momoko wraps Psycopg2's functionality for use in Tornado. Have a look at tutorial or full documentation.

Important: This is the 2.x version of Momoko. It requires 4.0 <= Tornado < 6.0, uses futures instead of calllbacks and introduces a slightly different API compared to 1.x version. While transition is very straightforward, the API is not backward compatible with 1.x!

Maintainer wanted

Unfortunately none of the developers of this project actively use it anymore in their work. Test-covered pull requests will be happily accepted, but no active development is planned so far. If you have serious intentions to maintain this project, please get in touch.

Installation

With pip:

pip install momoko

Or manually:

python setup.py install

Testing

Set the following environment variables with your own values before running the unit tests:

make -C tcproxy
export MOMOKO_TEST_DB='your_db'
export MOMOKO_TEST_USER='your_user'
export MOMOKO_TEST_PASSWORD='your_password'
export MOMOKO_TEST_HOST='localhost'
export MOMOKO_TEST_PORT='5432'

And run the tests with:

python setup.py test

momoko's People

Contributors

beregond avatar bitwolaiye avatar boringcrypto avatar cswingley avatar eggspurt avatar friedcell avatar fsx avatar haizaar avatar jbowes avatar jchumnanvech avatar kamelzcs avatar mpyatishev avatar nkonin avatar peterbe avatar tesh11 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

momoko's Issues

an error for installation in python 2.5

(mypy)hzg@debian:~$ pip install momoko
Downloading/unpacking momoko
Downloading Momoko-0.3.0.tar.gz (109Kb): 109Kb downloaded
Running setup.py egg_info for package momoko
Installing collected packages: momoko
Running setup.py install for momoko
File "/home/hzg/mypy/lib/python2.5/site-packages/momoko/client.py", line 365
self._db.execute(*query, callback=self._collect)
^
SyntaxError: invalid syntax

Successfully installed momoko

Exception when db connection failed and reconnect.

Hi, all

I am trying to perform some reconnect-procedures of my program.
Assume the DB is down at startup, I will try to reconnect the db when I found the db is down which means the _pool size is 0 at the execution point.

def init_setting(self, db_host, db_port, db_name, user_name, user_pwd, pool_size):
        """ Initialize the DB connect setting
        """

        self.cache_dsn = 'dbname=%s user=%s password=%s host=%s port=%s' % (db_name, user_name, user_pwd, db_host, db_port)
        self.cache_size = pool_size

        self.db = momoko.Pool(dsn=self.cache_dsn, size=self.cache_size)

@gen.coroutine    
    def execute(self, *commands):
        """ execute commands
            Args:
                commands: {tuple of SQL commands}
        """
        error_occur = False
        while len(self.db._pool) == 0 or error_occur:
            logger.debug("Connection Pool is empty!! Re-connect the db. cmd=%s", commands)
            yield gen.Task(IOLoop.instance().add_timeout, time.time() + 2)
            try:
                self.db = momoko.Pool(dsn=self.cache_dsn, size=self.cache_size)
            except:
                error_occur = True
                logger.debug("Reconnect the DB Failed!!")
            yield gen.Task(IOLoop.instance().add_timeout, time.time() + 1)

       # ...... skip the rest
       result = yield momoko.WaitAllOps(tuple(op_name_list))
       raise gen.Return(result[-1])

When I run the execute function above, when the pool size is 0, I will try to re-initialize the momoko.Pool and I always get the following exception.

Traceback (most recent call last):
  File "/lib/python2.7/site-packages/tornado/web.py", line 1115, in _stack_context_handle_exception
    raise_exc_info((type, value, traceback))
  File "/lib/python2.7/site-packages/tornado/stack_context.py", line 343, in _handle_exception
    if tail.exit(*exc):
  File "/lib/python2.7/site-packages/tornado/stack_context.py", line 186, in exit
    return self.exception_handler(type, value, traceback)
  File "lib/python2.7/site-packages/tornado/gen.py", line 204, in handle_exception
    future.set_exc_info((typ, value, tb))
  File "/lib/python2.7/site-packages/tornado/concurrent.py", line 125, in set_exc_info
    self.set_exception(exc_info[1])
  File "/lib/python2.7/site-packages/tornado/concurrent.py", line 87, in set_exception
    self._set_done()
  File "/lib/python2.7/site-packages/tornado/concurrent.py", line 95, in _set_done
    for cb in self._callbacks:
TypeError: 'NoneType' object is not iterable

Assume the DB is down during the above test.
I found out the problem is the connection recreation: "momoko.Pool" inside the while loop of execute function. When I comment the out, the exception is gone and also cannot reconnect .

And if the DB is down at startup and I restart the db before the execute function, the above connection recreation procedure works fine.

Does anyone has any suggestions?

Ideas?

Cheers

Ivan

Query chaining

Chain multiple queries and execute them one by one, passing the results to the next one and/or execute a callback for a certain query when specified.

It's too much work to manually make callbacks in a request handler for queries.

momoko not working with tornado 4.0

Just thought I'll play with latest tornado. Seem that some of the upstream tornado changes (gen.Task is now a function not a class) has broken Momoko. Hope this bug report can help with the fix.

(ve3_t4)➜ momoko git:(master) python setup.py test
Using psycopg2
running test
running egg_info
creating Momoko.egg-info
writing dependency_links to Momoko.egg-info/dependency_links.txt
writing top-level names to Momoko.egg-info/top_level.txt
writing requirements to Momoko.egg-info/requires.txt
writing Momoko.egg-info/PKG-INFO
writing manifest file 'Momoko.egg-info/SOURCES.txt'
reading manifest file 'Momoko.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'Momoko.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "setup.py", line 58, in
'Topic :: Database :: Front-Ends'
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/core.py", line 149, in setup
dist.run_commands()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/Users/martinslabber/Documents/coding/momoko/ve3_t4/lib/python3.4/site-packages/setuptools/command/test.py", line 138, in run
self.with_project_on_sys_path(self.run_tests)
File "/Users/martinslabber/Documents/coding/momoko/ve3_t4/lib/python3.4/site-packages/setuptools/command/test.py", line 118, in with_project_on_sys_path
func()
File "/Users/martinslabber/Documents/coding/momoko/ve3_t4/lib/python3.4/site-packages/setuptools/command/test.py", line 164, in run_tests
testLoader = cks
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 92, in init
self.parseArgs(argv)
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 139, in parseArgs
self.createTests()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 146, in createTests
self.module)
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 146, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 146, in
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 105, in loadTestsFromName
module = import('.'.join(parts_copy))
File "/Users/martinslabber/Documents/coding/momoko/momoko/tests.py", line 44, in
import momoko
File "/Users/martinslabber/Documents/coding/momoko/momoko/momoko/init.py", line 15, in
from .connection import Pool, Connection
File "/Users/martinslabber/Documents/coding/momoko/momoko/momoko/connection.py", line 33, in
from .utils import log
File "/Users/martinslabber/Documents/coding/momoko/momoko/momoko/utils.py", line 28, in
class Op(gen.Task):
TypeError: function() argument 1 must be code, not str
(ve3_t4)➜ momoko git:(master) pip freeze
Momoko==1.1.3
certifi==14.05.14
psycopg2==2.5.3
tornado==4.0

Using Python 3.4.0 on Mac OS X 10.9.4
Steps to reproduce:

  • Created a new virtualenv.
  • Pip install tornado
  • git checkout momoko
  • pip install checkedout momoko
  • run test

Quite sure it is related to this upstream change:
tornadoweb/tornado@518ecf8

Connecting to a database but then not - causes obscure stacktrace

We had a staging issue where the IP address for our node was not in the databases OK list. momoko made the connection, but then ofc it was refused. You can replicate exactly what we got if you try to connect with a bad user name:
See traceback:
https://gist.github.com/kacieh80/644445ef51162e7faada

Please don't judge me, I did not name our service.
We are on 1.0 still. We tried upgrading to the latest version and we got the same error.

So what happens when it connects, but then is kicked out is pools == []. Our hacky solution is to throw out the pools - throw a pretty error in the logs about issues connecting to the database - and try to connect again when the user requests a query. This should really be fixed though. Even though we catch the error thrown by the stacktrace, we then get an OSError file not found here: https://github.com/FSX/momoko/blob/v1.0.0/momoko/connection.py#L257
Because the code isn't aware of being kicked out yet? That's for you guys to figure out.

We did not test the second OS error on line 257 in the new version and I can't give you a traceback as I was troubleshooting on another devs box and this service is in 2.7 and my box only does 3 because pls pls move to 3 everyone! Also we got our IP address in so everything is working now. It would be nice to have this error more gracefully though.

Thanks.

'connection_factory' argument doesn't work

Here is a sample

import tornado.gen
import tornado.ioloop
import momoko
from psycopg2.extras import RealDictConnection, RealDictCursor

@tornado.gen.coroutine
def query():
    db = momoko.Pool('dbname=jvhua user=jvhua', connection_factory=RealDictConnection)
    cur = yield momoko.Op(db.execute, 'SELECT * FROM users;')
    print(cur.fetchone())

tornado.ioloop.IOLoop.instance().run_sync(query)
tornado.ioloop.IOLoop.instance().start()

What it prints is a tuple, while expected a dict.If I add cursor_factory=RealDictCursor to momoko.Op, things work fine.

Momoko tutorial -- Op, WaitOp, WaitAllOp ValueError

Using python 2.7, tornado 3.2 momoko 1.0.0 psycopg2 2.5, with Op, WaitOp, and WaitAllOp examples in tutorial, Response fails with:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tornado/web.py", line 1232, in _when_complete
raise ValueError("Expected Future or None, got %r" % result)
ValueError: Expected Future or None, got <generator object get at 0x1027b8fa0>

Bug? Suggested workaround?

Tests

Tests covering all functionality and fixed bugs.

Pool seems to try to use busy connections

I keep seeing this in my logs:

Traceback (most recent call last):
File "/usr/local/lib/python3.2/dist-packages/tornado/web.py", line 1115, in _stack_context_handle_exception
raise_exc_info((type, value, traceback))
File "", line 3, in raise_exc_info
File "/usr/local/lib/python3.2/dist-packages/tornado/stack_context.py", line 302, in wrapped
ret = fn(*args, **kwargs)
File "/usr/local/lib/python3.2/dist-packages/momoko/connection.py", line 414, in exec_statement
self.execute(operation, parameters, cursor_factory, exec_statement)
File "/usr/local/lib/python3.2/dist-packages/momoko/connection.py", line 289, in execute
cursor.execute(operation, parameters)
psycopg2.ProgrammingError: execute cannot be used while an asynchronous query is underway

It looks to me like Connection.busy() should maybe return true while the Connection is in the middle of a transaction(). Otherwise perhaps you can get into the situation where two asynchronous web requests can end up trying to use the same underlying psycopg connection?

Errors cannot be caught

This is actually 2 issues.

1:

The first is that errors cannot be caught, for example, if I provide the wrong database login and start an AsyncPool, the momoko event handler is just called repeatedly and the IOLoop keeps printing the stack trace "Asynchronous connection failed". A mechanism is required to handle errors gracefully, remove the bad connections from the IOLoop and notify the caller that something went wrong.

2:

The request StackContext is not maintained. Assuming that the AsyncPool can always handle all it's exceptions, this is still a problem, for example:

@gen.engine
def get(self):
    result = yeild gen.Task(pool.new_cursor, 'execute', "select 1")
    # Don't like it
    raise web.HTTPError(404)

The 404 is logged but the client is left hanging, since the StackContext was lost. The fix is wrapping user callbacks with tornado.stackcontext.wrap().

Regards

Asynchronous connection failed when several users test site

ERROR:root:Cannot send error response after headers written
ERROR:root:Uncaught exception POST /url/is/here/ (127.0.0.1)
HTTPRequest(protocol='http', host='localhost:8888', method='POST', uri='/url/is/here/', version='HTTP/1.1', remote_ip='127.0.0.1', body='some_data_here', headers={'Content-Length': '96', 'Connection': 'close', 'Content-Type': 'application/x-www-form-urlencoded', 'Host': 'localhost:8888', 'Accept-Encoding': 'gzip'})
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/tornado/web.py", line 1021, in _stack_context_handle_exception
raise_exc_info((type, value, traceback))
File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line 259, in _nested
yield vars
File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line 229, in wrapped
callback(_args, *_kwargs)
File "/usr/lib/python2.7/site-packages/momoko/pools.py", line 313, in _io_callback
state = self._conn.poll()
OperationalError: asynchronous connection failed

more details here http://stackoverflow.com/questions/14494885/asynchronous-connection-failed-by-momoko

When to commit?

Greetings, I've read through the documents and stumbled upon this note in psycopg2 faqs, namely this one:

Why does psycopg2 leave database sessions “idle in transaction”?
Psycopg normally starts a new transaction the first time a query is executed, e.g. calling cursor.execute(), even if the command is a SELECT. The transaction is not closed until an explicit commit() or rollback().

If you are writing a long-living program, you should probably make sure to call one of the transaction closing methods before leaving the connection unused for a long time (which may also be a few seconds, depending on the concurrency level in your database). Alternatively you can use a connection in autocommit mode to avoid a new transaction to be started at the first command.

So when I use db.execute to run a INSERT or UPDATE statement, should I commit it immediately in another db.execute call? Or should I just ignore it?

I found this isn't mentioned in the docs. So I'd ask it here. Thanks in advance.

OperationalError when polling puts connection in a bad state

I put together simple handler to test momoko and hit it with ApacheBench, and after a certain level of load, it throws an OperationalError in the Poller object and puts the connection in a bad state. I can't make any more queries with the connection, and I have no way to catch the exception to re-establish the connection. Any suggestions?

[E 110526 20:57:16 web:900] Uncaught exception GET /test (127.0.0.1)
    HTTPRequest(protocol='http', host='127.0.0.1:10000', method='GET', uri='/test', version='HTTP/1.0', remote_ip='127.0.0.1', body='', headers={'Host': '127.0.0.1:10000', 'Accept': '*/*', 'User-Agent': 'ApacheBench/2.3'})
    Traceback (most recent call last):
      File "/Users/carlo/Projects/python/lib/python2.6/site-packages/tornado-1.2.1-py2.6.egg/tornado/stack_context.py", line 171, in wrapped
        callback(*args, **kwargs)
      File "/Users/carlo/Projects/python/lib/python2.6/site-packages/momoko/momoko.py", line 394, in _io_callback
        self._update_handler()
      File "/Users/carlo/Projects/python/lib/python2.6/site-packages/momoko/momoko.py", line 383, in _update_handler
        state = self._connection.poll()
    OperationalError: asynchronous connection failed

This is the script:

import logging

# async postgres client
from momoko import Momoko

from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from tornado.options import define, parse_command_line, options
from tornado.web import Application, RequestHandler, asynchronous

class MainHandler(RequestHandler):
    def initialize(self):
        self._db = None

    @property 
    def db(self):
        if not self._db:
            # Setup the db connection when first accessed instead of at 
            # initialization so that each child process gets its own
            # connection.
            self._db = Momoko({
                'host': 'localhost',
                'database': 'db',
                'user': 'user',
                'password': 'password',
                'min_conn': 5,
                'max_conn': 20,
                'cleanup_timeout': 10
            })
        return self._db


    @asynchronous
    def get(self):
        self.db.execute("""
            select * from test 
            order by random()
            limit 1
        """, callback=self._on_response)

    def _on_response(self, cursor):
        columns = [f[0] for f in cursor.description]
        row = cursor.fetchone()
        self.write(dict(zip(columns, row)))
        self.finish()

def start(port, reload_on_change=False, num_processes=None):
    logging.basicConfig(format="[%(asctime)s] %(levelname)s: %(message)s")

    settings = {
        'debug': reload_on_change
    }

    app = Application([
        (r"/test", MainHandler),
    ], **settings)

    logging.info('Starting test on port {0}'.format(port))
    server = HTTPServer(app)
    server.bind(port)
    server.start(num_processes)
    IOLoop.instance().start()

def main():
    define("port", type=int, default=10000, help="Port to listen on")
    define("reload", type=bool, default=False, help="Automatically restart the server when code changes are detected") 
    define("num_processes", type=int, default=None, help="Number of child processes to prefork. Defaults to the number of cpus on the machine.") 

    parse_command_line()

    start(
        port=options.port, 
        reload_on_change=options.reload,
        num_processes=options.num_processes
    )

if __name__ == '__main__':
    main()

Exception connection reuse bug

I am new to Python and Tornado and I am not sure if this is a bug or I am doing some mistake. I have source code where I send SQL to PostgreSQL database and get back an exception, I catch it and send to the browser appropriate json - everything is quite OK. If I wait 10 seconds until connection cleanup, I get a new connection from pool and everything is OK again. But if I wait only 2 seconds I get unexpected error "Cannot send error response after headers written". Following is minimal source code that is able to reproduce the error:

#!/usr/bin/env python

import sys
import traceback

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
from tornado import gen

import psycopg2
import psycopg2.extras
import momoko


class TestHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @gen.engine
    def get(self):
        try:
            cursor = yield gen.Task(self.application.db["nameservice"].execute, "select 1/0", {})
        except Exception as err:
            self.write({"err": "message"})
            self.finish()


def main():
    try:
        tornado.options.parse_command_line()
        routes = [
            (r'/test', TestHandler),
        ]
        application = tornado.web.Application(
            routes,
            debug=True,
            cookie_secret='bla bla'
        )

        application.db = {
            "nameservice": momoko.AsyncClient({
                'host': "localhost",
                'database': "nameservice",
                'user': "miiza",
                'password': "aaaaaaaaaaa",
                'min_conn': 0,
                'max_conn': 25,
                'cleanup_timeout': 8,
                'connection_factory': psycopg2.extras.NamedTupleConnection
            })
        }

        http_server = tornado.httpserver.HTTPServer(application)
        http_server.listen(8888)
        print "Tornado web server now runs."
        tornado.ioloop.IOLoop.instance().start()
    except KeyboardInterrupt:
        print('Exit')


if __name__ == '__main__':
    main()

After first run everything is quite OK, exception was cought by my exception handler and the request is finished:

[I 130303 23:07:47 web:1462] 304 GET /test (127.0.0.1) 33.01ms

But when I send from my browser the same request after 2 seconds I get this error. It seems like if the exception hangs in the connection:

[E 130303 23:07:48 web:1085] Uncaught exception GET /test (127.0.0.1)
    HTTPRequest(protocol='http', host='localhost:8888', method='GET', uri='/test', version='HTTP/1.1', remote_ip='127.0.0.1', body='', headers={'Accept-Language': 'cs,en-us;q=0.7,en;q=0.3', 'Accept-Encoding': 'gzip, deflate', 'Host': 'localhost:8888', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0', 'Connection': 'keep-alive', 'Cache-Control': 'max-age=0', 'If-None-Match': '"62e84d336c5973652977bb0379ea846f163bcfaf"'})
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1021, in _stack_context_handle_exception
        raise_exc_info((type, value, traceback))
      File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 232, in wrapped
        callback(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/momoko/pools.py", line 313, in _io_callback
        state = self._conn.poll()
    DataError: division by zero

[E 130303 23:07:48 web:739] Cannot send error response after headers written

So, it is strange. I think my exception handler should catch the database exception anytime I send request from my browser. But the reality is that in case of db connection reuse it is not true. It seems like if connection which an exception went through becomes ill and breaks it completely.

Unexpected behaviour of BlockingClient as a context manager

For my unit tests I have another db that I use to stuff in fixture data. Unlike the app, I want this one to be blocking since I don't want additional hassle as I'm setting up the test. Here's what I tried:

#tests/base.py
    @property
    def db(self):
        # Create a database connection when a request handler is called
        # and store the connection in the application object.
        if not hasattr(self._app, 'db'):
            from socorro.unittest.config.commonconfig import (
              databaseHost, databaseName, databaseUserName, databasePassword)

            self._app.db = momoko.BlockingClient({
                'host': databaseHost.default,
                'database': databaseName.default,
                'user': databaseUserName.default,
                'password': databasePassword.default,
                'min_conn': 1,
                'max_conn': 20,
                'cleanup_timeout': 10
            })
        return self._app.db
#sometest.py
    def setUp(self):
        ...
        with self.db.connection() as conn:
            cursor = conn.cursor()
            cursor.execute("""
                INSERT INTO users ....

But unfortunately I'm getting this:

======================================================================
ERROR: test_get_no_data (tests.test_handlers.CrashesCommentsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterb.../middleware2/tests/test_handlers.py", line 20, in setUp
    with self.db.connection() as conn:
TypeError: 'GeneratorContextManager' object is not callable

problem accession a connection from the pool

Hi,

I need to use the register_composite() function from psychopg2 and it ask for a cursor or a connection. I wich to do this just after creating a Momoko pool:

application.db = momoko.Pool(
            dsn="dbname=assurance user=assurance password=assurance host=localhost port=5432",
            minconn=1,
            maxconn=10,
            cleanup_timeout=10)
psycopg2.extras.register_composite('_result', application.db._get_connection(), globally=True)

but i got this error : AttributeError: 'NoneType' object has no attribute 'cursor' even if Momoko works great in other parts of the application. So where can I access a connection ?

By the way thanks for the great work, Momoko is awesome!

TypeError: putconn() got an unexpected keyword argument 'callback'

It important to return connection back to the pool once you’ve done with it, 
even if an error occurs in the middle of your work. 
Use either putconn() method or manage() manager to return the connection.

so. i tried this example:

@gen.coroutine
def get(self):
    chunk = 1000
    try:
        connection = yield momoko.Op(self.db.getconn)
        with self.db.manage(connection):
            yield momoko.Op(connection.execute, "BEGIN")
            ...
    except Exception as error:
        self.write(str(error))
    finally:
        yield momoko.Op(self.db.putconn, connection)

but error occured:

TypeError: putconn() got an unexpected keyword argument 'callback'

and i found solution, add param callback=None to putconn function at connection.py line 444:

    def putconn(self, connection, callback=None):

and it should be fixed, but i've not test this

How do I recycle cursors?

I have a max_conn of 1000 (just for argument's sake) and I run out of cursors. This is the big batch that I run:

cursors = yield gen.Task(self.db.batch, queries)
            for key,cursor in cursors.items():
                worddata = {}
                if verse[0] not in results.keys():
                    results[verse[0]] = []  
                datapoints = cursor.fetchone()
                results[verse[0]].append( {
                        'wordid': datapoints[0],
                        'word': datapoints[1],
                        'inflectedmeaning': datapoints[2],
                        'root': datapoints[3],
                        'rootmeaning': datapoints[4],
                        'case': datapoints[5],
                        'person': datapoints[6],
                        'gender': datapoints[7],
                        'number': datapoints[8],
                        'tense': datapoints[9],
                        'voice': datapoints[10],
                        'mood': datapoints[11],
                } )     

I use this to render a page and each word has a link. At this point, I'm getting regular errors to the effect of PoolError: connection pool exausted or InterfaceError: cursor already closed. I'm pretty sure that I'm doing something wrong but am unsure as to what it is.

def get(self, word):            
        queries = {}
        queries['worddetails'] = ['select * from worddetailsbyword(%s)', (word,)]       
        queries['wordlocations'] = ['select v.id, reference, value from verse v join version vv on v.version_id = vv.id where to_tsvector(analysis_text) @@ to_tsquery(%s)', (word,)]       

        cursors = yield gen.Task(self.db.batch, queries)

        for key, cursor in cursors.items():
            self.write('Query results: %s = %s<br>' % (key, cursor.fetchall()))
        self.finish()

AsyncPool new_cursor method

I am crating AsyncPool instance with argument ioloop set from tornado.ioloop.IOLoop.instance() and I am damn sure it is ok.

The problem is, I don't know what to put into new_cursor( function ...). I do not know where the execute/batch? I am not sure, but those methods are inaccesible from AsyncPool instance.

I think they should be -> In my understanding AsyncPool is "all in one object".

OSError: [Errno 2] No such file or directory

I was evaluating momoko for a project and noticed it was throwing errors if i instantiate the connection pool, trying to track this down i took the example from the momoko manual and that is also throwing the error,

this is running on OSX with the home brew version of python 2.7.3

$ cat test.py
from tornado import gen
from tornado.ioloop import IOLoop
from tornado.httpserver import HTTPServer
from tornado.options import parse_command_line
from tornado.web import *

import psycopg2
import momoko


class BaseHandler(RequestHandler):
    @property
    def db(self):
        return self.application.db


class TutorialHandler(BaseHandler):
    def get(self):
        self.write('Some text here!')
        self.finish()


if __name__ == '__main__':
    parse_command_line()
    application = Application([
        (r'/', TutorialHandler)
    ], debug=True)

    application.db = momoko.Pool(
        dsn='dbname=your_db user=your_user password=very_secret_password '
            'host=localhost port=5432',
        size=1
    )

    http_server = HTTPServer(application)
    http_server.listen(8888, 'localhost')
    IOLoop.instance().start()


$ python -m test
[I 141119 14:42:04 connection:536] Opening new database connection
[E 141119 14:42:04 ioloop:585] Exception in callback (8, <function null_wrapper at 0x104b337d0>)
    Traceback (most recent call last):
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/ioloop.py", line 837, in start
        handler_func(fd_obj, events)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper
        return fn(*args, **kwargs)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/momoko/connection.py", line 599, in io_callback
        self.ioloop.update_handler(self.fileno, IOLoop.WRITE)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/ioloop.py", line 681, in update_handler
        self._impl.modify(fd, events | self.ERROR)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/platform/kqueue.py", line 45, in modify
        self.unregister(fd)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/platform/kqueue.py", line 50, in unregister
        self._control(fd, events, select.KQ_EV_DELETE)
      File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/platform/kqueue.py", line 64, in _control
        self._kqueue.control([kevent], 0)
    OSError: [Errno 2] No such file or directory
^CTraceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/daniel/foo/test.py", line 37, in <module>
    IOLoop.instance().start()
  File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/ioloop.py", line 812, in start
    event_pairs = self._impl.poll(poll_timeout)
  File "/Users/daniel/.venvs/foo/lib/python2.7/site-packages/tornado/platform/kqueue.py", line 67, in poll
    kevents = self._kqueue.control(None, 1000, timeout)
KeyboardInterrupt

$ python --version
Python 2.7.3

$ pip freeze
Momoko==1.1.5
backports.ssl-match-hostname==3.4.0.2
brukva==0.0.1
certifi==14.05.14
geoip2==2.0.2
psycopg2==2.5.4
requests==2.4.3
tornado==4.0.2
wsgiref==0.1.2

Pip momoko version 1.0.0b2

I tried to install new momoko in heroku, look the error:

remote: -----> Python app detected
remote: -----> Using Python runtime (python-3.3.1)
remote: -----> Installing dependencies using Pip (1.3.1)
remote:        Downloading/unpacking Momoko==1.0.0b2 (from -r requirements.txt (line 1))
remote:          Could not find a version that satisfies the requirement Momoko==1.0.0b2 (        (from -r requirements.txt (line 1)) (from versions: 0.4.0, 0.3.0, 0.2.0, 1.0.0, 0.1.0, 0.5.0, 0.1.0, 0.5.0, 0.4.0, 0.2.0, 0.3.0, 1.0.0, 0.2.0, 0.1.0, 0.4.0, 0.3.0, 0.5.0, 0.1.0, 0.5.0, 0.4.0, 0.2.0, 0.3.0, 1.0.0, 0.1.0, 0.5.0, 0.4.0, 0.2.0, 0.3.0, 1.0.0, 0.4.0, 0.3.0, 0.2.0, 1.0.0, 0.1.0, 0.5.0, 0.2.0, 0.4.0, 0.3.0, 0.5.0, 0.1.0, 1.0.0)

To fix it, I needed to change the "pip freeze" file to momoko == 1.0.0 instead momoko == 1.0.0b2

No connection available

After 2 days of clean work, I am getting these rows in my log file:

WARNING:momoko:Execute: no connection available, operation queued.

Thousands of them actually, which eventually kills my application and i get 500/502 responses or no responses at all from server. Then, after restart application runs normally. I wonder if this is momoko bug or problem with postgresql or my code.

I use momoko in rather an odd way:

class MomokoWrapper(object):
    def __init__(self):
        pass

    def pool(self):
        self.db = momoko.Pool(
            dsn='dbname=%s user=%s password=%s host=%s port=%s' % (
                database['name'], database['user'], database['password'],
                database['host'], database['port']
            ),
            size=1
        )
        return self

    def sample_fun(self, params):
       return momoko.Op(self.db.execute, 'my query')


Then in tornado Application init

self.db = MomokoWrapper().pool()

And I am able to execute wrappers functions like this:

yield self.db.sample_fun(params)

Actually, I doubt that code organization and momoko usage is the problem (despite the fact that this may be good/wrong layout). I presume that postgresql or pool configuration might be an issue in my case. I am dealing with these annoying warnings/errors for 2 weeks now and have absolutely no ideas left. All suggestions are appreciated.

getconn() doesn't work with NamedTupleCursor

Greetings. As the title suggests, getconn() doesn't play well with NamedTupleCursor.

Here's a test case:

import psycopg2
import momoko
from psycopg2.extras import NamedTupleCursor
from tornado.testing import AsyncTestCase, gen_test
import unittest
import os

class ConnNamedTuple(AsyncTestCase):
    def setUp(self):
        super().setUp()
        self.db = momoko.Pool(
            dsn='dbname=dbdb user=%s host=localhost port=5432' % os.environ['USER'],
            size=3,
            ioloop=self.io_loop,
            callback=self.stop,
            cursor_factory=NamedTupleCursor
        )
        self.wait()

    def tearDown(self):
        self.db.close()
        self.io_loop.close()

    @gen_test
    def test_conn_namedtuple(self):
        yield momoko.Op(self.db.getconn)

unittest.main()

Here's the stack trace:

======================================================================
ERROR: test_conn_namedtuple (__main__.ConnNamedTuple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/tornado/testing.py", line 427, in wrapper
    functools.partial(f, self), timeout=timeout)
  File "/usr/local/lib/python3.4/dist-packages/tornado/ioloop.py", line 389, in run_sync
    return future_cell[0].result()
  File "/usr/local/lib/python3.4/dist-packages/tornado/concurrent.py", line 129, in result
    raise_exc_info(self.__exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/usr/local/lib/python3.4/dist-packages/tornado/gen.py", line 206, in handle_exception
    if runner is not None and runner.handle_exception(typ, value, tb):
  File "/usr/local/lib/python3.4/dist-packages/tornado/gen.py", line 580, in handle_exception
    self.run()
  File "/usr/local/lib/python3.4/dist-packages/tornado/gen.py", line 529, in run
    yielded = self.gen.throw(*exc_info)
  File "tmp/conn_namedtuple.py", line 26, in test_conn_namedtuple
    yield momoko.Op(self.db.getconn)
  File "/usr/local/lib/python3.4/dist-packages/tornado/stack_context.py", line 302, in wrapped
    ret = fn(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/momoko/connection.py", line 594, in io_callback
    self.callback(None)
  File "/usr/local/lib/python3.4/dist-packages/momoko/connection.py", line 640, in _ping_callback
    cursor.fetchall()
  File "/usr/local/lib/python3.4/dist-packages/psycopg2/extras.py", line 317, in fetchall
    nt = self.Record = self._make_nt()
  File "/usr/local/lib/python3.4/dist-packages/psycopg2/extras.py", line 340, in _make_nt
    return namedtuple("Record", [d[0] for d in self.description or ()])
  File "/usr/lib/python3.4/collections/__init__.py", line 338, in namedtuple
    'identifiers: %r' % name)
ValueError: Type names and field names must be valid identifiers: '?column?'

----------------------------------------------------------------------
Ran 1 test in 0.031s

FAILED (errors=1)

I'm on Python 3.4. pip freeze output, all are PyPI releases.

Momoko==1.1.2
psycopg2==2.5.2
tornado==3.2

I think it's easy to workaround but it's a little bit annoying. Thanks in advance.

invalid arugment error

I got following error when I started my web app with momoko added:

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/tornado-2.0-py2.6.egg/tornado/web.py", line 927, in _execute
    getattr(self, self.request.method.lower())(*args, **kwargs)
  File "hw.py", line 130, in post
    user_id = self.db.get(query)
  File "hw.py", line 37, in db
    'cleanup_timeout': 10
  File "/usr/local/lib/python2.6/dist-packages/momoko/client.py", line 29, in __init__
    self._pool = Pool(**settings)
  File "/usr/local/lib/python2.6/dist-packages/momoko/client.py", line 193, in __init__
    self._new_conn()
  File "/usr/local/lib/python2.6/dist-packages/momoko/client.py", line 209, in _new_conn
    conn = psycopg2.connect(async=1, *self._args, **self._kwargs)
TypeError: 'async' is an invalid keyword argument for this function

link broken in readme

Psycopg should be referencing http://initd.org/psycopg/, without the www. www.initd.org is not working.

Catching exceptions

Catching exceptions is not possibly (yet), because the exception if raised in the Poller class and not in the request handler where the query is executed. tornado.stack_context should be used to fix this.

Trouble testing with AsyncHTTPTestCase

I'm using AsyncHTTPTestCase and some other stuff so that I can do integration style tests by GET fetching URLs on the application in my tests. The code looks something like this:

from tornado.testing import AsyncHTTPTestCase
class BaseHTTPTestCase(AsyncHTTPTestCase, HTTPClientMixin):

    def setUp(self):
        super(BaseHTTPTestCase, self).setUp()
        self.client = TestClient(self)  # from tornado-utils

    def get_app(self):
        # pick up the defaults
        config = TEST_DATABASE_CONFIG
        return app.Application(config)

class CrashesCommentsTestCase(BaseHTTPTestCase):

    def test_get_no_data(self):
        url = self.reverse_url('crashes:comments', 'signature/FooBar')
        response = self.client.get(url)

The app works and its inner core looks something like this:

class BaseHandler(tornado.web.RequestHandler):

    @property
    def db(self):
        # Create a database connection when a request handler is called
        # and store the connection in the application object.
        if not hasattr(self.application, 'db'):
            config = self.application.settings
            self.application.db = momoko.AsyncClient({
                'host': config['database_host'],
                'database': config['database_name'],
                'user': config['database_user'],
                'password': config['database_password'],
                'min_conn': config['database_min_connections'],
                'max_conn': config['database_max_connections'],
                'cleanup_timeout': config['database_cleanup_timeout']
            })
        return self.application.db

@route('/crashes/comments/(.*)', name='crashes:comments')
class CrashesCommentsHandler(BaseHandler):
    @tornado.web.asynchronous
    @tornado.gen.engine
    def get(self, parameters):
        print "pre-cursor"
        cursor = yield tornado.gen.Task(
          self.db.execute,
          ...,
          ...
        )
        print "cursor", repr(cursor)
        total, = cursor.fetchone()
        print "total", repr(total)    
        ...
        self.write(stuff)
        self.finish()

Whenever I try to run the tests I get:

pre-cursor
F
======================================================================
FAIL: test_get_no_data (tests.test_handlers.CrashesCommentsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/middleware2/tests/test_handlers.py", line 8, in test_get_no_data
    response = self.client.get(url)
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado_utils/http_test_client.py", line 58, in get
    follow_redirects=follow_redirects)
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado_utils/http_test_client.py", line 22, in get
    follow_redirects=follow_redirects)
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado_utils/http_test_client.py", line 39, in _fetch
    return self.wait()
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado/testing.py", line 202, in wait
    self.__rethrow()
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado/testing.py", line 146, in __rethrow
    raise_exc_info(failure)
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/tornado/testing.py", line 183, in timeout_func
    timeout)
AssertionError: Async operation timed out after 5 seconds

So, is there something I need know about running tests "on top of" momoko?
Any other hints?

Non psycopg2 exception can lock connection

For example I have code like this and only one connection in pool:

        try:
            cursor = yield momoko.Op(self.db.execute, 'SELECT %s FROM %s'
                ,()) # rises TypeError
        except Exception as e:
            print e
            cursor = yield momoko.Op(self.db.execute, 'SELECT 1'
                ,())
            print 'Done' #unreachable

This code rises TypeError, but second query can't obtain connection.
Debug log is:

[D 141115 18:00:26 connection:234] Connection attempt complete. Success: True
[D 141115 18:00:31 connection:252] Getting connection
[D 141115 18:00:31 connection:327] Connection obtained, proceeding
tuple index out of range
[D 141115 18:00:31 connection:252] Getting connection
[D 141115 18:00:31 connection:272] There are busy connections
[D 141115 18:00:31 connection:321] No connection available right now - will try again later

Changing psycopg2.Error to Exception in _catch_early_errors decorator fixes that issue, but I not sure that it right.

Extra connection is created on pool creation

When the application is started and the first query is going to be executed the connection pool is also created. It looks like there is not yet a connection available and a new one is created.

no db.batch method in 1.0b

after updating to v1.0b, I have found there is no db.batch exists any more. would you please add this method back again?

gen_example.py tries to init momoko.Pool with unexpected keyword argument 'register_hstore'

It looks like register_hstore is in the Pool constructor any more.

$python gen_example.py 
Traceback (most recent call last):
  File "gen_example.py", line 194, in <module>
    main()
  File "gen_example.py", line 183, in main
    size=1
TypeError: __init__() got an unexpected keyword argument 'register_hstore'

Checked on 1.0.0, but looks like its the issue on master as well.

Momoko doesn't recover from lost connections

Came across an opportunity to restart PostgreSQL and did so. My Tornado app ceased functioning and did not recover. Logs reveal the following exception:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/tornado/web.py", line 1045, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/tornado/gen.py", line 91, in wrapper
    Runner(gen).run()
  File "/usr/lib/python2.6/site-packages/tornado/gen.py", line 318, in run
    self.yield_point.start(self)
  File "/usr/lib/python2.6/site-packages/tornado/gen.py", line 207, in start
    self.func(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/momoko/clients.py", line 125, in execute
    self._pool.new_cursor('execute', (operation, parameters), callback)
  File "/usr/lib/python2.6/site-packages/momoko/pools.py", line 215, in new_cursor
    cursor = connection.cursor()
  File "/usr/lib/python2.6/site-packages/psycopg2/extras.py", line 116, in cursor
    return _connection.cursor(self, cursor_factory=DictCursor)
 InterfaceError: connection already closed

Given that Momoko maintains a pool of connections, catching and recovering from this in client code would be difficult.

ORM

Since the blocking-style API is done it's easier to write top-down code.

Maybe Peewee can be adapted so it uses Momoko.

Pool initialization when database is down

When trying to initialize the a momoko pool while the database is down, an exception will be raised, but it is uncatchable by the caller.
After the call the caller does not know whether or not the pool was successfully initialized. Calling for example execute on the failed pool will hang trying to queue the command, spamming the log with this:
WARNING:momoko:Execute: no connection available, operation queued.

Static connection pool and queueing

The dynamic connection pool is causing some difficulties. See the last four comments in issue #34 for information.

Currently connections are considered "busy" if they are executing a query or are in a transaction. They stay in the same list at the connections that are not busy. The connection pool is cleaned every N seconds. And new connections are created when needed.

The problem is that connections are only "busy" when executing queries and not when results are being fetched from the cursor. When the connection is closed at that time the cursor throws an error. To prevent this the connection needs to know if its cursors are still being used. Psycopg2 does a get and put action on a connection pool. I don't see that really working with Tornado. And I couldn't find a way to see if a cursor still needs to connection.

I would like to remove the dynamic connection pool and replace it with a static one. This simplifies things a bit and there's no need for a pool cleaner that closes a connection when it's still needed. A simple queue can be used to queue operations when all the connections are in use. Like Tornado's HTTP client does.

Why would I write this here? I you to know what I'm doing and I want some feedback if possible.

Transactions

There's currently support for transactions in the master branch contributed by @aleksj. You put a bunch of queries in and get a bunch of queries out. I'm planning to implement the same method in the rewrite.

Something that might also be useful is getting a connection from the connection pool, use it for the entire request and then put it back in the pool.

Here's a pseudo code example:

def get(self):
    conn = self.pool.get()  # Connection is removed from the pool

    idnr = conn.execute('INSERT SOME STUFF RETURN id;')

    conn.execute('INSERT SOME STUFF BASED ON %s;', (idnr,))

    self.pool.put(conn)  # Connection is put back into the pool

This would be asynchronous of course, but I left the callbacks away for simplicity.

Feedback?

Asynchronous connection failed

Hi. I use tornado + momoko + postgresql 8.4 and all works fine. Then, I drop postgresql 8.4 and install 9.1 and momoko cannot connect to postgresql:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 202, in wrapped
    callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/momoko/utils.py", line 140, in _io_callback
    self._update_handler()
  File "/usr/local/lib/python2.7/dist-packages/momoko/utils.py", line 127, in _update_handler
    state = self._connection.poll()
OperationalError: asynchronous connection failed

BaseHandler:

class BaseHandler(tornado.web.RequestHandler):
    @property
    def db(self):
        if not hasattr(self.application, 'db'):
            self.application.db = momoko.AdispClient({
                'host': settings.host,
                'port': settings.port,
                'database': settings.database,
                'user': settings.user,
                'password': settings.password,
                'min_conn': settings.min_conn,
                'max_conn': settings.max_conn,
                'cleanup_timeout': settings.cleanup_timeout
            })
        return self.application.db

Query:

@tornado.web.asynchronous
@momoko.process
def get(self):
    cursor = yield self.db.execute('select 1,2,3');
    print cursor.fetchall()

What I`m doing wrong ?

Documentation

Make some documentation with Sphinx and put it on Github pages and the PyPi page.

volatile_db functionality should work with local connections

psychopg2 connects synchronously to local socket. When Pool is initialized and database is down, the pool fails to start because psycopg2 raises connect error immediately.
The solution would be probably to catch connect errors early and immediately mark connection as dead.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.