Worker sometimes fails "Exceeded maximum connections" about rocketmap HOT 42 CLOSED

jokes i m just scared of issues piling up, trying to close them asap lel

from rocketmap.

@JZ6 i'll close and merge issues as my main contribution. Keep them from piling up too much.

from rocketmap.

@FlarumHostingCo No.

from rocketmap.

fuck i just got that error too, should ve listended to my own words

from rocketmap.

@FlarumHostingCo even at 200, I think its masking a real issue. I'd be willing to bet you'll hit a point where it'll start failing again.

from rocketmap.

can we switch to redis? the amount of requests going in it would make sense. It's what most crypto currency pools use for database entry.

from rocketmap.

@JZ6 dont use mysql

from rocketmap.

I can concur this happens when using sqlite or there is not enough database connections set on the mysql database.

from rocketmap.

use mysql

from rocketmap.

This happens with mysql as well

from rocketmap.

I am using mariadb mysql

from rocketmap.

dont use mysql

from rocketmap.

lel

from rocketmap.

in utlis.py I edited it to 200 and works fine now with 50 workers.

parser.add_argument('--db-max_connections', help='Max connections for the database', type=int, default=200)

I'm using MariaDB 10.1 on Ubuntu16.04

from rocketmap.

rofl

from rocketmap.

LOL You missed I said ---> works fine now <----

from rocketmap.

JZ6 commented 2 minutes from now: use mysql
JZ6 commented 2 minutes from now: dont use mysql

perfect

from rocketmap.

.remindme comment dont use mysql at 12:32 am

from rocketmap.

Should make it a PR after every merge

from rocketmap.

use the feckin manual
parser.add_argument('--db-max_connections', help='Max connections for the database', type=int, default=5)
don't edit it, just set the config

from rocketmap.

Yeah prolly so invisiblek. I'll watch for it tomorrow. Atleast it happens less often. Bandaid atleast for now.

from rocketmap.

Bandaid would be to cronjob restarting your workers every few hours. This is more like giving aspirin for a broken leg. 😜

from rocketmap.

@FrostTheFox dont use mysql

from rocketmap.

@BaIthamel i better not see 600 issues by the end of the week....

from rocketmap.

@JZ6 use mysql

from rocketmap.

What about MongoDB?

from rocketmap.

Yes any chance of redis alternative? Or some form of caching and reach certain amount of entries and use 1 transaction to push all those entries?

from rocketmap.

setting db-max_connections insanely high (200) does seem to make the issue...

from rocketmap.

also not increasing the default mysql number of connections will do this. I'm at a point now where i also need to increase the open file limit on the server, to increase the mysql connections anymore.

increasing the open file limit to 200k allowed me to increase the mysql database connection limit to 100k, after that i havent seen the issue. but I still think redis > mysql

from rocketmap.

@Infernoman check my mytop output definitely not hitting the max of 200 connections on the server.

from rocketmap.

Its not connections to your mysql server, its connections from the client (peewee) (AKA its a problem with peewee itself, not the connection to the db)

https://github.com/coleifer/peewee/blob/master/playhouse/pool.py#L133

I think we need to up our connections, or possibly use something like args.db-max_connections * num_accounts

from rocketmap.

Ah what does max connections default to?

from rocketmap.

5, I'm at 100, but its probably overkill. Haven't had an issue since though

from rocketmap.

try this: https://github.com/PokemonGoMap/PokemonGo-Map/pull/239

from rocketmap.

OMG default is 5!

from rocketmap.

https://github.com/PokemonGoMap/PokemonGo-Map/pull/296

from rocketmap.

As mine was a copy #305 - I believe it to be an issue with Flask itself. Rather than anything database related, as my database runs perfectly (mysql) and inserts everything as expected. I'm assuming Flask can only handle a certain amount of concurrent connections at one time. I may be exceeding this significantly.

from rocketmap.

@SimonTellsAll: You should look into these for your purposes then:

http://flask.pocoo.org/docs/0.11/deploying/#deployment

uWSGI should be able to handle significantly higher loads than running Flask directly.

from rocketmap.

Is this related to the same issue I'm having with ngrok and local tunnel trying to create a URL to my localhost map? Want to make sure I'm not duplicating issues.

Getting this error from ngrok a couple minutes after generating a new URL: "Too many connections! The tunnel session '###' has violated the rate-limit policy of 20 connections per minute. Please decrease your inbound connection volume or upgrade to a paid plan for additional capacity. ERR_NGROK_702"

Amd getting this error from local tunnel a couple minutes after generating a new URL: "no active client for ###"

from rocketmap.

got this error twwwo i['m baaaaad

from rocketmap.

This error is related to Flask when there are many connections and has nothing to do with database.
Still happening is the latest builds.
Anyone has a fix for this?

from rocketmap.

Same error happened today, after updating. I'm using mysql, no worker running, only webserver displaying gyms.

2016-09-29 11:33:42,852 [     Thread-7235][           app][   ERROR] Exception on /static/016db14/dist/js/map.min.js [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1637, in full_dispatch_request
    rv = self.preprocess_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1837, in preprocess_request
    rv = func()
  File "/usr/local/lib/python2.7/dist-packages/playhouse/flask_utils.py", line 165, in connect_db
    self.database.connect()
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 3402, in connect
    **self.connect_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/playhouse/pool.py", line 127, in _connect
    raise ValueError('Exceeded maximum connections.')
ValueError: Exceeded maximum connections.

EDIT : Seems like raising --db-max_connections solve the issue

from rocketmap.