Code Monkey home page Code Monkey logo

sequelize-pool's Introduction

Sequelize logo

npm version npm downloads contributors Open Collective sponsor Merged PRs semantic-release License: MIT

Sequelize is an easy-to-use and promise-based Node.js ORM tool for Postgres, MySQL, MariaDB, SQLite, DB2, Microsoft SQL Server, Snowflake, Oracle DB and Db2 for IBM i. It features solid transaction support, relations, eager and lazy loading, read replication and more.

Would you like to contribute? Read our contribution guidelines to know more. There are many ways to help! πŸ˜ƒ

πŸ’» Getting Started

Ready to start using Sequelize? Head to sequelize.org to begin!

πŸ’Έ Supporting the project

Do you like Sequelize and would like to give back to the engineering team behind it?

We have recently created an OpenCollective based money pool which is shared amongst all core maintainers based on their contributions. Every support is wholeheartedly welcome. ❀️

πŸ“ Major version changelog

Please find upgrade information to major versions here:

πŸ“– Resources

πŸ”§ Tools

πŸ’¬ Translations

⚠️ Responsible disclosure

If you have security issues to report, please refer to our Responsible Disclosure Policy for more details.

sequelize-pool's People

Contributors

blax avatar bryandonovan avatar chriskchew avatar coopernurse avatar dougwilson avatar eseliger avatar fauxfaux avatar felipou avatar felixfbecker avatar gdusbabek avatar holm avatar jasonrhodes avatar joez99 avatar johnjdooley avatar kevinwilson541 avatar kurayama avatar lewisjellis avatar mikemorris avatar molipet avatar poetro avatar renovate-bot avatar renovate[bot] avatar simonschick avatar sushantdhiman avatar thefourtheye avatar tikonen avatar tilgovi avatar turtlesoupy avatar uzitech avatar windyrobin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

sequelize-pool's Issues

maxUses based on time instead of uses

So we're having the exact same problem that maxUses seems to describe and we're going to implement it (pgbouncer instances are getting 4/5 and 1/5 split of traffic).

It looks great but feels quite difficult to implement dynamically (especially since a server may be long lived, how do you dynamically update the maxUses option based on traffic and how do you even get that data?).

Wouldn't it be much easier to just reconnect every 30 minutes instead of counting uses of a connection?

I'm not too versed in how this library works, but looking at the maxUsesPerInstance code, I believe it would be as simple as a setInterval to swap a flag from false -> true and then calling destroy on release()

Moving sequelize-pool inside sequelize monorepo

This message is mostly for @sushantdhiman but others are welcome to chime in as well.

As you might have seen, sequelize has been a monorepo for a while now. With the release of v7 nearing we want to move other repositories like sequelize-cli and the unmaintained pg-hstore towards the monorepo as well. Would you be okay if we move sequelize-pool in there as well and release future major releases as @sequelize/pool?
A while back we've had the same discussion for umzug but because there are various use cases outside of sequelize we decided that it will remain it's own repository. So we're able to go both ways but we'd like your input on this.

cc @ephys

Question - Monitoring connection pool

πŸ‘‹ Hello community, hope you are doing great.
Do we have any way to monitor the connection pool?
Would like to see connections being added and released, maybe even using Prometheus.

Destroying inactive resources can cause unhandled promise rejection

Hi,
I wanted to share a possible bug. If destroy method passed to Pool constructor is async, then its rejection might cause an unhandled promise rejection, for example in idle timer (it doesn't seem to await resource destruction). Here's a test that demonstrates the behavior (it fails with an unhandled rejection)

import * as tap from 'tap';
import { Pool } from '../../src';

tap.test('should finish the test without unhandled rejection', async (t) => {
  const pool = new Pool<{ foo: number }>({
    create: () => Promise.resolve({ foo: 1 }),
    // Pool doesn't await destroy when closing idle connections
    destroy: () => Promise.reject('Ohnoo'),
    validate: () => true,
    max: 5,
    min: 0,
    idleTimeoutMillis: 500,
    reapIntervalMillis: 1000,
  });

  const res = await pool.acquire(); // create first resource, it will be kept
  t.equal(res, { foo: 1 });
  pool.release(res);
  // wait 3 seconds for idle timer to cause unhandled rejection
  await new Promise((resolve) => {
    setTimeout(resolve, 3000);
  });
  await pool.destroyAllNow().then(
    () => console.log('Destroyed'),
    (err) => console.log('Error, whatever' + err),
  );
});

Thanks for looking at this,

Custom keepalive on idle connections protected due to minimum pool size

Greetings!

While diagnosing some db issues, we realized there may be a feature that could help mitigate or remove aborted connections.

The current state of our setup is:

  • a pool with a min size
  • pool resource idle timeout of 10s
  • db connection idle timeout of 60s

On the surface, one would think there should never be any aborted clients because the pool idle timeout is lower than the db connection idle timeout. However, this is not the case because the pool (rightly) does not reap idle connections once you have reaped down to the pool minimum size.

This means that in periods of low activity, the db will end up aborting the pool idle connections because they are not being used but the pool is protecting them from reaping.

With a goal of reducing the number of db aborted clients to zero, I wonder if it is possible to allow some configuration in [1]. The goal being something like:

  1. removeIdle triggers
  2. all idle connections down to min pool size are reaped as normal
  3. all idle connections below the min pool size allow a config option to be called with that resource to basically tell the resource "do what you need to do in order to keep yourself healthy"

In my specific case using mysql2, this would probably be something like pooledObject.resource.ping().

The end result would be that even in periods of low traffic, the pooled connections below the min threshold would keep themselves healthy by continually resetting the idle timeout on the server so that they are not closed due to lack of usage.

Would a feature like that be useful or accepted? If so, I may be able to work on a PR.

[1]

sequelize-pool/src/Pool.ts

Lines 263 to 277 in da269d7

// Go through the available (idle) items,
// check if they have timed out
for (i = 0; i < available && maxRemovable > toRemove.length; i++) {
timeout = this._availableObjects[i].timeout;
if (now >= timeout) {
// Client timed out, so destroy it.
this._log(
'removeIdle() destroying obj - now:' + now + ' timeout:' + timeout,
'verbose',
);
toRemove.push(this._availableObjects[i].resource);
}
}
toRemove.forEach(this.destroy, this);

Update connection host. impact on connection pool?

Sorry, I am not sure where I should put my question.

I have a project needing to switch MySQL host at a moment to make the application readonly.

So at the beginning, I configured two hosts:

  • master is read-write
  • and a read replica

internally, this pool will create two pools in Sequelize.

Assume 2 hours late, I need to update database host both using read replica.

I add the code in Sequelize hook like the below:

sequelize.beforeConnect((options) => {
	options.host = 'readonly.host'
	options.username = 'readonly.user'
	options.password = 'readonly.password'
})
sequelize.beforePoolAcquire((options) => {
	options.host = 'readonly.host'
	options.username = 'readonly.user'
	options.password = 'readonly.password'
})

So at the moment, in the writer pool, there are some connections actually retaining the read-write host, right?

then when Sequelize acquires a connection, is it still possible to acquire the old connection pointing to read-write host? as I hope after I switch host to readonly, all subsequent connections are supposed to be readonly connections pointing to readonly host.

Please help clarify.

Thanks

Acquire timeout not respected (seen on MSSQL)

Hello,

I noticed that on MSSQL, the pool has acquire timeout 60 seconds, but the value is not passed to tedious driver which uses by default 15 secs.
Is this intentional or it is a bug ?

Thanks in advance

Not respecting 0 as idle timout as suggested by AWS lambda doc

Sequelize uses lodash default to set default (respecting falsy if 0 set) while sequelize-pool does a simpler OR statement ||

https://github.com/sequelize/sequelize/blob/main/src/dialects/abstract/connection-manager.js#L34
https://github.com/sequelize/sequelize-pool/blob/master/src/Pool.ts#L167

The docs for using sequelize in AWS lambda https://sequelize.org/docs/v6/other-topics/aws-lambda/ context says to set it to 0, but that falls back to 30000ms since 0 is falsy.

Suggestion: use proper detection of undefined instead of simple or statement.

Connection do not close after call sequelize.close()

Hi I faced with issue of closing connecting to db after calling of sequelize.close().

You can find below example of source code with reproducing this issue.
I also prepared pull request #8 that fix this issue, please review it.

const Sequelize = require('sequelize');

process.on('exit', () => {
  console.log('process.on exit');
});

const sequelize = new Sequelize('mysql://user:pass@localhost:3306/dbname', {
  dialect: 'mysql',
});

async function cleanDB() {
  await sequelize.authenticate();
  console.log('DB connection successfully established');
  return sequelize.transaction(async t => {
    const options = {
      transaction: t,
    };
    await sequelize.query('SELECT 1+1', options);
    console.log('Query was processed successfully');
  });
}

cleanDB()
  .then(() => {
    console.log('cleanDB finish successfully ');
  })
  .catch(err => {
    console.log('cleanDB throw error => ', err);
  })
  .finally(() => {
    sequelize
      .close()
      .then(() => {
        console.log('Connection has been close successfully.');
      })
      .catch(err => {
        console.error('Unable to close connect to the database:', err);
      });
  });

Min Pool is not respected

Issue Description
In the sequelize configuration, I've set

pool: {
min: 10,
max: 30
}

When the connection is acquired it does not check if additional connections need to be made and hence only 1 connection is made instead of 10.

What do you expect to happen?
I expect the min setting to be respected and 10 connections to be made.

Issue is a copy of: sequelize/sequelize#11440 however I believe the issue is in sequelize-pool lib and not in the main sequelize project.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.