Code Monkey home page Code Monkey logo

graphile-scheduler's Introduction

Graphile Worker now has support for scheduling built in. And it's really good! You should use that instead.

graphile-scheduler

Reliable job scheduling for PostgreSQL running on Node.js built on top of graphile-worker. Allows you to run jobs on a regular schedule (e.g. sending reminders once a week, cleaning up data nightly, etc) with fault tolerance and resilience. Can be used with any PostgreSQL-backed application. Pairs beautifully with PostGraphile.

Why not use something like node-cron or a timer? These in process approaches have 2 downsides: 1) you have to make sure that there is only 1 process running the schedule or you risk running the tasks multiple times and 2) if that 1 process happens to be down (either due to an outage or a deploy) when the scheduled job is suppose to be excecuted, it will be skipped.

graphile-scheduler keeps track of it's schedules in a PostgreSQL database. On startup it will queue any jobs that should have occurred since the last time it was checked, whether that was 1 minute ago or 1 day ago (up to a certain limit determined by your configuration). It uses PostgreSQL locking so it's safe to have multipel schedulers running at once. And because it is integrated with graphile-worker, jobs are queued and automatically retried with exponential backoff if they fail.

Quickstart

Add the scheduler to your project:

yarn add graphile-scheduler
# or: npm install --save graphile-scheduler

Schedule and run jobs:

run({
  connectionString: "postgres:///",
  schedules: [
    {
      name: "send_reminder",
      pattern: "0 10 * * 1-5", // every weekday at 10AM
      timeZone: "America/Los_Angeles",
      task: async ({ fireDate }) => {
        console.log("send a reminder for", fireDate);
      },
    },
  ],
});

Every weekday at 10AM the task function will be called. You can use fireDate to access the time the job was originally suppose to be run. If the scheduler goes down and retroactively queues schedules that it missed or the job is retried, this will be when it should have been queued.

Schedule and run jobs separately:

If you provide a task function an instance of graphile-worker will also be started to run the task. However you can have the runner only schedule jobs and have a separate process actually run them off of the queue:

run({
  connectionString: "postgres:///",
  schedules: [
    {
      name: "send_reminder",
      pattern: "0 10 * * 1-5", // every weekday at 10AM
      timeZone: "America/Los_Angeles",
    },
  ],
});

Or you can create schedules manually in the database and only run schedule checks:

INSERT INTO "graphile_scheduler"."schedules"
(
	"schedule_name",
	"minute",
	"hour",
	"day",
	"month",
	"dow",
	"timezone",
	"task_identifier"
)
VALUES(
	'foo',
	'{0}',
	'{10}',
	graphile_scheduler.every_day(),
	graphile_scheduler.every_month(),
	'{1,2,3,4,5}',
	'America/Los_Angeles',
	'foo'
);
run({
  connectionString: "postgres:///",
  schedules: ["send_reminder"],
});

If you omit schedules completely, every schedule in the database will be checked.

run({ connectionString: "postgres:///" });

Status

This project is feature complete and in use, but has not yet been battle tested and test coverage is incomplete. It is possilbe that there will be breaking changes leading up to a 1.0.0 release.

Additionally, graphile-worker is still in early development as well, so until it reaches 1.0.0 the project pins it's dependency to it at a fixed version number.

graphile-scheduler's People

Contributors

bchrobot avatar davbeck avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

graphile-scheduler's Issues

runOnce function does not run at all

Hi,
my code example is below

        const name = `send_reminder`
        const cronResp = await runOnce({
            connectionString: CONN_STRING,
            schedules: [
                {
                    name: name,
                    pattern: "* * * * *",
                    timeZone: "Asia/Kuala_Lumpur",
                    task: async (payload) => {
                        console.log("---------------------- --------", bidderDetails, payload, name)`
                    }

                }
            ]
        }

when i use runOnce, the code does not run at all, eventhough it's recorded in graphile_worker.jobs. However, it works for run

In graphile_worker.jobs they seem to be recording twice.

Any ideas?

Regards

App does not start up on node 14

PR coming shortly ... we need to upgrade "pg" so that this app will run on node 14: graphile/worker#107

FYI we verified your current code works on node 10-13 before we stumbled upon the issue above (wish we found it earlier ... would have saved us time :))

The PR will also have an update to the latest worker version since we have been using the worker directly for other asynchronous tasks (on-demand).

only want to run task once, and then end

Hi,
I want to run a task once, after X minutes, and then stop.
Would graphile-scheduler be suitable for this. I noticed that it's aimed at running repeated times rather than just once.

Thanks

Schedules to not take timezone into account

Hi,

Regardless of what is entered into the timezone of a schedule, it seems to always kick off according to UTC.

The simplest fix is to change the following line

IF :GRAPHILE_SCHEDULER_SCHEMA.schedules_matches(v_schedule, v_next_check) THEN
to the following ...

FROM

  	IF :GRAPHILE_SCHEDULER_SCHEMA.schedules_matches(v_schedule, v_next_check) THEN

TO

  	IF :GRAPHILE_SCHEDULER_SCHEMA.schedules_matches(v_schedule, v_next_check  AT TIME ZONE v_schedule.timezone) THEN

It would probably be better if the schedules_matches function itself could be passed any timestamp and did that translation internally but my PG plsql is very rusty.

Once I did the fix the schedules kicked off at the correct time. The job entries still had the fireDate specified in UTC time but it was the correct equivalent time.

Run a job at specific timestamp

Hi @davbeck,
I have deleted_at column on foo table and I need to run a job at the mentioned deleted_at timestamp, how to do such thing with graphile-scheduler? is it even possible?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.