Comments (9)
See this - #11. Also we need dequeue method
from jesque.
I think, Jesque should use expire keys for workers and some cleaning up logic.
We can't be sure that application ends in safe way. And now there is no way to clean up wrong data in redis.
from jesque.
Does Resque have this feature? Do you have a suggestion as to how to make this cleanup happen safely?
from jesque.
Sorry. Resque... Jesque... =)
I'm talking about Jesque. As i understand, worker information is: key and values in some sets.
You can set expire for worker keys and use utility worker thread, that time to time updates expire information on worker keys that are started on current JVM, and cleans information from sets on workers, that are expired (keys not found).
from jesque.
So, if I understand you, your proposal is to have a separate thread on each worker JVM that updates an exipry key in Redis for each worker in that JVM. Then there is an external reaper process that periodically scans those exipry keys and if a key is stale, the reaper then assumes the corresponding worker's JVM has crashed hard and will cleanup all Redis keys associated with that worker. Is that correct?
Have you had an issue with your JVM's crashing? Have you tried adding a shutdown hook or a signal handler?
from jesque.
Have you had an issue with your JVM's crashing? Have you tried adding a shutdown hook or a signal handler?
Ofc, i use hooks. But they can't give you 100% guarantee =)
from jesque.
OK, just checking!
Would you be willing to start on this feature and submit a pull request?
from jesque.
@Kondratovich Any more thoughts on this?
from jesque.
@gresrun I am in the process of migrating from Resque to Jesque and have a heartbeat feature that needs to be migrated over - sounds like it fits this bill. Time permitting I'll try and get a PR for you.
from jesque.
Related Issues (20)
- java.net.UnknownHostException thrown when `ip-#-#-#-#` pattern has no reverse lookup HOT 2
- No worker running when facing `Terminating in response to exception`
- jackson 2.9 incompatibility
- jackson databind 2.9 includes a breaking change HOT 5
- successful job is re-enqueued during shutdown HOT 1
- Worker queue in redis not reset when restarting?
- client.enqueue() error `java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.DeserializationConfig.with` HOT 1
- Throw JedisDataException when enqueue a job HOT 1
- Requesting new public release. HOT 1
- Version 2.1.3 breaks API compatibility from 2.1.2 HOT 1
- why does jesque not use namespace as prefix for recurring jobs? HOT 2
- RESET_TO_HIGHEST_PRIORITY Strategy not work for recurring job
- Implement a way to set FailQueueStrategy on WorkerImplFactory
- Jedis 3.X compatibility HOT 9
- NoSuchConstructorException HOT 1
- 通过jesque的延时任务实现支持cron表达式的调度任务 HOT 1
- Redis Cluster Issue
- ETA on current master release HOT 1
- Job moved from inflight queue to comma-joined queue on worker shutdown in RESET_TO_HIGHEST_PRIORITY strategy.
- Worker existing on uncaught exception after 10 minutes of no connection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jesque.