Comments (3)
I think it is better to let user decide whether to run a particular callback synchronously or asynchronously.
If you decide to run it asynchronously, you can use thead, threadpool, coroutine or anything you like to run it. pymesos
neither have the knowledge of the user's need, nor is able to force user to use any kind of these solutions.
And assume, we are now callback in another thread, for example _io_thread
. we would also be blocked infinitely, because statusUpdate
cannot be called in _io_thread
as it is occupied by reconcile_all_running_tasks
.
If you aware that the old pymesos
(<0.2.0) is using two threads instead of one, the main concern is we should not make IO lagged by slow callback. it is not for blocking issue. In this new version, we decide to simplify our design for better maintainability. and we see 20% performance boost in our production environment. I think it is worthy.
Hope my explanation is helpful.
thx
from pymesos.
you are right, running all callback in a new thread can't solve this problem, but this is a trap to the user of this package. this package needs documentation. anyway thx
from pymesos.
yes, documentation is needed badly.
we will see whether we can spare some time to document this.
thank you for your support during this long time.
from pymesos.
Related Issues (20)
- Using 'offer_id' instead of 'inverse_offer_id' in RescindInverseOffer
- MesosOperatorMasterDriver cannot subscribe to events HOT 7
- Process.start() method does not call Process._notify() HOT 7
- Missing comma in list.
- missing kazoo dependency HOT 5
- Exponential backoff for executors incompatible with default & max executor_reregistration_timeout HOT 1
- Add support for HTTPS HOT 2
- Add support for executor authentication HOT 1
- Add specific timeout for http requests HOT 3
- pymesos doesn't install on python 3.7 HOT 5
- Ability to run 10,000 tasks HOT 1
- Pymesos doesn't install properly on python3.11 due to http_parser
- python version, is python2 and python3 compatible? HOT 2
- Scheduler failover flag is unsettable HOT 13
- Socket errors are not handled properly which is resulting in unexpected behavior HOT 2
- how do i get status of a task from scheduler? HOT 3
- Custom field on update dictionary not working HOT 2
- Error parsing `mins` and `ns` time units since items()
- Framework does not get HeartBeats HOT 1
- Timeout for connection refused errors?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pymesos.