framework receives less than master offers about pymesos HOT 5 CLOSED

Hmm，did your script actrually launch tasks with the offered resources？
If no tasks found in your framework UI，you may check Offers tab. All the offered resources should be displayed there.

PyMesos just a bridge for mesos and framework，and should not modify the allocation behaviour of mesos.

from pymesos.

got it - i can see jobs launching, and completing, but it seems like the resources requested are never released for some reason

from pymesos.

Looking closer, in the mesos web ui the tasks show up as finished, but future offers sometimes contain the executor ids for some past (completed) tasks. For every task im generating a new executor (with a new uuid id) and it seems like these executors gradually build up. Because I give the executors some resources as well (as the example does), mesos continues to count their resource usage as well, eventually causing the machine to run out of capacity while running nothing.

Is there something I should specifically do to terminate the executor and have mesos collect it?

from pymesos.

it looks like i needed to make sure to call driver.abort() to actually stop the driver - doing this fixes the problem

from pymesos.

Thanks for your share. Typically we just re-use the existing executor when possible, and the executor is running till framework ends. If you actually need short-lived executors, you may write a custom executor and let it exit when the task is finished. and you may receive a executorLost message to notify you the premature exit of the executor.

from pymesos.