By Leveraging the scale unit, you will get the two tasks which will create the table storage, as well as the Task.

Batch keeps on growing, even with large processing times.

How do you ensure that a MAX number of dequeuing is occuring?

Create a -poison queue for every queue, and handle messages that are poisoned (ie. over a certain threshold) and push them into the poison queue.

Enable scaling of dequeuers based on queue length.

Enable an easy way to:

• Create Shards
• Queue to Shard 1-10
• Dequeue from Shard 1-10

Logging output:

Starting 2 tasks
Finished starting tasks 0/2 successfully.
Run finished

Look at if it would be better to lock on a blob, instead of pumping data into table storage.

The ability to start tasks with delays between groups.
use case: initialization Tasks, where secondary tasks are dependent on resources being created.

Update dependency versions to latest version.

Do the batch sizes grow even when a smaller number is dequeued?

Determine if we need to have an auto-scaler based on queue length, might be enough to base it on the Dynamic Runners.

Enable the Role Task Manager to take an array of factories, so that people can break down their factory code; into simple focused pieces.

Make the docker container -e for the Storage Connection String.

To make sure that logs are readable, we need to have the class name of the processor.

Coordinated service; needs to check the 'queue' more frequently than timing is set to.

When multiple processes are started at near the same time, with similar timings there can be competition for resources when running.
All tasks should be started with a second or a half second in between so that they are spread out better.

We need a way to log metrics, of runs and duration of runs.
Look into pluggable, open source frameworks.
Worst case we plug into table storage and take the dependancy when it is enabled.

There should be a basic runner, which allows dequeue pollers to be run at even intervals.

We should have timings that are the opposite of 'trying' to run as fast as possible...

Inverting the Backoff task might be valuable in some cases.

If the current batch size isn't full, how do you know you can successfully handle a bigger batch size?

King.Service.BackoffRunner+King.Service.Timing.BackoffTiming: Task Completed (Duration: 00:00:00.0231845).;

Currently we use IRunnable to create a task, that has a one time run. I would like to create a base class, or make it really simple to have a task that is run first in priority for initialization tasks.

Switch to NUnit for testing

Restart Tasks when they fail....
Tasks like Init tasks are not retried/restarted when they fail. However, if you are running long running tasks, like WCF/Event based task, if a failure occurs you need it reloaded.

We should be able to call code using a Lambda:
{
x => this.Method(22);
}

Code:

yield return new InitializeQueueTask(config.QueueName, config.ConnectionString);
yield return new InitializeTopicTask(config.TopicName, config.ConnectionString);
yield return new InitializeSubscriptionTask(config.TopicName, config.Subscription, config.ConnectionString);

yield return new CompanyQueuer(client);
yield return new CompanyDequeuer(client);

Logs:

King.Service.ServiceBus.Demo.Tasks.CompanyDequeuer started: True.
King.Service.ServiceBus.Demo.Tasks.CompanyQueuer started: True.
King.Service.ServiceBus.InitializeSubscriptionTask started: True.
King.Service.ServiceBus.InitializeTopicTask started: True.
King.Service.ServiceBus.InitializeQueueTask started: True.

Create range class for general ranges
Range
Max X
Min X

By providing a Task wrapper that knows how the task is performing, we can infer how many instances of said task should run.
Take for example a dequeue task that is sustaining 'full speed' for a long duration; it is easy to add another dequeue instance working off of the same queue to enable more throughput.
To ensure that no one piece of code hogs all of the systems resources we can put in place a governer that would ensure a minimum instance count, and a maximum instance count.

Adaptive Timing; buys some flexibility, but mostly is for cost saving purposes (let the service 'sleep' at night)
Dequeue Single/Batch: has to be done 'manually' in code
Scaling based on message build up in queue; add more handlers as the queue fills up

I think that the matrix here is really: Cost Vs. Throughput; where cost and throughput can be optimized automatically. low volume queues are find to dequeue single messages at low rates. as soon as you are hitting the single digit seconds time frame then it starts to make sense taking batches at slower rates; lastly if you are still unable to keep up, add more dequeue handlers.

Further breakdown is required, and figuring out 'when' the right times are is important, as well as ensuring that a single workload doesn't saturate a server (is something to think about).

Run init tasks once, and pull them off the stack

Currently they are held in memory as a background thread, which is relatively expensive.

https://github.com/Microsoft/dotnet/blob/master/dotnet-developer-projects.md

Determine the best way for the framework to handle the release renew.

For WCF, we use Initialize task, I don't think there is a great way to scale this...
For Service Bus Event Handlers, we can scale those by having more instances...

There should be a trace "Started y of x services successfully"

Prefer TimeSpan over integers; to ensure that all code is convenient

Current Backoff task: when work is done goes to minimum value...
Would it be more beneficial, if it stepped down to the previous attempt time frame?
That would mean that it would slowly get faster, as well as slowly get slower.

It wouldn't be ideal in a case where a large amount of work comes in quickly, especially if there are few instances dealing with the work load.

It is possible to miss the minute that the Run At task should fire at. Currently signalling every 61 seconds, this needs to be every 60 seconds to ensure we hit the minute.

Initialize tasks not firing as period is too long for Timer?

[WaWorkerHost.exe] King.Azure.BackgroundWorker.Data.InitializeTableTask: Period must be less than 2^32-2.

log the instance that completed the work in the scheduling table.

Dequeueing is failing and not grabbing items out of the queue. Please assist.

It will be great if readme will cover the difference with Azure Functions and Azure Automation.

Check out what it would take to get this running on Mono

We should have a dequeuing task baked in with proper interfaces, so that we can use them to build out Service Bus dequeuing.

There should be a service which knows how to do linear back off, when the service is not needed.