I want to do a certain task periodically (per day) on our Web/Worker role. I have multiple instances in my Cloud Service, and I want only one of these instances to do this task per day (for example Instance0 can do it one day, next day it could be Instance1 doing the work, but 0 and 1 will not try to do the same work during the same day/period)
Azure queues seem to be a great way to achieve this because by design only one instance will dequeue the message (assuming it deletes it after doing the work).
What I am having trouble with is figuring out a way to put only one copy of this message in the queue per day. The only way I have figured to do this is by enqueuing a message every day from Azure scheduler jobs.
My problem with Azure scheduler is the fact that I need to create a job for every single storage account I have across all of my deployments.
Is there a way to do this from within the cloud service, without taking the scheduler dependency?
If you don't want to have Scheduler dependency, consider using Blob leases as a semaphore of sorts. http://justazure.com/azure-blob-storage-part-8-blob-leases/
At a certain time of the day, have your worker instances compete to get a lease on some central storage blob. Whoever gets the lease, prevents other instances from getting that lease and can queue up messages into the queue.
Having said that, why are you afraid of Scheduler dependency? Have it kick off a single job that will queue up a "start work" message. Have your instances monitor that queue. Whoever picks up that message, can then run thru all of your storage accounts and queue up individual storage-work messages for all instances to pickup.
If you know you need to perform one job a day I don't understand why you need to use a queue - you could just have a scheduled job run once per day. If the job only needs to run if a certain condition is met I would just build this logic into your scheduled task - maybe by setting a property in a Table or something like that.