Hi Team, Could someone please confirm the maximum number of events that the Workday New/Updated Workers trigger can support at a time? I have a scenario where approximately 150K worker updates could occur simultaneously in Workday, which could result in the same volume being picked up by the Workato trigger. Can the trigger reliably handle this volume?
technically yes it aligns to Workdays 1M instance cap, but id have some concerns about the full end to end processing depending on the structure. is this the batch trigger or the single? is this a bulk update to 150k users so 150k seperate events or would some of these be on the same worker object? can you put in filtering? this would create a large backlog which depending on the recipe may take a very very long time to clear
it's single worker trigger and yes all 150k users will be updated at the same time so it's expected to reach workato trigger almost same time.So all 150k users will trigger same 150k events.All will be unique workers.
thats not the best fit use case wise for us since it would likely be fairly costly as well. ideally we could batch it somehow either doing the batch trigger or like csv export import
Since this is urgent and we are working within a tight timeline, we will need to proceed with the existing recipes, which use the single-worker trigger.For future implementations, I will definitely take your recommendation into consideration and use the batch trigger approach. On another note, just to better understand the trigger’s scalability: if the volume were as high as 2 million workers, including contingent workers, how would the trigger behave? Is there a limit on the number of events it can process, or can it handle volumes beyond 1 million events(keeping cost perspective aside for the time being)?
I believ Workday would fail. our connector says 1M docs.workato.com/en/connectors/workday/new_updated_object#… which is linked back to their docsdoc.workday.com/admin-guide/…/dan1370797408285.html
working to confirm but batching may be required. polling triggers typlically have a max queue of 10k jobs
so after 10k jobs is it going to throw 429 to many requests error?
Nikhil K. i chatted a bit with some of our workday experts and with the non batch trigger i do believe the recipe job queue would max and out likely fail. I believe the full process is we send the Get request for all new/updated workers, Workday sends back that there are 150k and on the first page gives the first 999. we then turn those into 999 jobs and ask for page 2 repeating through the 150k. however the polling queue will will at 10k causing errors. This is where batch would work. batching 50 at a time would put the cap of the jobs at 3k. so while it would take a while to process the queue would not fill. youd want to process each back in a loop in the main recipe since dumping them to a recipe function would also fill that queue.