RussellJ thanks again for taking the time to think through this and share your suggestions—I really appreciate it. I tried creating a small POC based on your REPEAT...WHILE idea, and it actually worked! I was able to repeatedly invoke the Wait for async calls to complete action until all the child recipe executions completed. I created a small POC with one parent recipe and three child recipes. Each child recipe had a different sleep time configured—5 minutes, 15 minutes, and 20 minutes, respectively. From the parent recipe, I invoked all three child recipes asynchronously, stored their job reference IDs in a list, and configured the Wait for async calls to complete action with a timeout of 12 minutes. Using the approach you suggested—repeatedly invoking the Wait for async calls to complete action inside a REPEAT...WHILE loop based on the Job timed out count—I was able to make the parent recipe wait until all the child recipe executions had completed successfully. I also noticed that in the Wait for async calls response, there were jobs where Job Completed at was null, and those responses included the Job ID. I'm not entirely sure whether the Job ID is the same as the Job Reference ID used by the wait action, but if they are equivalent, that could be another possible approach by querying the job status until all jobs complete. For now, though, I think the REPEAT...WHILE approach is much simpler and cleaner, so I'm going to test it with my actual use case and data volume. I'll post another update once I've completed that testing. Thanks again for pointing me in this direction—it was a really helpful suggestion!😄 I'd also love to connect with you on LinkedIn and stay in touch. It's always great to build a stronger community and learn from each other's experiences. My LinkedIn profile is below: linkedin.com/in/vivek-sharma-0097a2218
Hi RussellJ,Thanks for taking the time to reply and share your thoughts—I really appreciate it! I completely agree with your point that having child recipe executions run for more than 24 hours isn't ideal. In a typical scenario, I'd also look at optimizing the processing time. However, in my use case, we're integrating millions of records, and the third-party platform API we're calling doesn't support batch processing. As a result, the child recipes end up processing a very large volume of requests, which is why some of them exceed the 24-hour wait limit. I think your suggestion about checking the Job timed out count and repeatedly invoking the Wait for async calls to complete action is definitely worth exploring. I'm going to try this approach as a POC and see whether calling the wait action multiple times with the same job reference IDs continues waiting for the remaining jobs. I'll share my findings here once I've tested it. Thanks again for the suggestion!😄
Hi Everyone, I'm looking for some guidance on a use case involving asynchronous child recipes. In my parent recipe, I invoke a child recipes asynchronously multiple times for paralled processing. For each async call, I store its job reference ID in a list. Later in the recipe, I use the "Wait for async calls to complete" action, passing the list of job reference IDs so that the parent waits until all child recipes have finished. The challenge is that some of these async calls are taking a very long time to complete. The "Wait for async calls to complete" action has a maximum timeout of 24 hours. One or more async calls are still running after this timeout, the wait action times out and the parent recipe continues its execution. As a result:
Some child recipe jobs are still in the Processing state.
The parent recipe proceeds with downstream steps, even though not all child recipes have completed.
This leads to downstream processing starting before all required child recipe processing has finished, causing inconsistencies. Has anyone faced a similar scenario? I'm looking for recommendations or best practices for handling long-running asynchronous child recipe calls. Specifically:
Is there any workaround to wait beyond the 24-hour limit?
Is there a recommended design pattern for coordinating very long-running async recipes?
