Introducing Wait for Workers - Workflow Synchronization Made Easy
Workflow synchronization just got a whole lot easier with our new Wait for Workers worker! This powerful addition to the ApudFlow platform allows you to coordinate parallel workflow branches and ensure operations run in the correct order.
What is Wait for Workers?
The Wait for Workers worker monitors the execution status of other workers in your workflow and waits until all specified workers have completed their tasks. It's perfect for scenarios where you need to:
- Synchronize parallel data processing branches
- Wait for multiple API calls to complete
- Coordinate dependent operations
- Ensure data availability before proceeding
How It Works
Auto-Detection Mode (Recommended)
Simply connect workers to your Wait for Workers node, and it will automatically detect and monitor all connected workers. No manual configuration needed!
Manual Mode
For advanced use cases, you can manually specify worker IDs to wait for specific workers that may not be directly connected.
Key Features
- Automatic Detection: Intelligently detects connected workers from workflow topology
- Real-time Monitoring: Periodically checks worker status in the database
- Timeout Protection: Configurable timeout to prevent infinite waiting
- Error Handling: Optional failure on any worker error
- Parallel Coordination: Perfect for synchronizing multiple parallel branches
Configuration Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
worker_ids | array | [] | Worker IDs to wait for (leave empty for auto-detection) |
check_interval | number | 1.0 | Seconds between status checks |
timeout | number | 300.0 | Maximum wait time in seconds (0 = no limit) |
fail_on_error | boolean | false | Fail immediately if any worker encounters an error |
Return Values
The worker returns a comprehensive status report:
{
"completed": ["worker_id_1", "worker_id_2"],
"failed": [],
"timeout": false,
"total_waited": 2.5,
"auto_detected": true
}
Example Use Case
Imagine you have a workflow that:
- Fetches stock data from Yahoo Finance
- Simultaneously processes the data with an LLM for analysis
- Needs both results before generating a final report
With Wait for Workers, you can ensure the final report generation waits for both the data fetch AND the LLM analysis to complete.
Getting Started
- Add a "Wait for Workers" node to your workflow
- Connect your parallel workers to the wait node
- Configure timeout and error handling preferences
- Connect the wait node to your downstream processing
The worker will automatically detect and wait for all connected workers to complete!
Watch the Tutorial
For a visual guide on how to use the Wait for Workers worker, check out our tutorial video:
This new worker significantly simplifies workflow coordination and makes building complex, parallel processing pipelines much more intuitive. Try it out in your next workflow!