Orchestrate apps
Orchestration in Tower gives you control over when and how your apps execute, allowing you to automate and compose more complex flows.
Tower's orchestration capabilities help you:
- Run apps on demand
- Define a control flow and run several apps in parallel or in a sequence
- Schedule regular runs at specific intervals
Run Apps On Demand
Run Apps in CLI
Use the Tower CLI when you need to run an app on demand or as part of your own automation scripts:
tower run my-app
For additional options like passing parameters, see the tower run command documentation.
Run Apps in Tower UI
Navigate to the App Details page, click on the "Create Run" button on the top right of the screen and follow the prompts to launch a Run.
Run Apps via Tower API
For programmatic control or integration with external systems, use the Tower API:
curl -X POST https://api.tower.dev/v1/apps/<MY_APP>/runs \
-H "Authorization: Bearer <MY_API_TOKEN>"
See the Run App API documentation for complete details.
Compose Flows
Tower apps can call other Tower apps and wait for their completion. The run and wait helper functions allow you to define flows of any complexity.
The control flows can be as simple as running two or three apps in a sequence. They can also be as complex as creating a list of app runs depending on the input parameters or the data your received, launching these apps in parallel execution, waiting for the completion of all or some of these apps, and then doing some post-processing at the end.
Run Apps
To run an app, use the run_app helper function and specify the app slug.
You can also pass parameters to the app the same way you pass parameters via the CLI or API.
from tower import run_app
# Run an app with default environment
run = run_app("my-app")
# Run an app with custom environment and parameters
params = {"input_file": "data.csv", "output_dir": "results"}
run = run_app("my-app", parameters=params)
To run multiple apps in parallel, call run_app multiple times.
You can even launch apps dependent on the input data, and you can do it in loops or conditionals. This allows you creating dynamic control flows.
Tower example fan-out-ticker-runs shows how to:
tickers_str="AAPL,MSFT,GOOGL"
ticker_list = [ticker.strip() for ticker in tickers_str.split(",")]
# Process each ticker's data
child_runs = []
for ticker in ticker_list:
params = {
"PULL_DATE": f"{pull_date_str}",
"TICKERS": f"{ticker}"
}
run = tower.run_app("write-ticker-data-to-iceberg", parameters=params)
child_runs.append(run)
In the above example, multiple runs of the same app "write-ticker-data-to-iceberg" will be started, one per ticker.
Each run will have a different value for the ticker parameter.
Wait for App Run Completions
In a typical control flow you will want to wait for the completion of a run or set of runs before proceeding to the next step. You might also want to deal with failure of one of the runs.
The wait_for_run() helper function will wait for the completion of the run that you pass as parameter.
It will return a Run object that you can inspect for run status.
run = tower.run_app("my-app")
# Wait for a run to complete
final_run = tower.wait_for_run(run)
Its multi-run sibling wait_for_runs() waits for the completion of multiple runs that you started.
It will return a tuple containing two lists on Run objects: successful_runs and failed_runs.
You can inspect these lists to define what to do next, e.g. retry one of the apps, log the errors and terminate etc.
In our Tower example fan-out-ticker-runs, we use wait_for_runs and pass the child_runs list that we created by launching multiple apps.
In return we get the runs that were successful and not.
successful, unsuccessful = tower.wait_for_runs(child_runs)
print(f"Successful ticker downloads: {len(successful)}")
print(f"Unsuccessful ticker downloads: {len(unsuccessful)}")
If, instead of waiting for the completion of every run you want to break on any failure, use the raise_on_failure=True parameter.
# Wait for multiple runs with custom settings
try:
successful, unsuccessful = tower.wait_for_runs(child_runs, raise_on_failure=True)
except RunFailedError as e:
print(f"One or more runs failed: {e}")
Schedule App Runs
Schedules allow your apps to run automatically at predetermined times recurrently. You manage app schedules separately from apps and environments. Schedules can be managed in using the Tower CLI or the Tower UI.
Adding a schedule
Tower CLI provides commands for adding schedules directly from your command line.
tower schedules create --help
Create a new schedule for an app
Usage: tower schedules create [OPTIONS] --app <app> --cron <cron>
Options:
-a, --app <app> The name of the app to schedule
-e, --environment <environment> The environment to run the app in [default: default]
-c, --cron <cron> The cron expression defining when the app should run
-p, --parameter <parameters> Parameters (key=value) to pass to the app
-h, --help Print help
tower schedules create --app="hello-world" --cron="every 15 minutes" --parameter=friend=Brad --parameter=foe=Alice
Alternatively, you can use the Tower UI to create and manage schedules. You can find the schedules editor in the top right corner of every Tower screen.
Listing schedules you have configured
tower schedules list
ID App Environment Cron Status
-------------------------------------------------------------------------------------
529fa2f2-6568-4fed-842c-a765f7e94228 tower-daily production 0 0 6 * * * active
fdbd0276-787b-4c2e-8db1-b2c6be1f546c tower-daily production 0 0 7 * * * active
297af2b9-0e35-49de-8ce6-6eac6945b425 tower-daily production 0 0 6 * * * active
Schedule Syntax Options
Tower supports two schedule syntax formats:
Natural Language Format: Some examples of acceptable schedules.
every 15 minutes
every hour
Cron Syntax:
For more complex scheduling needs, use standard cron syntax:
*/15 * * * * # Every 15 minutes
0 * * * * # Top of every hour
0 2 * * * # 2am daily
0 9 * * 1 # 9am every Monday
0 0 1 * * # Midnight on first of every month
Schedule Limitations
- Minimum interval is 1 minute
- Schedules are executed in UTC time zone
Managing Schedules
Updating a Schedule
To modify an existing schedule, you can again use either the Tower CLI or the schedules editor in Tower UI.
tower schedules update --help
Update an existing schedule
Usage: tower schedules update [OPTIONS] [SCHEDULE_ID]
Options:
-c, --cron <cron> The cron expression defining when the app should run
-p, --parameter <parameters> Parameters (key=value) to pass to the app
-h, --help Print help
Removing a Schedule
To delete a schedule, you can also use either the Tower CLI or the schedules editor in Tower UI.
tower schedules delete --help
Delete a schedule
Usage: tower schedules delete [COMMAND]
Options:
-h, --help Print help
Viewing Scheduled Runs
Monitor your scheduled runs in the Tower UI:
- Navigate to your app's details page
- The next scheduled run shows you the next scheduled execution
- The history of scheduled runs and their status are also found on this page
Orchestration Best Practices
For effective app orchestration in Tower:
Schedule Frequency
- Consider data volumes: Schedule frequency should align with expected data processing volumes
- Resource efficiency: Avoid scheduling too frequently if data doesn't change often
- Dependency awareness: Schedule dependent apps after their upstream data sources or dependencies get updated. Or use control flows to orchestrate multiple apps.
Monitoring
- Regularly check the Tower dashboard for failed scheduled runs
- Set up notifications for scheduled run failures (Coming soon)
Troubleshooting
If scheduled runs aren't executing as expected:
- Verify the app has been successfully deployed with the schedule
- Check for any error messages in the Tower logs
- Ensure your app has the necessary permissions and that all Secrets are up to date
- Confirm your schedule syntax is valid