Skip to main content

End-to-End Data Platform Demo

The tower-demo repository showcases a complete data platform for Orbita Supply Co., a fictional retail company. This comprehensive example demonstrates how Tower orchestrates a modern data lakehouse architecture with ingestion, transformation, analytics, and AI automation.

Overview​

The demo includes 20+ Tower apps organized into five categories:

CategoryAppsDescription
Ingestion5Extract data from Shopify, IoT sensors, and operational systems
Transformation4Build analytics-ready tables with dbt and Tower
Analytics4Interactive Marimo notebook dashboards
Orchestration2Multi-app pipelines for daily ETL and anomaly detection
Utilities1Demo data regeneration

Data Architecture​

The platform follows a medallion lakehouse architecture:

┌─────────────────────────────────────────┐
│             GOLD LAYER                   │
│  Business-ready aggregates & KPIs       │
│  • customer_360                          │
│  • product_performance                   │
│  • inventory_ledger                      │
└─────────────────────────────────────────┘
                   ▲
┌─────────────────────────────────────────┐
│            SILVER LAYER                  │
│  Cleaned, conformed, enriched data       │
│  • ticket_summaries                      │
│  • product_descriptions                  │
│  • anomaly_explanations                  │
└─────────────────────────────────────────┘
                   ▲
┌─────────────────────────────────────────┐
│            BRONZE LAYER                  │
│  Raw ingested data from sources          │
│  • orders, inventory, products           │
│  • warehouse_telemetry, returns          │
└─────────────────────────────────────────┘

Key Workflows​

Daily Retail Pipeline​

The daily_retail_pipeline app orchestrates a complete daily ETL:

  1. Ingest orders, products, and returns
  2. Build customer_360 and product_performance tables
  3. Generate AI-powered sales report
  4. Send Slack notification
tower run daily_retail_pipeline

Real-Time Anomaly Detection​

The warehouse_anomaly_pipeline demonstrates operational intelligence:

  1. Ingest latest warehouse telemetry
  2. Detect anomalies in sensor data
  3. Generate AI explanations
  4. Alert on critical issues
tower run warehouse_anomaly_pipeline

Technologies Used​

  • Tower: App orchestration, scheduling, secrets, and catalog management
  • Apache Iceberg: Open table format for the data lakehouse
  • dltHub: Python-first data ingestion
  • dbt: SQL-based transformations
  • Marimo: Reactive Python notebooks for dashboards
  • Claude: LLM for AI automation and insights
  • Polars & PyArrow: Columnar data processing

Getting Started​

Clone the repository and run any app locally:

git clone https://github.com/tower/tower-demo
cd tower-demo/ingest_shopify_orders

# Install dependencies
uv sync

# Run locally
tower run --local

Deploy all apps to Tower:

# From the repo root
for app in ingest_* run_* customer_* product_*; do
  cd $app && tower deploy && cd ..
done

Highlights: Medallion architecture, dltHub, dbt, Marimo notebooks, LLM automation, Slack notifications