Airflow vs. Prefect: Cron Schedules vs Event-Driven Flows
Two answers to the same two questions - who decides the shape, what starts the run
- Left column - Airflow, static and config-driven:
- Scheduler parses DAG files into a static graph
- Structure is declared before anything runs
- Triggers are time-based, plain cron lines
- Fits known, fixed batch pipelines
- DAG: a static graph of tasks
- Right column - Prefect, dynamic and code-first:
- Python code registers flows with an API server
- Agents schedule and run the flows
- Workflow is built at runtime, not ahead
- Reacts to events: new data, API calls
- Flow: a code-defined dynamic workflow
- One axis, not a hard limit:
- Default philosophies, not hard walls
- An Airflow DAG can be generated in code
- A Prefect flow can run on a plain schedule
- The honest test:
- Build the same 3-step ETL in both tools
- Change one branch mid-deploy, time the fix
- Airflow re-parses files; Prefect re-registers code
- Fire both from a file-arrival and a cron line
- Drop one event message and watch each fail
Sticky note - the one line:
Static before the run, dynamic at the run.