Data Pipelines & ETL

PR

Prefect

4.6/5.01 verified review

Free·Open Source·Apache-2.0

Workflow orchestration with Python for data engineering and AI systems.

• Native Python workflow definition without complex domain-specific languages.
• Durable execution architecture with automatic retries and error recovery.
• Solid open-source core with self-hosted and cloud deployment options.

24.1kstars on GitHub

DA

Dagster

4.6/5.01 verified review

Freemium·Open Source·Apache-2.0

Asset-centric data orchestration platform for modern engineering and AI pipelines.

• Asset-centric approach instead of simple scheduled tasks
• Excellent end-to-end data lineage and quality visibility
• Solid open-source core with managed cloud options

16.4kstars on GitHub

AS

Astronomer

4.5/5.01 verified review

Paid·$250/mo·Apache-2.0

Data orchestration and AI pipeline platform powered by Apache Airflow.

• Optimized runtime that outperforms standard open-source Apache Airflow.
• Built-in support for AI-driven workflows and data agent architectures.
• Disaster recovery capabilities with under 15-minute RPO.

459stars on GitHub

DL

dbt Labs

4.5/5.01 verified review

Freemium·Open Source·Apache-2.0

The open standard for modern, AI-ready data transformation using SQL workflows.

• SQL-first data transformation with native version control
• Built-in testing and local validation before production deployment
• Interactive lineage and clear visual dependency graphs across models

14.1kstars on GitHub

DA

DataHub

4.5/5.01 verified review

Freemium·Open Source·Apache-2.0

Data context and governance platform for engineering and artificial intelligence teams

• Unified context graph connecting lineage, quality, and real-time metadata
• Strong open-source community adoption with thousands of active organizations
• Extensible architecture designed to integrate with AI agent workflows

12.9kstars on GitHub

GE

Great Expectations

4.4/5.01 verified review

Free·Open Source·Apache-2.0

Open source data quality framework for testing, validating, and documenting pipelines.

• Automatic generation of visual HTML data documentation.
• Native integration with Python and notebook environments like Jupyter.
• Extensive support for various storage backends and SQL engines.

11.9kstars on GitHub

AT

AtroPIM

4.1/5.01 verified review

Freemium·Open Source·GPL-3.0

Highly configurable, modular, open-source PIM system with headless capabilities.

• Highly modular and extensible architecture.
• Full deployment flexibility (cloud or on-premises).
• Comprehensive REST API covering all custom configurations.

211stars on GitHub