Dagster
Dagster is a cloud-based data line orchestrator designed to improve and manage data assets. Python enables data assets defined as functions to be run at the right time and offers observatability with integrated genealogy throughout the entire development lifecycle. It serves as a central control plane at all stages, from local development to unit testing and production.
Dagster
90dagster-io · Workflow & Automation
Common commercial products in this category: Zapier, Make, Microsoft Power Automate
- Using Python functions, you define the data assets you want to build.
- With integrated observativity and genealogical properties, you monitor metadata centrally.
- By integrating with popular data tools, you take pipelines into production.
- • High number of open issues (2,602)
- ↓ NegativeHacker News
The new local data stack: Integrating Dagster, dbt and DuckDB
“Hey Georg, thanks for posting. I've been working on building a thing with almost the same stack lately and the Dagster integration is my next large-ish step. In the last couple weeks I've kicked around Duck and Clickhouse for the backend. It's been a lot of fun. Ultimately I'd…”
- ↑ PositiveHacker News
The new local data stack: Integrating Dagster, dbt and DuckDB
“Snowflake isn't local, you have to pay for cloud ... Airflow is Airflow, complexity, steep learning curve, there is a whole industry trying to be as/more powerful than Airflow with less complexity and cleaner integration with modern things you might want to do like k8s, although…”
My note
Alternative pages
More in: Workflow & Automation
- taste-skill★ 106,430 · today
- awesome-gpt-image-2★ 35,284 · 17 days ago
- Conductor★ 32,401 · today
- Kestra★ 30,086 · today
- Activepieces★ 25,301 · today
- Prefect★ 24,102 · today