#

airflow

Here are 42 public repositories matching this topic...

josephmachado / beginner_de_project

Beginner data engineering project - batch edition

python emr docker engineering airflow database spark etl redshift redshift-cluster

Updated Jan 22, 2025
HTML

andresionek91 / airflow-autoscaling-ecs

Airflow Deployment on AWS ECS Fargate Using Cloudformation

airflow data-engineering airflow-ecs airflow-autoscaling-ecs airflow-deployment

Updated Jun 22, 2022
HTML

apache / airflow-site

Apache Airflow Website

airflow hugo apache

Updated Aug 6, 2025
HTML

ms32035 / airflow-dag-dependencies

Visualize dependencies between Airflow DAGs

airflow

Updated May 7, 2021
HTML

josephmachado / data_engineering_for_beginners_code

Code for DE101 book at https://de101.startdataengineering.com/

python airflow sql spark dbt

Updated Jul 25, 2025
HTML

luchonaveiro / open-source-data-stack

Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.

docker airflow etl postgresql superset dbt great-expectations

Updated Jun 19, 2022
HTML

erogluegemen / Earthquake-Emergency-Response-Robots

The Earthquake Emergency Response Robots project aims to create, develop, and implement systems specifically designed to handle post-earthquake situations. The main focus of the project is to build adaptable robots that come equipped with sensors and communication capabilities.

python docker raspberry-pi airflow django mongodb postgresql cnn embedded-systems apache-server yolov10

Updated Jun 13, 2024
HTML

mathewsrc / ETL-Chicago-Cafe-Permits

This ETL (Extract, Transform, Load) project employs several Python libraries, including Airflow, Soda, Polars, YData Profiling, DuckDB, Requests, Loguru, and Google Cloud to streamline the extraction, transformation, and loading of CSV datasets from the U.S. government's data repository at https://catalog.data.gov.

python docker bigquery airflow continuous-integration data-visualization astro soda data-quality-checks extract-transform-load github-actions duckdb polars looker-studio astro-python-sdk ydata-profiling

Updated Dec 10, 2023
HTML

piyush-an / NYC-Restaurant-Inspection

A data warehousing application on NYC Food Inspection

docker postgres airflow metabase dbt great-expectations

Updated Apr 26, 2023
HTML

gaurav-aiml / ecommerce-data-pipeline

As a Data Engineer for a fictional E-commerce startup, this project addresses the task of analyzing the web server logs to find the number of product pages visited and the number of items in the cart.

airflow real-time kafka spark-streaming data-pipeline data-engineer

Updated Mar 28, 2021
HTML

hectoramirez / Covid19

Live COVID-19 tracker with Airflow

airflow notebook bokeh visualizations covid-19

Updated Aug 13, 2024
HTML

ElfatihZiad / BBCNews-scraper-nlp

A data pipeline to extract News articles from BBC News, storing it to MongoDB, performing NLP analysis (Topic modeling, sentiment analysis), and orchestrating with airflow.

nlp docker data airflow mongodb sentiment-analysis data-engineering scrapy topic-modeling

Updated Aug 31, 2022
HTML

rocs-org / data-pipelines

Data engineering monorepo with Airflow, dbt and postgres

python postgres airflow dbt

Updated Apr 13, 2023
HTML

anilkulkarni87 / airflow-tables-postgres

I generated the list of tables using Schemaspy and hosting it on my github. Definitely helps in understanding the backend tables of airflow.

airflow airflow-db

Updated Aug 12, 2021
HTML

gurug-dev / distributed_data_systems_project

Sentiment Analysis and SparkML modeling on Financial Data using HuggingFace, Spark, MongoDB, Airflow and GCS.

airflow mongodb sparkml finbert

Updated Mar 23, 2023
HTML

drobim-data-engineering / Capstone-Project-Data-Engineering-Nanodegree

python aws airflow data-engineering redshift datawarehouse

Updated Sep 18, 2020
HTML

nebasam / Migration-to-postgres-and-superset

The main aim of this project is to migrate a data warehouse tech stack from MySQL to PostgreSQL data warehouse and redash to superset dashboard.

mysql postgres airflow superset redash dbt

Updated Oct 5, 2021
HTML

AjNavneet / Loan-Eligibility-Prediction-Pipeline-Airflow-Docker

End-to-end ML pipeline model and drift monitoring for Loan Eligibility classification.

docker airflow pipeline end-to-end-testing drift-detection loan-eligibility

Updated Dec 9, 2023
HTML

khadkarajesh / wine-prediction

White and Red Wine classification using logistic regression

Updated Dec 24, 2021
HTML

aroraayush / airflow-spark-stackoverflow-logs-analysis

Ask Ubuntu Logs analysis with PySpark on GCP | Pipeline with Airflow (Cloud Composer)

airflow spark ubuntu stackoverflow batch-processing askubuntu cloudcomposer log-processing

Updated Jan 22, 2021
HTML

Improve this page

Add a description, image, and links to the airflow topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the airflow topic, visit your repo's landing page and select "manage topics."