Scalable Data Science, course sets in big data Using Apache Spark over databricks and their mathematical, statistical and computational foundations using SageMath.

Using U-Net Model to Detect Wildfire from Satellite Imagery

DEPRECATED: Integrating Jupyter with Databricks via SSH

End-to-end data engineer project

The Goal of this project is to provide documentation for the Lakehouse Engine framework.

Samples for Azure Databricks Orientation

Microsoft Azure Container Ecosystem - "nugget" series

Implementation of the "CCF: Fast and Scalable Connected Component Computation in MapReduce" paper with Spark. Study of its scalability on several datasets using various clusters' sizes on Databricks and Google Cloud Platform (GCP)

This is a group project I worked on with my classmates. This project uses pyspark ML and created a SparkSession object.

🏂 A machine learning model that performs topic classification of news articles for media bias analysis. Final project for UC Berkeley MIDS 266 (Natural Language Processing)

Making ODBC connection from Databricks (Azure Databricks) to Azure SQL Database with Azure AD User Access Token.

Stocks Data Analysis In DataBricks - Using SQL and Pyspark

This is a code sample repository for demonstrating how to perform Databricks Delta Table operations.

R on Databricks Compendium

Tutorial on how to use the Python API for Spark dataframes.

Alumni Profile Matching is a project aimed at facilitating networking between graduate students and alumni with similar backgrounds and career goals. By leveraging machine learning techniques and data processing pipelines, the project aims to provide graduate students with personalized recommendations of alumni profiles to connect with.

A place to learn data engineering

Meu décimo primeiro projeto em que crio um datalakehouse usando computação distribuído no databricks

Analysis for a streaming daily retail data using Spark structured streaming and querying this data to get insights

POC projects working on Cloud Platforms