Data Engineering Salaries -- Open Source Contribution

Each quarter we run a salary survey in our community to help increase transparency around salary and compensation for Data Engineering. This application is for exploring and analyzing the salary data.

Slack Bot (in development)

Slack Bot which consumes incoming events and pushes data to BigQuery and transforms data using dbt (Data Build Tool).

DBT Dimensional Modelling

DBT adventureworks model using STAR Schema and OBT Mart.

DBT BigQuery Pipeline

Online Retail data set contains all the transactions occurring for a UK-based and registered, non-store online retail between 01/12/2009 and 09/12/2011.The company mainly sells unique all-occasion gift-ware. Many customers of the company are wholesalers.

DBT MRR Model

dbt Monthly Recurring Revenue (MRR) using One Big Table (OBT) approach.

Introduction to Redis

Sample project using Redis with python.

Introduction to FastAPI

Sample project using FAST API with python to build custom API endpoints and postman to test it.

DBT udemy course

DBT udemy course - 2024 Mastering dbt (Data Build Tool) - From Beginner to Pro.

DBT Log Analytics

DBT data pipeline to inject raw log data to BigQuery table using kafka.

DBT Covid Dashboard

DBT project running on a local PostgreSQL database connected to a Streamlit dashboard.

DBT Jaffel Shop

DBT jaffel shop connected to snowflake database.

Kafka Event Processing

kafka application running on custom Docker alpine image, to listen to dummy app event data.

DBT (Data Build Tool) init

DBT bootcamp course setup files.

Big Query REST API Client

Python REST client to connect with BigQuery API and perform CURD Operations.

  • Create database with custom schema.
  • Create database from CSV/Excel file.

Expense Tracker App - Streamlit

Streamlit App, connected to Deta collection database to store user income/expense records🧾and visualize it.

Airflow Weather Service - Data Pipeline

An airflow DAGs to get weather data and store it in a CSV file.

Airflow Docker Compose - Data Pipeline

Airflow DAG to execute a Extract, Transform, and, Load operation.

Streamlit Weather Dashboard - Streamlit

Weather Data dashboard which displays current temperature (C), wind speed, wind direction and past 3 day forcast data. Made using openweather API and Streamlit.

Docker Weather API - Data Pipeline

A Weather service to get the live (every 10 minutes) weather data for Mumbai city and save it to PostgreSQL.

Kafka init - Data Pipeline

Kafka application to product data, and consume it using 1 consumer and 1 topic. Outputs data in every 3 seconds.

Notion + Streamlit ELT Pipeline - Data Pipeline

ELT workflow to read data from Notion Page using API and push it to Database.

Yahoo Finance ETL Pipeline - Data Pipeline

The Yahoo Finance ETL script extracts financial data from Yahoo Finance's API, transforms and cleans the data, and loads it into an PostgreSQL database.

Finance dashboard - Streamlit

Finance Dashboard utilizes the Yahoo Finance API and Streamlit to create a user-friendly and interactive financial analysis tool. With real-time market data and customizable visualizations, this project empowers users to track stocks, and analyze trends.

SQL Question Dashboard - Streamlit

Front End for Notion + Streamlit ELT Pipeline. The app displays my daily progress in solving SQL problems from various platforms such as LeetCode, StrataScratch, and more.

Avocado Analytics - Tableau

Analyze the behaviour of Avocado prices and the numbers of Avocados sold in the US 2017 Year. Build using Tableau

UFO Analytics - Tableau

A UFO Analytics dashboard, to analyze UFO sightings data from 1998 to 2014. Part of a Udemy Course Assignment.

Data Jersey - Dash

A powerful analytical Web-App designed for Football Clubs to report, strategize and analyze gameplay, player stats and performance.

Avocado Analytics - Dash

Analyze the behaviour of Avocado prices and the numbers of Avocados sold in the US 2017 Year. Build using Plotly Dash

mini-projects - python

This contains bunch of mini projects written in python.

Hackathons

Streamlit Connections Hackathon

Configuring data connections can be one of the most frustrating and time-consuming parts of building robust Streamlit apps. You have to locate and install external packages, determine the best way to securely manage your credentials, and figure out how to retrieve the data you need in the desired format.

Batlytics

Current college baseball analytics tools are far behind the professional ones, offering a large room for improvement, so we developed analytical tools aiming to improve college baseball performance.

Predicting Potential Yellow Jackets

The shift in the focus of student athlete recruitment from a performance-basis to a character-basis for a cultural fit led to a question of leveraging SNS feeds in student athlete recruitment.

#12 StonkViz

Creating new investment insights by visualizing trading attitudes on /r/WallStreetBets.

Gradebook

An analytics tool for teachers designed to recommend action plans using Machine Learning.

Syrecuse

Many small businesses have been negatively impacted by Covid 19 and are struggling to survive. We created a centralized location where small business owners can go to learn about resources available.

HackerEarth Machine Learning Challenge: Carnival Wars!

You plan to unleash the inner Machine Learning expert in you and build a sophisticated Machine Learning model that predicts selling prices of products based on the mentioned factors.