Ayush Kumar Sahu - Portfolio

About Me

Data Engineer with a strong foundation in building scalable data infrastructures, relational databases, and real-time processing pipelines. Proficient in the modern data stack including Azure, Databricks, Snowflake, and Kafka. Recently completed an MSc in Big Data Science with Distinction. Proven track record of automating complex workflows, enforcing data normalisation, optimising SQL performance, and deploying robust ETL/ELT pipelines in hybrid cloud environments.

Current Role

Data Engineer | ByteIQ Analytics

Jan 2026 – Present | Remote

Managed hybrid cloud infrastructure across Azure and NIC Cloud, implementing CI/CD pipelines to automate deployments.
Engineered Python (Pandas, SQLAlchemy) pipelines between MySQL and PostgreSQL, enforcing strict data normalisation and ACID compliance.
Architected robust ELT pipelines using Databricks for distributed processing and dbt for modular SQL transformations.

GitHub Activity

Technical Expertise

Languages & Databases

Python (Pandas, SQLAlchemy) SQL (Advanced) PostgreSQL MySQL Scala Bash MongoDB

Cloud & Data Eng

Azure AWS Databricks Snowflake Apache Kafka Airflow dbt

AI & DevOps

LangGraph Azure OpenAI Docker CI/CD Terraform Git

Tools & Arch

DBMS Architecture ACID Compliance Data Normalisation TCP/IP Networking Power BI Agile/Scrum

Education

Queen Mary University of London

M.Sc. Big Data Science (Distinction)

2024 - 2025

SRM IST Chennai

B.Tech Electronics & Communication

2019 - 2023

Certifications

Azure Data Engineering (Grow Data Skills)
Google Data Analytics (Coursera)
SQL Associate (DataCamp)
Anthropic Claude Code in Action
AI & Machine Learning (upGrad Campus)
Data Science & ML Course (Coding Ninjas)
Python Foundation & Data Structures (CareerLabs)
Power BI Data Analytics (Codebasics)
Google Cloud & AWS Networking (Coursera)
RHCSA, RHCE, AWS & Docker (COSS)
Introduction to Python (Coding Ninjas)

Data Engineering & AI Projects

Data Engineering

End-to-End Databricks Data Engineering Pipeline (Flight Data)

Architected a Medallion Architecture pipeline on Databricks to process large-scale flight data. Engineered ETL transformations using PySpark, Auto Loader, and Delta Live Tables.

Databricks PySpark Delta Live Tables

AI & Engineering

Multimodal AI Compliance Agent (LLMOps)

Engineered a scalable AI compliance agent using LangGraph and Azure OpenAI. Deployed a robust FastAPI backend with comprehensive LLM observability via LangSmith.

LangGraph Azure OpenAI FastAPI

Data Engineering

Real-Time Stock Market Data Pipeline

Designed a high-throughput streaming architecture using Apache Kafka. Orchestrated complex data workflows via Airflow in Docker, loading into Snowflake.

Apache Kafka Snowflake Airflow

AI & Engineering