Your Data Guy

Expert in

Get in Touch

About Me

I am a results-driven Data Engineer with a proven track record of architecting scalable data platforms that convert complex datasets into actionable business intelligence. At BGC (Australia) Pty Ltd, I led the modernisation of data pipelines across Microsoft Fabric, Azure Data Factory, and Apache NiFi—achieving a 40% reduction in batch processing time and significantly improving time-to-insight.

My recent initiatives include building a cost-efficient data lakehouse leveraging Apache Kafka, Hadoop, and Snowflake—delivering 30% infrastructure savings while maintaining enterprise-grade auditability and compliance standards. I excel at bridging the gap between technical execution and strategic objectives, mentoring high-performing teams, and collaborating cross-functionally to deliver business-aligned data solutions.

Fluent in English, Hindi, and Punjabi, I work effectively across diverse, cross-cultural teams. I’m also deeply engaged with emerging technologies such as Agentic AI and large language models, exploring their potential to revolutionize decision-making and unlock competitive advantage through autonomous data-driven systems.

Core Capabilities

Data Architecture Icon

Data Architecture

Design scalable, secure data platforms and pipelines from ingestion to consumption, delivering real‑time insights and strategic value.

Data Governance Icon

Data Governance

Implement robust governance frameworks, elevating data quality and compliance while maintaining lineage and auditability.

Cloud Solutions Icon

Cloud Solutions

Architect and optimise cloud‑native data solutions on Azure, AWS and Snowflake, delivering cost‑effective scalability and performance.

Experience

Data Engineer – BGC (Australia) Pty Ltd

Jan 2023 – Present
  • Streamlined ETL pipelines across Microsoft Fabric, Azure Data Factory and NiFi, reducing batch processing latency by 40% and accelerating time‑to‑insight.
  • Built a cost‑optimised data lakehouse using Apache Kafka, Hadoop and Snowflake, cutting infrastructure costs by 30% and ensuring governance.
  • Implemented data quality and governance checks that increased compliance from 85% to 98% and improved auditability.
  • Delivered enterprise‑grade data warehouse and real‑time integrations with high‑volume systems; designed dashboards with optimised SQL for a 35% performance lift.
  • Led proof‑of‑concepts (Fivetran/Airflow) and mentored engineers to elevate organisational data maturity.

Associate Data Engineer – INX Software

Jan 2022 – Jan 2023
  • Standardised and automated data workflows across teams using DataOps principles; cut deployment cycles by 50%.
  • Built and maintained cloud‑native ETL pipelines with Python, DBT and Snowflake, slashing processing times by 90% and reducing costs.
  • Integrated data from mining and workforce systems, enabling advanced analytics while ensuring data accuracy and compliance.
  • Implemented CI/CD and proactive monitoring for data products to improve reliability and minimise incidents.
  • Developed SQL warehouses and lakehouse solutions on Azure/AWS; mentored juniors to foster continuous improvement.

Skills

Data Engineering & Lakehouse Architecture
ETL Pipelines (ADF, NiFi, Fabric)
Cloud Platforms: Azure, AWS, Snowflake
Apache Spark, Databricks & Delta Lake
SQL, Python, PySpark (Advanced Analytics)
Machine Learning, LLMs & Agentic AI
Vector DBs (FAISS, Pinecone, ChromaDB)
DataOps, CI/CD & Data Observability

Licenses & Certifications

Confluent Data Streaming

Data Streaming Engineer

Confluent certification demonstrating proficiency with Apache Kafka and streaming data pipelines. Credential ID 153657031. Issued Jun 2025 – Expires Jun 2027.

SnowPro Core

SnowPro Core Certification

Snowflake credential validating expertise in Snowflake architecture, data warehousing and cloud data engineering. Credential ID 144446911. Issued May 2025 – Expires May 2027.

AI & ML

AI & ML Specialisations

Completed multi‑course specialisations on Coursera and edX covering self‑driving cars, natural language processing, TensorFlow and time‑series prediction from the University of Toronto and DeepLearning.AI. Demonstrates solid foundations in machine learning and deep learning.

Cloud & Data

Cloud & Data Platform Certifications

Certified across Azure (AI Fundamentals, Data Fundamentals, Fabric Analytics Engineer and Data Scientist Associate), AWS (Fundamentals, Security, Migration, Serverless & Cloud‑Native) and Google Cloud machine‑learning programmes, demonstrating versatile cloud data expertise.

Data & Analytics

Data & Analytics Tools

Skilled in dbt fundamentals and a variety of data science and programming courses including Python programming, geospatial analysis, database design, WordPress site development and more—equipping me to deliver end‑to‑end data solutions.

Recommendations

Nelson Goh

Nelson Goh

Senior Data Engineer at INX Software (Mentor)

Madhur’s drive and passion for data science are unmatched. His facial‑recognition project for INX Hack Day 2022 won the Most Innovative prize.

Satya Prakash

Satya Prakash

Utilities Engineer & Deputy Manager

A great motivator and exemplary senior who shares knowledge generously. Working with Madhur has been an inspiring learning experience.

Amin Beheshti

Amin Beheshti

Professor & Director, Centre for Applied AI

Madhur’s pristine work ethic and adaptability stood out in my classes at Macquarie University. He goes out of his way to help others.

Hassan Doosti

Hassan Doosti

Program Director & Senior Lecturer, Macquarie University

Knowledgeable, articulate and hardworking, Madhur brought integrity and intelligence to every task during his Master’s programme.

Vaibhav Rai

Vaibhav Rai

Business Analyst & Data Analyst

Multi‑talented and enthusiastic, he’s always eager to learn and implement new technologies and to lend a helping hand.

Gaurav Gupta

Gaurav Gupta

Director of Education, Macquarie University

One of the hardest‑working students I’ve taught; Madhur combines perseverance with honed analytical skills to achieve great results.

Er Gurupreet Singh

Er. Gurupreet Singh

Pre‑owned Vehicle Ecosystem Leader

A tech geek with an innovative mind and a gifted writer, he’s equipped with the right tools and mindset to excel.

Paras Sarna

Paras Sarna

Full‑Stack Developer

Can step into any area and conquer it; always learning new technologies and ready to support others.

Awards & Achievements

Hire Me

I'm currently open to both freelance and full-time opportunities. I specialize in building modern data solutions—from pipelines and lakehouses to real-time dashboards and AI integrations.

Contact Me or Book a Call

Contact

Interested in collaborating or discussing data engineering challenges? Feel free to reach out.