Posted 5 months ago

Lead Data Engineer - R01558812

ChennaiOn-siteFull-time

AI Summary

Senior Data Engineer skilled in Azure Data Platform and Microsoft Fabric, responsible for designing and building end-to-end data pipelines, lakehouse architectures, and analytics-ready data models using Fabric and Azure services.

About this role

Lead Data Engineer

Primary Skills

AKS, Event Hub, Azure DevOps, Cosmos DB, Azure Functions

Specialization

Azure Data Engineering Advanced: Senior Data Engineer

Job requirements

Job Description

We are seeking a Senior Data Engineer with strong hands‑on expertise in Azure Data Platform and Microsoft Fabric to design, build, and scale modern, enterprise‑grade data solutions. The role involves working on end‑to‑end data engineering pipelines, lakehouse architectures, and analytics‑ready data models using Azure and Fabric’s unified analytics capabilities.

Key Responsibilities

Design, develop, and manage end‑to‑end data pipelines using Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake) and Azure Data Factory

Build scalable data transformation and processing solutions using Fabric Notebooks, PySpark, Python, and SQL

Implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold) to support analytics and reporting workloads

Develop and optimize SQL-based solutions including T‑SQL, stored procedures, and performance‑tuned queries

Integrate data from multiple sources including RDBMS, APIs, files, streaming sources, and cloud storage

Work with ADLS Gen2, Blob Storage, and OneLake for structured and unstructured data management

Enable analytics and reporting by integrating Fabric datasets with Power BI

Implement data quality checks, monitoring, logging, and error‑handling mechanisms

Support migration and modernization initiatives from ADF / Databricks to Microsoft Fabric, where applicable

Ensure adherence to data governance, security, lineage, and compliance standards (Microsoft Purview exposure preferred)

Collaborate closely with architects, analysts, and business stakeholders to translate requirements into scalable data solutions

Provide technical guidance and mentoring to junior engineers

Required Skills & Expertise

Microsoft Azure & Fabric

Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake)

Azure Data Factory (ADF)

Azure Data Lake Storage Gen2 (ADLS G2)

Azure Blob Storage

Programming & Query Languages

SQL (Basic & Advanced)

T‑SQL

Stored Procedures

Python

PySpark / Spark SQL

Data Engineering & Architecture

ETL / ELT Fundamentals

Modern Data Platform Concepts

Data Warehousing & Lakehouse Architecture

Data Modeling Fundamentals (Star / Snowflake schemas)

Incremental loads, CDC, and watermarking techniques

Monitoring & Governance

Pipeline monitoring & performance optimization

Azure Monitoring / Fabric monitoring tools

Microsoft Purview (Data lineage & governance – good to have)

Good to Have

Experience with streaming or near real‑time processing

Exposure to Databricks in hybrid Azure–Fabric environments

CI/CD for data pipelines using Git / Azure DevOps

Experience supporting enterprise‑scale analytics platforms

Soft Skills

Strong problem‑solving and analytical mindset

Excellent communication and stakeholder management skills

Ability to work independently in fast‑paced delivery environments

Ownership mindset with focus on quality and reliability

Why Join Us

Work on cutting‑edge Azure & Microsoft Fabric implementations

Exposure to large‑scale enterprise modernization programs

Growth opportunities across cloud, data platforms, and analytics

Collaborative, high‑impact engineering culture

Skills

ADFADLS G2ADLS Gen2AzureAzure Blob StorageAzure Data FactoryAzure Data Lake Storage Gen2Azure DevOpsAzure FunctionsBlob StorageBronze Silver Gold ArchitectureCDCCI/CDDatabricksData GovernanceData ModelingData PipelinesELTETLGitLakehouseMicrosoft FabricMicrosoft PurviewMonitoringOneLakePerformance OptimizationPower BIPySparkPythonSpark SQLSQLSQL ServerStored ProceduresStreamingT-SQLWarehouseWatermarking