Senior Data Engineer (PySpark)

We are seeking a highly skilled Senior Data Engineer with strong expertise in ** PySpark** and ** data modelling** to join our Digital Platforms delivery team. The ideal candidate will design, build, and optimize scalable data pipelines and contribute to enterprise-grade data architecture supporting analytics, reporting, and digital banking initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and distributed data processing frameworks
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets
• Implement robust **data models **(dimensional, relational, and data vault) aligned with business requirements
• Work closely with stakeholders, data analysts, and business teams to translate requirements into technical solutions
• Ensure data quality, governance, and consistency across multiple data sources
• Optimize performance of Spark jobs and handle large datasets efficiently
• Collaborate with DevOps teams for deployment, CI/CD, and monitoring of data pipelines
• Troubleshoot data issues and ensure high availability and reliability of data systems
• Contribute to data platform architecture and best practices within the Data Engineering chapter

Required Skills & Experience

• 5–8+ years of experience in data engineering or related roles
• Strong hands-on experience with PySpark / Apache Spark
• Expertise in **data modelling **(Star Schema, Snowflake, Data Vault)
• Experience with **big data ecosystems **(Hadoop, Hive, Spark, Kafka – preferred)
• Proficiency in Python and SQL
• Experience with **cloud platforms **(AWS / Azure / GCP – any one preferred)
• Strong understanding of ETL frameworks and data warehousing concepts
• Experience working in agile environments

Good to Have

• Experience in banking or financial services domain
• Knowledge of real-time data processing / streaming (Kafka, Spark Streaming)
• Exposure to data governance and security practices
• Familiarity with tools like Airflow, Databricks, or similar platforms