Senior Data Engineer

We are seeking a highly skilled Senior Data Engineer to join our Data Engineering team. The ideal candidate will play a critical role in designing, building, and optimizing scalable data pipelines and enabling advanced analytics and machine learning initiatives.

You will work closely with Data Scientists, Analytics Delivery Leads, and cross-functional teams to transform raw data into actionable insights in a fast-paced Agile environment.

Key Responsibilities

• Collaborate with stakeholders to gather and analyze data requirements
• Perform Exploratory Data Analysis (EDA) to understand data patterns and quality
• Design and develop robust, scalable, and high-performance data pipelines
• Ingest, process, and transform large-scale structured and unstructured datasets
• Implement feature engineering techniques to support machine learning models
• Optimize Spark jobs for performance, scalability, and cost efficiency
• Ensure data quality, integrity, and security across pipelines
• Work closely with Data Scientists and Analytics teams to deploy ML pipelines
• Participate in Agile ceremonies and contribute to continuous improvement
• Communicate technical solutions effectively to both technical and non-technical stakeholders

Required Skills & Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
• 8–10 years of experience in Data Engineering and Big Data ecosystems
• Strong programming expertise in Python
• Hands-on experience with **PySpark / Apache Spark **, including performance tuning
• Solid understanding of Hadoop ecosystem
• Advanced proficiency in SQL
• Experience with data pipeline development and ETL frameworks
• Familiarity with Machine Learning pipelines and feature engineering
• Experience with version control systems (Git)
• Strong problem-solving and analytical skills

Good to Have

• Experience with cloud platforms (AWS / Azure / GCP)
• Knowledge of data warehousing solutions
• Exposure to workflow orchestration tools (Airflow, etc.)
• Experience working in banking or financial services domain

Key Competencies

• Strong collaboration and communication skills
• Ability to work in a fast-paced Agile environment
• Ownership mindset and attention to detail
• Stakeholder management and coordination