
Posted 13 days ago
DT - Senior Software EngineerProdapt · Chennai - Guindy
DT - Senior Software Engineer
Chennai - GuindyOn-site
AI Summary
Designs, builds and optimizes scalable data pipelines on Google Cloud Platform using BigQuery, Dataproc and Cloud Composer, and troubleshoots production data issues while collaborating with cross-functional stakeholders.
About this role
Overview
We are looking for a skilled and detail-oriented GCP Data Engineer to join our growing data engineering team. The ideal candidate will have hands-on experience designing, building, and optimizing scalable data pipelines on Google Cloud Platform, along with strong problem-solving and communication skills to collaborate effectively with cross-functional and global stakeholders.
Responsibilities
Key Responsibilities
- Design, develop, and maintain robust, scalable, and efficient data pipelines on Google Cloud Platform (GCP).
- Work extensively with BigQuery (BQ) for data warehousing, query optimization, and large-scale data analysis.
- Build and manage big data processing workflows using Dataproc (Hadoop/Spark ecosystem on GCP).
- Write, optimize, and troubleshoot complex SQL queries for data transformation, validation, and reporting.
- Design, schedule, and monitor data workflows using Cloud Composer (Apache Airflow).
- Perform root-cause analysis and troubleshooting of pipeline failures, data quality issues, and performance bottlenecks in production environments.
- Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions.
- Ensure data governance, security, and compliance standards are followed across all GCP data engineering solutions.
- Participate in code reviews, design discussions, and knowledge-sharing sessions within the team.
- Document technical designs, data flows, and operational runbooks for supportability.
Requirements
Required Technical Skills
| Skill | Proficiency Expected |
|---|---|
| Google Cloud Platform (GCP) | Strong hands-on experience |
| BigQuery (BQ) | Query optimization, partitioning, clustering, cost management |
| Dataproc | Spark/Hadoop job design and management |
| SQL | Advanced query writing, performance tuning |
| Cloud Composer / Apache Airflow | DAG design, scheduling, monitoring |
| Troubleshooting | Strong analytical and debugging skills for pipeline/data issues |
Good to Have (Additional Skills)
- Ab Initio – ETL development and pipeline migration experience.
- DevOps – CI/CD pipelines, version control (Git), containerization (Docker/Kubernetes), infrastructure as code (Terraform) is a plus.
- Prior Data/ETL Migration Experience – Hands-on experience migrating data pipelines, ETL workloads, or on-premise data warehouses to cloud platforms (e.g., on-prem to GCP, or cross-cloud migrations) is a strong value add.
Soft Skills
- Excellent communication skills – ability to clearly articulate technical concepts to both technical and non-technical stakeholders.
- Strong ownership mindset and ability to work independently as well as within a team.
- Good stakeholder management skills, especially in a global/MNC delivery environment.
- Ability to work under pressure and manage multiple priorities in a fast-paced environment.
Qualifications
- Bachelor's/Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- GCP certification (e.g., Professional Data Engineer) is a plus.
Skills
Ab InitioApache AirflowApache SparkBigQueryCI/CDCloud ComposerData PipelinesDataprocDockerGCPGCP Professional Data EngineerGitHadoopKubernetesQuery OptimizationSQLTerraform
Explore related jobs
More jobs at Prodapt
Similar BigQuery jobs
Browse these categories
Market data for software engineer roles
All reports →- Role reportSoftware engineer jobs, Sep 2026: new listings down 9.7%New software engineer listings fell 9.7% to 1,506 this week, the first weekly drop in a month. Prioritize applications while 1,506 fresh roles remain open.
- Salary reportSoftware engineer salary 2026: median midpoint is $190,587The median published range midpoint for US software engineer roles is $190,587, with the middle half from $166,000 to $222,000.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.