
Posted 1 month ago
Senior Data EngineerAnblicks · Hyderabad
Senior Data Engineer
HyderabadOn-siteFull-time
AI Summary
Designs, builds and optimizes scalable Lakehouse data pipelines and platforms using Databricks, PySpark, Python, SQL and AWS services. Also sets engineering standards, tunes performance and supports governance and production reliability.
About this role
Job Role: Senior Data Engineer
Location: Hyderabad, India
Experience: 6–8 Years
Work Mode: Hybrid
About Anblicks
Anblicks is a Great Place to Work® Certified Data & AI company helping enterprises build modern, intelligent, and scalable data platforms. As a Snowflake Elite Services Partner, Databricks Consulting Partner, AWS Advanced Consulting Partner, and Microsoft Azure Gold Partner, we work with leading organizations to solve complex data, cloud, analytics, and AI challenges.
We are looking for an experienced Senior Data Engineer to design and build scalable data platforms and high-performance data solutions using Databricks, PySpark, Python, SQL, and cloud technologies.
Role Summary
As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing modern data engineering solutions and Lakehouse platforms. You will work on large-scale data processing, data pipelines, data modeling, performance optimization, governance, and cloud-native technologies while contributing to technical design and engineering best practices.
Key Responsibilities
- Design and develop scalable, reliable, and high-performance data pipelines using Databricks, Spark, PySpark, and cloud technologies.
- Design and implement complex ETL/ELT workflows using Databricks, PySpark, SQL, and cloud services.
- Establish engineering standards, design patterns, and best practices for data engineering solutions.
- Make technical and architectural recommendations focused on scalability, performance, maintainability, security, and cost optimization.
- Develop reusable frameworks, libraries, and accelerators to improve engineering productivity.
- Design and implement optimized data models for analytics, operational, and AI/ML use cases.
- Identify and resolve performance bottlenecks through Spark optimization, query tuning, partitioning, and resource management.
- Implement and support data quality, monitoring, observability, governance, and security capabilities.
- Follow software engineering best practices including code reviews, automated testing, CI/CD, Infrastructure as Code, and DevSecOps.
- Lead root-cause analysis and resolution of complex production issues.
- Drive improvements in platform reliability, resiliency, scalability, and operational efficiency.
- Contribute to modern DataOps and Data Product practices.
- Collaborate with architects, developers, data scientists, business stakeholders, and cross-functional teams.
Required Skills & Experience
- Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
- 6–8 years of overall Data Engineering experience, including 5+ years of hands-on Databricks experience.
- Strong experience building large-scale data platforms on public cloud environments.
- Expert-level proficiency in Python, PySpark, SQL, and distributed data processing.
- Strong understanding of Lakehouse architecture, data warehousing, and modern data engineering concepts.
- Hands-on experience with Spark optimization, performance tuning, partitioning strategies, and workload management.
- Strong experience with AWS services such as S3, EMR, Glue, Lambda, ECS/EKS, Redshift, and IAM.
- Experience designing scalable data models and data architectures for analytical and operational workloads.
- Strong understanding of CI/CD, automated testing, and software engineering best practices.
- Hands-on experience with Databricks Asset Bundles (DABs).
- Experience implementing security, governance, privacy, and compliance controls within data platforms.
- Ability to drive technical initiatives and contribute to architectural decisions.
- Strong problem-solving, communication, and stakeholder management skills.
- Experience working in Agile, cross-functional, and globally distributed teams.
Good to Have
- Experience with Infrastructure as Code tools such as Terraform or CloudFormation.
- Experience implementing data quality, observability, and metadata management solutions.
- Experience with streaming technologies such as Kafka, Azure Event Hubs, or Amazon Kinesis.
- Experience deploying and operationalizing Machine Learning and Generative AI solutions.
- Strong knowledge of Delta Lake optimization, Unity Catalog, and Databricks governance.
- Experience leading technical design reviews and mentoring engineering teams.
Why Anblicks?
- Work on large-scale Data, Cloud, AI & Analytics projects.
- Opportunity to work with leading cloud and data technologies including Databricks, Snowflake, AWS, Azure, and AI/ML.
- Exposure to enterprise-scale transformation and modernization initiatives.
- Collaborative, technology-driven engineering environment.
Skills
Apache SparkAWSCI/CDDatabricksDatabricks Asset BundlesData GovernanceData ModelingDelta LakeEMRGlueKafkaLambdaPySparkPythonRedshiftS3SQLTerraformUnity Catalog
Explore related jobs
More jobs at Anblicks
Similar AWS jobs
- B
Apache Spark DeveloperBright Vision Technologies · Edison / Middlesex County, New Jersey - E
Apache Spark Java DeveloperElite Workforce Inc. · Brooklyn Park, MN - T
Desarrollador Java – Apache SparkTalycap Global · Bogotá, Colombia - E
JAVA and Scala/ Apache Spark developer.EROS Technologies Inc. · New York, NY - N
Senior Machine Learning Engineer, Accelerated Apache SparkNVIDIA USA · US, CA - D
Sr. Designated Support Engineer, Apache Spark™Databricks · Sao Paulo, Brazil
Jobs in Hyderabad
Browse these categories
Market data for data engineer roles
All reports →- SeriesRole reportsOne role family at a time: how many openings, what changed this week, who is hiring, what it pays.
- SeriesSalary reportsWhat employers publish in job postings, by level and workplace. Not self-reported pay.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.