Senior/Lead Data Engineer
Ho Chi Minh, Ho Chi Minh, VietnamRemoteFull-time
AI Summary
This is a position we're hiring on behalf of our client in the digital banking industry. DescriptionIn this role, you can expect to:Design, develop, test, deploy and monitor data pipelines in Databricks on AWS from a wide variety of data sources.Design, develop, test, deploy and monitor scalable code with PySpark and SQL in Databricks.Identify opportunities to improve internal process through code optimisation and automation.Build data quality dashboards, lineage flows / and or monitoring tools
About this role
This is a position we're hiring on behalf of our client in the digital banking industry.
Description
In this role, you can expect to:
- Design, develop, test, deploy and monitor data pipelines in Databricks on AWS from a wide variety of data sources.
- Design, develop, test, deploy and monitor scalable code with PySpark and SQL in Databricks.
- Identify opportunities to improve internal process through code optimisation and automation.
- Build data quality dashboards, lineage flows / and or monitoring tools to utilize the data pipeline, providing active monitoring and actionable insight into overall data quality and data governance.
- Assist in migrating data from legacy systems onto newly developed solutions.
- Follow and lead best practices on all data security, retention, and privacy policies.
Requirements
- Bachelor’s degree.
- 6+ years of experience, with 3+ years’ experience of building ETL/ELT pipelines.
- Proven competency in solution design, development, implementation, reporting and analysis.
- Proficiency in Apache-Spark, Python and SQL languages.
- Proficiency in working with Text, Delta, Parquet, JSON, CSV, and XML data formats.
- Working knowledge of Spark structured streaming.
- AWS infrastructure experience, specifically working with S3.
- Solid understanding of git-based version control, DevOps, and CI/CD. Experience of working on Atlassian stack a plus.
- Knowledge of common web API frameworks and web services.
- Strong teamwork, relationship, and client management skills, and the ability to influence peers and senior management to accomplish team goals.
- Willingness to embrace modern technology, best practice, and ways of work.
Benefits
- Performance-based bonus and equity participation
- 13th month salary pro-rata
- 15-day annual leave+ 3-day sick leave + 1 birthday leave + 1 Christmas leave
- Meal and parking allowance
- Full benefits and salary rank during probation.
- Insurances as Vietnamese labor law and premium health care for you and your family
- Overseas travel opportunities for training and working related.
- Hybrid work mode, work-life balance 40-hr per week from Mon to Fri.