Senior Data Engineer Python
AI Summary
Data Engineer responsible for designing and implementing data pipelines, ETL processes, and data quality measures to enable data-driven decision making for clients.
About this role
馃憢 Greetings, future tech savant:
Embark on an exciting journey into the realm of software development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries. If your passion for data analytics solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! 馃殌馃捇
Key Responsibilities:
Understanding the business requirements
Writing code and implementing the proposed solutions
Creating data pipelines, versioning and change management
Manage the complexity inherent in versioned data pipelines
Building and maintaining ETL processes
Logging and instrumentation of pipelines and services
Ensuring data quality
Strong debugging skills to analyze and fix a defect in the data processing
Should be a good team player and a great collaborator
Minimum Qualifications:
Demonstrated expertise with a minimum of 3+ years of experience as data engineer or similar role
Advanced SQL skills and experience with relational databases
Strong Python skills with hands-on experience on Panda, NumPy and other data-related libraries
Strong exposure to write ETL / pipeline code
Experience in Data collection from multiple sources and integrating based on business needs
Experience in working with Google Cloud Platform and Amazon Web Services
GCP: Cloud Functions, Cloud Storage, Dataform, etc.
AWS: ECS, Lambda, EC2, S3, etc.
GitHub Actions for CI/CD, workflows
Terraform
Proficiency in the English language.
Additional Experience Desired:
Exposure to AI / ML
Exposure to Looker Studio, Tableau, PowerBI;
Experience in working with:
Looker Enterprise
Google Analytics
Google Tag Manager
Firebase
Salesforce
Hightouch
Datadog
