Posted 3 months ago
Data Engineer
AI Summary
A Data Engineer designs and builds cloud-native data platforms, creating scalable and reliable data pipelines using Spark, Python, and SQL, and manages cloud resources and automation.
About this role
About The Dot Collective
We are a new generation consultancy based across UK and EU and founded on the premises of the engineering excellence and empowering people to make an impact.
We work with all modern tech stacks and typically run agile scrum on all our projects.
About you
Are you passionate about data and its transformational powers? Do you like being able to make a huge difference in a limited period of time? We might be just the right place for you.
Your key skills and capabilities:
· Implementing cloud-native data platforms
· Engineering scalable and reliable pipelines
· Good knowledge of distributed computing with Spark
· Understanding of cloud architecture principles and best practices
· Hands-on experience in designing, deploying, and managing cloud resources
· Excellent python and SQL skills
· Agile ways of working
· Experience in cloud automation and orchestration using tools such as CloudFormation or Terraform
· Monitoring and performance tuning of cloud-based applications and services
Nice to haves: (MLOps):
- Model Deployment & Serving – Deploy and manage ML models using MLflow, Azure ML, SageMaker, or similar, ensuring scalability and performance.
- Monitoring & Retraining – Set up model drift detection, performance monitoring, and automated retraining
- ML Pipelines & CI/CD – Automate end-to-end ML workflows
We expect you to have some knowledge about how to architect, design, develop, deploy, and operate a data platform.
Our promise to you
We will always see you as a human being and will do our very best to support your needs and wellbeing – well-designed co-working and collaboration spaces, remote working patterns that work for you, parenting leave, sabbaticals and ability to work on personal projects.
We believe that a geled team is worth its weight in gold – we will do everything we can to avoid breaking well-performing teams. Whilst continuity across every project is not always possible, we thoughtfully assemble high-performing, blended teams with the appropriate levels of experience and expertise to deliver the best outcomes for our clients — and teams you’ll enjoy working with.
We are committed to prioritising the wellbeing of our employees. To fulfill this promise, we provide a comprehensive employee wellbeing program that includes mental health support, flexible working arrangements, wellness activities, and a positive work culture.
We recognise that the world of tech delivery has moved on significantly in the last 15 years and know a thing or two about how to bring projects over the line without experiencing lots of despair and burn-out. In fact, we like to believe that our projects are the opposite of that – they are run smoothly and most of the time are fun to work on.
Skills
Explore related jobs
More jobs at Dot Collective
Similar Apache Spark jobs
- Software Engineer 2 (Hybrid) - Linux/Bash/Apache Airflow/Apache Spark/Docker/Podman/Git/AWSCaptivation Software · Annapolis Junction, MD - Hybrid
- Senior Data Engineer – Apache Spark | Kafka | Flink | Trino | Iceberg | Big Data | Streaming | Data Platform 8–12 YearsCisco Systems, Inc. · Bangalore, India
Software Engineer, Spark PlatformDoorDash USA · San Francisco, CA
Jobs in London
- HUrban Outfitters Merchandising Admin Assistant (12 Month FTC) - Future OpportunitiesHomeoffice Eu Urbn · London, United Kingdom
- PHygienist Laurel Dental CarePepperpointe · London, KY
- OPrincipal| Leadership Advisory | LondonOdgersberndtson · London, United Kingdom
- OPrincipal| Leadership Advisory | LondonOdgersinterim · London, United Kingdom
- MAssociate, Commercial Real Estate Debt (London)Marathonfund · London, United Kingdom
- MVice President, Commercial Real Estate Debt (London)Marathonfund · London, United Kingdom
Browse these categories
Market data for data engineer roles
All reports →- SeriesRole reportsOne role family at a time: how many openings, what changed this week, who is hiring, what it pays.
- SeriesSalary reportsWhat employers publish in job postings, by level and workplace. Not self-reported pay.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.