Jobless Developer
Jobgether logo

Posted today

Data EngineerJobgether

Data Engineer

United StatesRemoteFull-time

AI Summary

Designs, builds and maintains scalable data pipelines and transformation processes using Python, SQL, Databricks, PySpark and dbt, translating business requirements into production-ready data services with CI/CD and cloud-native deployment.

About this role

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer based in the United States.

We are seeking an experienced Data Engineer to design, develop, and scale robust data solutions that support critical business operations and strategic initiatives.
In this role, you will build high-performance data pipelines and reliable transformation processes, ensuring data is accessible, accurate, and ready for production use.
You will leverage modern technologies, including Python, SQL, Databricks, PySpark, and dbt, to develop scalable and efficient data engineering solutions.
Working closely with business stakeholders, you will translate complex requirements into production-ready systems and services.
You will also champion modern engineering practices, including automation, CI/CD, infrastructure-as-code, and cloud-native deployments.
With a strong focus on data quality, system reliability, and continuous improvement, you will contribute to the development of scalable data infrastructure.
This opportunity is ideal for a technically driven professional who enjoys solving complex challenges and working in a collaborative, technology-focused environment.

Accountabilities:

  • Design, build, scale, and maintain robust data engineering solutions that support business requirements and operational needs.
  • Develop, implement, and optimize high-performance data pipelines covering data extraction, loading, transformation, and workflow orchestration.
  • Ensure data infrastructure and pipelines are designed for scalability, reliability, maintainability, and optimal processing performance.
  • Leverage Python, SQL, Databricks, PySpark, and dbt to develop efficient data processing, transformation, and warehousing solutions.
  • Implement modern software engineering practices, including Continuous Integration and Continuous Deployment (CI/CD), infrastructure-as-code, containerization, and cloud-native deployments.
  • Collaborate closely with business stakeholders and cross-functional teams to understand requirements and translate business use cases into production-ready data services and solutions.
  • Take ownership of data engineering initiatives throughout their entire lifecycle, from initial design and development to deployment, monitoring, and ongoing optimization.
  • Establish and maintain rigorous testing, validation, and monitoring frameworks to ensure data accuracy, consistency, quality, and integrity.
  • Develop and manage scalable data workflows using orchestration tools such as Apache Airflow or Dagster.
  • Handle large-scale data engineering projects, including data cleaning, ETL processes, data transformation, and information retrieval.
  • Identify opportunities to improve existing data architectures, optimize processing workflows, and enhance overall system performance.
  • Explore and apply emerging technologies, including large language models, retrieval-augmented generation, and vector databases, where relevant to business and technical objectives.
  • Contribute to the continuous improvement of engineering standards, documentation, development practices, and data infrastructure.
  • Requirements:

  • Bachelor's degree or higher in a STEM-related discipline is required.
  • A degree concentration in Computer Science, Mathematics, Physics, or another engineering-related field is preferred.
  • At least 5 years of professional experience in data engineering or a closely related technical discipline, with a demonstrated record of delivering successful data solutions.
  • Advanced proficiency in Python and SQL, with a strong understanding of data manipulation, processing, querying, and analysis.
  • Hands-on experience with Databricks and PySpark, particularly for large-scale data processing and distributed computing.
  • Proficiency in dbt for data transformation, modeling, and data warehousing.
  • Experience with workflow orchestration platforms such as Apache Airflow or Dagster.
  • Demonstrated ability to design, develop, and implement complex data systems and architectures from the ground up.
  • Strong experience managing large-scale data projects, including data cleaning, ETL development, transformation workflows, and information retrieval.
  • Solid understanding of modern software engineering principles, including CI/CD, infrastructure-as-code, containerization, testing, and cloud-native deployment practices.
  • Familiarity with large language models (LLMs), particularly prompt engineering, retrieval-augmented generation (RAG), and vector databases, is considered an advantage.
  • Knowledge of fundamental machine learning concepts, feature engineering, and knowledge graphs is a plus.
  • Strong problem-solving and analytical skills, with the ability to troubleshoot complex technical challenges and develop scalable solutions.
  • Excellent written and verbal communication skills, with the ability to collaborate effectively with technical teams and business stakeholders.
  • Ability to work independently, manage priorities, and take ownership of projects in a collaborative and evolving environment.
  • Benefits:

  • Fully remote working arrangement.
  • Opportunity to work on complex, high-impact data engineering projects.
  • Exposure to modern data technologies, including Python, SQL, Databricks, PySpark, and dbt.
  • Opportunities to work with emerging technologies such as LLMs, RAG, vector databases, and machine learning.
  • Collaborative environment focused on innovation, technical excellence, and continuous improvement.
  • Opportunities for professional development and growth within a technology-driven organization.
  • Skills

    Apache AirflowCI/CDCloud-native DeploymentContainerizationDagsterDatabricksData WarehousingDbtETLInfrastructure-as-codeLarge Language ModelsPySparkPythonRetrieval-augmented GenerationSQLVector Databases

    Explore related jobs

    Browse these categories

    Market data for data engineer roles

    All reports →