Jobless Developer
Shyftlabs logo
Shyftlabs

Posted 2 months ago

Open

Data Engineer

NoidaOn-siteFull-time

AI Summary

Designs and builds scalable batch and streaming data pipelines, optimizes ETL/ELT workflows, and manages large analytical databases to power analytics and business-critical insights.

About this role

Position Overview:

We are seeking a skilled Data Engineer to design and build scalable data platforms that power analytics, reporting, and business-critical insights. You will develop high-performance batch and streaming data pipelines, optimize ETL/ELT workflows, and manage large-scale analytical databases while ensuring data reliability and performance. This role requires strong expertise in SQL, distributed data processing, cloud technologies, and event-driven architectures, along with close collaboration with Product, Analytics, and Backend teams to deliver robust data solutions.
At ShyftLabs, we live and breathe data. Since 2020, we’ve been helping Fortune 500 companies unlock growth with cutting-edge digital solutions that transform industries and create measurable business impact. We’re growing fast and we’re looking for passionate problem-solvers who are ready to turn big ideas into real outcomes.

Job Responsibilities:

  • Design and build scalable batch and streaming data pipelines.
  • Develop and optimize ETL/ELT workflows using distributed data processing frameworks.
  • Own and optimize ClickHouse clusters for large-scale analytical workloads.
  • Design efficient data models for reporting and dashboarding use cases.
  • Build and maintain data ingestion pipelines from MongoDB, PostgreSQL, Kafka, APIs, and other data sources.
  • Improve performance of large SQL workloads and analytical queries.
  • Build reliable monitoring, health checks, and data anomaly detection systems.
  • Work closely with Product, Analytics, and Backend teams to deliver reliable reporting and insights.

    • Basic Qualifications:

  • 3+ years of experience in Data Engineering.
  • Strong SQL skills with expertise in query optimization.
  • Experience with ClickHouse or other OLAP databases (BigQuery, Redshift, Snowflake, Druid, Pinot, etc.).
  • Strong knowledge of PostgreSQL.
  • Experience building ETL/ELT pipelines.
  • Proficiency in Java or Python.
  • Experience with Apache Kafka and event-driven architectures.
  • Strong understanding of data modeling and partitioning strategies.
  • Experience working on cloud platforms (AWS/GCP/Azure).
  • Experience with Docker and Kubernetes.
  • Knowledge of monitoring and observability tools.

    • Preferred Qualifications:

  • Experience with Looker or BI platforms.
  • Experience with Apache Spark or Dataproc.
  • Experience with Airflow or workflow orchestration tools.
  • Understanding of advertising technology (DSP, RTB, Attribution, Campaign Reporting).
  • Experience with large-scale analytical systems processing billions of records.

    • Skills

    AirflowApache KafkaApache SparkAWSAzureClickHouseDockerETL/ELTGCPJavaKubernetesPostgreSQLPythonSQL

    Explore related jobs

    More jobs at Shyftlabs

    Similar Airflow jobs

    Jobs in Noida

    Browse these categories

    Airflow JobsAWS JobsAzure JobsDocker JobsGCP Jobs

    Market data for data engineer roles

    All reports →