Jobless Developer
PharmaLink logo

Posted 25 days ago

Senior Data EngineerPharmaLink · Largo

Senior Data Engineer

LargoOn-site

AI Summary

Designs, builds and operates PharmaLink's enterprise data, analytics and AI platform on Databricks, including ETL/ELT pipelines, Unity Catalog governance, dashboards and data applications. Owns solutions from design through production with a hands-on engineering focus.

About this role

Description


JOB DESCRIPTION

JOB TITLE: Senior Data Engineer

DEPARTMENT: 500 – IT

REPORTS TO: Chief of Information Officer

JOB SUMMARY:

The Senior Data Engineer – Databricks is responsible for building and evolving PharmaLink’s enterprise data, analytics, and AI platform. This hands-on role spans the modern Databricks ecosystem, including data engineering, lakehouse architecture, analytics and dashboards, AI, Databricks Apps, and Lakebase. The role requires strong engineering fundamentals, independent problem-solving, technical ownership, and the ability to design, build, and operate reliable, scalable production solutions.

A hands-on builder approach is essential, with a focus on developing solutions rather than simply administering platforms. The ideal candidate can work across data engineering, platform architecture, analytics, AI, and data applications; independently investigate complex problems; and take solutions from initial design through production.

KEY RESPONSIBILITIES:

  • Design, build, and operate enterprise data solutions on Databricks.
  • Develop scalable ETL/ELT pipelines using Python, PySpark, Spark SQL, and Delta Lake.
  • Design curated data models and reusable data products for analytics, applications, and AI.
  • Implement Unity Catalog, security, lineage, access controls, and data-governance capabilities.
  • Build and optimize Databricks SQL Warehouses and AI/BI Dashboards.
  • Develop data and AI applications using Databricks Apps.
  • Design and implement appropriate solutions using Databricks Lakebase.
  • Enable AI/ML and Generative AI use cases using Databricks capabilities.
  • Build automated data-quality, reconciliation, anomaly-detection, and monitoring processes.
  • Integrate data from databases, SaaS platforms, APIs, files, and other enterprise systems.
  • Optimize Spark workloads, SQL queries, pipelines, performance, and platform cost.
  • Establish engineering standards for Git, CI/CD, testing, monitoring, and production deployment.
  • Troubleshoot complex production and data issues and perform root-cause analysis.
  • Partner with Data Governance, BI, application, and business teams to deliver trusted data products.

Requirements

QUALIFICATIONS:

  • 8+ years of experience in data engineering, data platforms, or related engineering roles.
  • 4+ years of hands-on Databricks experience in production environments.
  • Advanced SQL, Python, PySpark, and Apache Spark skills.
  • Strong experience with Delta Lake and Unity Catalog.
  • Strong ETL/ELT, data modeling, and enterprise data architecture experience.
  • Experience with Databricks SQL and AI/BI Dashboards.
  • Familiarity with Databricks capabilities across AI/ML, Generative AI, Databricks Apps, and Lakebase.
  • Strong relational database and data-engineering fundamentals.
  • Experience with Git, CI/CD, automated testing, monitoring, and production support.
  • Experience working with cloud-based enterprise data platforms.
  • Must maintain a positive attitude in a high-performance team environment.
  • Must pass background check and random drug screenings.
  • Must be adaptable and flexible in dealing with a diverse work force

PREFERRED QUALIFICATIONS:

  • Databricks on Microsoft Azure.
  • Azure Data Factory.
  • Microsoft SQL Server.
  • Fivetran or similar CDC/ELT technologies.
  • Power BI.
  • Databricks Workflows.
  • MLflow and AI/model lifecycle capabilities.
  • API and application development experience.
  • Azure DevOps or GitHub-based CI/CD.
  • Databricks professional certification.
  • Experience in healthcare, life sciences, or another regulated industry.

PHYSICAL REQUIREMENTS:

  • Requires light physical activity performing non-strenuous daily activities of an administrative nature (i.e., sustained phone, headset, and keyboard use) as well as periods of prolonged sitting.

Skills

AI/BI DashboardsApache SparkAzure Data FactoryCI/CDDatabricksDatabricks AppsDatabricks SQLDelta LakeGitLakebaseMicrosoft SQL ServerMLflowPower BIPySparkPythonSpark SQLSQLUnity Catalog

Explore related jobs

Browse these categories

Market data for data engineer roles

All reports →