
Posted 1 month ago
Senior Data EngineerVivSoft Technologies · Herndon
Senior Data Engineer
HerndonRemoteFull-time$160k–$180k
AI Summary
Leads design and operation of an enterprise AWS lakehouse using Apache Iceberg and Spark, building data pipelines, optimizing query performance, and mentoring junior engineers on a federal data platform program.
About this role
Job Title: Senior Data Engineer
Location: Remote
Position Type: Full-Time
Clearance Required: Secret Clearance
About the company:
At VivSoft, we aim to solve complex federal problems using emerging and open technologies in a collaborative and rewarding environment. VivSoft is a diverse team of strategists, engineers, designers, and creators experienced in building high-performance, effective software, with a focus on impactful organisational design and software delivery dynamics. We build secure Software Factories based on DoD reference designs and NIST Frameworks for Cloud and DevSecOps. These factories deliver AI/ML Applications, Data Science Platforms, Blockchain and Microservices for DoD, Healthcare and Civilian Agencies
Job Summary:
We are seeking a Senior Data Engineer to support a DoW program responsible for building and operating a modern, scalable, and secure data platform. This role will lead the design and optimization of an enterprise lakehouse architecture on AWS, leveraging Apache Iceberg, Apache Spark, and cloud-native technologies to enable advanced analytics, AI/ML initiatives, and operational reporting. The ideal candidate will possess deep expertise in large-scale data platforms, distributed processing, data governance, and cloud infrastructure while providing technical leadership and mentoring across engineering teams.
Key Responsibilities:
Required Skills:
Preferred Skills:
Benefits:
Salary Range: $160K to $180K per Annually
Location: Remote
Position Type: Full-Time
Clearance Required: Secret Clearance
About the company:
At VivSoft, we aim to solve complex federal problems using emerging and open technologies in a collaborative and rewarding environment. VivSoft is a diverse team of strategists, engineers, designers, and creators experienced in building high-performance, effective software, with a focus on impactful organisational design and software delivery dynamics. We build secure Software Factories based on DoD reference designs and NIST Frameworks for Cloud and DevSecOps. These factories deliver AI/ML Applications, Data Science Platforms, Blockchain and Microservices for DoD, Healthcare and Civilian Agencies
Job Summary:
We are seeking a Senior Data Engineer to support a DoW program responsible for building and operating a modern, scalable, and secure data platform. This role will lead the design and optimization of an enterprise lakehouse architecture on AWS, leveraging Apache Iceberg, Apache Spark, and cloud-native technologies to enable advanced analytics, AI/ML initiatives, and operational reporting. The ideal candidate will possess deep expertise in large-scale data platforms, distributed processing, data governance, and cloud infrastructure while providing technical leadership and mentoring across engineering teams.
Key Responsibilities:
- Architect, build, and manage a cloud-based lakehouse environment using Apache Iceberg, AWS S3, and AWS Glue Catalog.
- Develop and optimize Apache Spark data pipelines on EMR and Kubernetes environments.
- Design and implement event-driven data ingestion solutions using S3 events and Amazon SQS.
- Optimize query performance across Athena, Trino, and Spark SQL environments.
- Orchestrate data workflows and ETL pipelines using Apache Airflow.
- Manage infrastructure deployment and automation using Terraform.
- Develop, publish, monitor, and maintain data products and analytical dashboards.
- Implement data quality, governance, lineage, access controls, and cost management best practices.
- Create technical documentation, architectural designs, and operational runbooks.
- Provide technical leadership and mentorship to junior engineering team members.
Required Skills:
- Must possess an active Secret Clearance
- 8+ years of professional experience in Data Engineering or large-scale data platform development.
- Expertise in Apache Spark, including performance tuning, partitioning, memory optimization, and handling data skew.
- Strong proficiency in Python or Scala and advanced SQL development.
- Experience with open table formats such as Apache Iceberg (preferred) or Delta Lake.
- Strong understanding of distributed query engines, including Athena, Trino, and Spark SQL.
- Hands-on experience with AWS services, including S3, Glue, EMR, Athena, EC2, SQS, and event-driven architectures.
- Experience with Apache Airflow for workflow orchestration.
- Proficiency with Terraform and Infrastructure as Code (IaC).
- Experience working with Kubernetes environments.
- Experience developing dashboards and data products using Grafana or similar visualization platforms.
- Strong understanding of data quality, data governance, lineage, and access control frameworks.
- Excellent technical leadership, mentoring, and stakeholder communication skills.
- Ability to translate business, operational, and analytics requirements into scalable data platform solutions.
- Strong collaboration skills with Data Scientists, AI Engineers, Cloud Engineers, and business stakeholders.
- Proven ability to lead technical discussions and mentor junior engineers.
- Excellent written and verbal communication skills.
- Strong attention to detail regarding data quality, governance, lineage, security, and operational reliability.
Preferred Skills:
- Experience operating and maintaining Apache Iceberg tables at enterprise scale.
- Experience with Trino administration and performance optimization.
- Knowledge of dbt or similar data transformation frameworks.
- Experience with Helm and GitOps deployment methodologies.
- Experience evaluating and implementing modern data visualization platforms beyond Grafana.
- Familiarity with data catalog, metadata management, lineage, and governance tools.
- AWS Data Analytics Certification and/or Certified Kubernetes Administrator (CKA).
- Prior DoD, USAF, or Federal Government data platform experience.
- Experience supporting AI/ML, data science, or advanced analytics workloads in cloud environments.
Benefits:
- Comprehensive Medical, Dental, and Vision Plans (Healthcare benefits are 100% employer-paid for employees only)
- Life Insurance
- Paid Time Off (Flexible/Combined PTO, Bereavement Leave, 11 Company Paid Holidays)
- 401K Retirement Plan with employer match
- Professional Development Training Reimbursement
Salary Range: $160K to $180K per Annually
Skills
Amazon AthenaAmazon SQSApache AirflowApache IcebergApache SparkAWS EMRAWS GlueAWS S3Delta LakeGrafanaKubernetesPythonScalaSecret ClearanceSQLTerraformTrino
Explore related jobs
More jobs at VivSoft Technologies
Similar Apache Iceberg jobs
- M
AWS Data Engineer (Senior) (Freelancer)Mactores · Mumbai, Marshall Islands - M
AWS Data Engineer (Senior)Mactores · Mumbai, Marshall Islands - A
Bi Platforms Operations EngineerAdastra Corporation · Toronto, ON - D
Data EngineerDelaware Nation Industries · Dahlgren, Virginia - H
Senior Data EngineerHealthEdge · Bangalore, India - HSenior Data EngineerHealthEdge · Bangalore, India
Jobs in Herndon
- C
Paint TechnicianCrash Champions · Herndon, VA - B
Shuttle Driver: Full time - Embassy Suites by Hilton Dulles AirportB.F. Saul Company Hospitality Group · Herndon, Virginia - G
Part-Time Evening Trash CollectorGreenway Waste & Recycling · Herndon, Virginia - D
ISSM - HQ preferred/Hybrid - Secret - Proving Grounds, etcDark Wolf Solutions · Herndon, Virginia - VProperty Manager (Maker's Rise)Van Metre Companies · Herndon, Virginia
- G
Systems EngineerGrvty · Herndon, Virginia
Browse these categories
Market data for data engineer roles
All reports →- SeriesRole reportsOne role family at a time: how many openings, what changed this week, who is hiring, what it pays.
- SeriesSalary reportsWhat employers publish in job postings, by level and workplace. Not self-reported pay.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.