Senior Big Data Engineer

Senior Big Data Engineer

Deegit Inc.

Job description:

Responsibilities

·Participate in the engineering and administration of big data systems.

·Apache Storm/Java development for both data transformation and augmentation.

·Employ best practices to ensure high availability and scalability.

·Install, configure, and maintain big data technologies and systems.

·Maintain documentation and troubleshooting playbooks.

·Respond to and resolve access and performance issues.

·Ability to work non-peak hours when needed.

·Assist in capacity planning.

·Proactively monitor performance.

·Collaborate with engineering teams to optimize data collection.

·Maintain change control and testing processes for all modifications and deployments.

·Conduct research and make recommendations on big data products, services, and standards.

Preferred Qualifications

·Experience with operational analytics and log management using ELK or Splunk.

·Understanding and experience with SQL and MPP databases such as Greenplum.

·Experience with SaltStack or comparable deployment automation tools.

·Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies.

·Experience with Apache Drill, Apache Hbase, Hive, Phoenix, or other Hadoop with SQL technologies.

·Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS)

Qualifications

Basic Requirements

·BS in Computer Science, Information Systems, or related technical degree.

·5 or more years’ experience with data transformation pipelines preferably Storm

·5 or more years’ experience in messaging frameworks preferably Kafka.

·5 or more years’ experience with Java programming.

Additional Information

All your information will be kept confidential according to EEO guidelines.