Bigdata Engineer

Position Details:

Job Title: Bigdata Engineer

Location: Tampa, FL

Duration: 12+ Months Contract to hire

Job Responsibilities:

Principal Responsibilities

· Design interfaces to the data warehouses/data storages and machine learning/Big Data

· applications using open source tools such as Scala, Java, Python, Perl and shell scripting.

· Design and create data pipelines to maintain stable dataflow to the machine learning models –

· both in batch mode and near real-time mode.

· Interface with Engineering/Operations/System Admin/Data Scientist teams to ensure data

· pipelines and processes fit within the production framework.

· Ensure that tools and environments adhere to strict security protocols.

· Deploy the machine learning model and serve its outputs as RESTful API calls.

· Understand the business needs in close collaborations with subject matter experts (SMEs)

· and Data Scientists to do efficient feature engineering for machine learning models.

· Maintain the code and libraries in code repository.

· Work with system administration team to proactively resolve issues/install tools and libraries

· on the AWS platform.

· Research and come up with architecture and solutions most appropriate for problems at hand.

· Maintain and improve tools to assist Analytics in ETL, retrospective testing, efficiency,

· repeatability, and R&D.

· Lead by example regarding software best practices, including code style and architecture,

· documentation, source control, and testing.

· Support the Chief Data Scientist/Data Scientists/Big Data Engineers in creating new and novel

· approaches to solve challenging problems using Machine Learning, Big Data and Cloud

· technologies.

· Handle ADHOC requirements to create reports for the end users.

Required Skills

· Strong skills with Apache Spark (Spark SQL) and SCALA with at least 2+ years of experience.

· Understanding of AWS Big Data components and tools.

· Strong Java skills with experience in web services and web development is required.

· Hands on experience with model deployment.

· Hands on experience in application deployment on Docker and/or Kubernetes or other similar technology.

· Linux scripting is a plus.

· Fundamental understanding of AWS cloud components.

· 2+ years of experience in data ingesting, cleansing/processing, storing and querying large datasets

· 2+ years of experience in engineering large-scale data solutions with Java/Tomcat/ SQL/Linux

· Experience working in a data intensive role including the extraction of data (db/web/api/etc.), transformation and loading (ETL)

· Exposure with structured and/or unstructured data contents

· Experience with data cleansing/preparation on Hadoop/Apache Spark Ecosystem – MapReduce/Hive/HBase/Spark SQL

· Experience with distributed streaming tools like Apache KAFKA.

· Experience with multiple file formats (Parquet, Avro, OCR)

· Knowledge in AGILE development cycle.

· Efficient coding skills to enhance the performance/cost savings of the job running on AWS platform.

· Experience in building stable, scalable, and high-speed live streams of data and serving web platforms

· Enthusiastic self-starter with ability to work in a team environment.

· Graduate (MS) or Undergraduate degree in Computer Science/ Engineering/relevant field

Nice to have:

· Strong Software development experience

· Machine Learning model deployment experience

· Ability to write custom Map/Reduce programs to clean/prepare complex data

· Familiarity with Streaming data processing - Experience with distributed real time computation system like Apache STORM/Apache Spark Streaming.

Additional Information

All your information will be kept confidential according to EEO guidelines.