Data Engineer Specialist

Offer by Addison Professional Financial Search LLC

hadoop

python

About this job

Job type: Full-time



Technologies

hadoop, python



Job description

Responsibilities Work independently on data projects for multiple business functions Implement data flows connecting operational systems, BI systems, and the big data platform Automate manual data flows for repeated use and scalability Develop data-intensive applications with API’s and streaming data pipelines Prepare and transform data into a usable state for analytics Document and maintain source-to-target mappings and data lineage Install, configure, and administer Hadoop ecosystem tools and technologies Operationalize machine learning models Assist data analysts and data scientists with query optimization, performance tuning, and data processing Identify opportunities for data improvements and presents recommendations to management Qualifications 5+ years of functional programming experience in Python 5+ years of distributed computing experience with Spark and/or Map Reduce 3+ years of experience developing SQL-based applications 3+ years of experience developing ETL data pipelines 3+ years of system administration on Linux systems 2+ years of experience with multiple Big Data storage tools such as MapR-DB, HBase, MapR-FS, HDF, Parquet, Avro, and ORC Experience in software development life-cycle including change management and testing

A new version is available REFRESH