。该工作已被公司管理人员暂停申请，不再接受简历投递

Data Engineer

Addo Ai

多个城市，巴基斯坦

Apply Now

发布 Oct 07, 2020 4 查看

This role is to build very large, scalable platforms using cutting edge data technologies, from the ground up and plan to leverage the most recent Big Data technologies, working closely with product engineering teams and data scientists.

Responsibilities:

Design and implement data infrastructure and processing workflows required to support data science, machine learning, BI and reporting
Build robust, efficient and reliable data pipelines consisting of diverse data sources
Design and develop real time streaming and batch processing pipeline solutions
Own the data expertise and data quality for the pipelines
Drive the collection of new data and refinement of existing data sources

Requirements:

A minimum of 2 years of experience in data engineering, ETL or work in SQL
Bachelor's degree in Computer Engineering or a related field
Proficiency in Python, Java, Scala, or any two.
Strong command of SQL
Proficient understanding of distributed computing principles
Experience with schema design and dimensional data modeling
Experience working with either a Map Reduce or a MPP system on any size/scale
Knowledge of NoSQL databases will be a plus
Knowledge of Talend
Good understanding of Lambda / Kappa Architecture, along with its advantages and drawbacks.
Familiarity or experience in Cloudera/Hortonworks
Experience with multiple data technologies and concepts such as Airflow, Kafka, Hadoop, Hive, Spark, MapReduce, SQL, NoSQL, and Columnar databases.