Posted Apr 22, 2022 7 views
- Responsible to expand data pipeline architecture, optimise data flow & data streams.
- Formulate, suggest, and manage data-driven projects aligned with the business requirements.
- Own data pipelines and working across the data lifecycle.
- Understand the Big Data architecture and the perform trouble shoot and deal with development issues.
- Build and lead a team of experienced data engineers and big data. Mentor and guide the professional and technical development team members.
- Efficiently deal with public cloud environments (GCP/AWS/Azure).
- Build & deploy in containers (Docker/K8s).
- Apply queries and data storage (Snowflake, PostgreSQL etc.).
- Work on events streaming frameworks (Spark/Kafka/Beam).
- Build data pipelines, data warehousing and exposure to batch/event-driven data.
- Work on Big Data using production level skills with one or more Big Data platforms, preferably either Cloudera or MapR.
- Access data from disparate systems, as well as clean and enrich existing data through the use of techniques such as fuzzy matching and sources like APIs.
- Monitor and control all phases of development process and analysis, design, Build, testing, and implementation as well as provide user and operational support on applications to business users.
- Slice and dice the data as needed for business reporting by using data analysis skills.
- Cross-validating models to ensure their generalizability.
- Suggest ways in which insights obtained might be used to inform business strategies.
Job Specifications:
- Minimum 5 years’ experience in Data Engineering with a focus on Big Data technologies.
- BS/MS in computer science degree or relevant.
- Experience with Big Data platforms (Cloudera), Kubernetes, and cloud object storage.
- Understanding of process flow, data management, GDPR and data lineage concepts.
- Prior involvement in open-source community, especially related Apache technologies.
- Experience in writing high-performance, enterprise-quality code in python/Java.
- Proficiency in R or Python, where the former is preferred and or other languages.
- Experience with Data Science Frameworks and tools e.g., TensorFlow, Scikit-learn, Weka, knime, SAS etc.
- Familiarity with SQL and NoSQL data storage such as MongoDB and visualization tools.
- Competent in machine learning, deep learning principles and techniques.
- Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event-based messaging.
- Knowledge of data processing frameworks & libraries i.e Pandas/PySpark.
- Experience with designing, building models and deploying pipelines to production using containerized microservices and/or orchestrated batch runs.
- Experienced working in an agile environment with a fast-paced changing requirements.
- Ability to work under pressure and manage teams and deadlines.
- Excellent planning, team building and organizational skills.
Skills
Job Details
Hive Worx (Pvt.) Ltd.
We are HiveWorx! A rapidly Growing Technology company catering to the Data needs of our Customers around the World! HiveWorx is a technology innovation house with proven expertise in niche product development and services. Our dynamic solutions provide our customers with valuable insight into their data, enabling their workforce to make better, informed decisions. HiveWorx’ innovative solutions are powered by strong, scalable & flexible frameworks, governance processes, methodologies and technologies. We aspire to deliver tangible value by building sustainable technology solutions which enable our customers in achieving their goals and objectives.
What is your Competitive Advantage?
Mona Sharif
Senior Team Lead
Konext PVT. LTD
Aqib Irshad
Head of HR, Training & OD
FINCA
Ayesha Shaukat
Social Media Marketing Manager
Ivoke
Babar Ahmed Khan
Territory Sales Manager
Engro Foods
Ali Aziz
CEO
Shopistan
Zohaib Ali
Marketing Manager
Digital Online Devices
Zubair Anjum
Business Excellence Specialist
Nestle Pakistan
Muzammil Baig
Digital Marketing Manager
IT Services Group
Sehrish Azhar
Motivational Speaker & Blogger
Living Upbeat
Usman Habib
Recruitment & Training Executive
ACS Synergy (Pvt) Ltd.
