Data Engineer

We are seeking a skilled Data Engineer to join our team in Saudi Arabia on a full-time basis. The successful candidate will be responsible for building, configuring, and optimizing data ingestion, transformation, and consumption workloads on a modern on-premises lakehouse platform. This role requires hands-on experience with Spark development, Informatica integration solutions, and enterprise data platforms. Candidates should have between 5 to 10 years of relevant experience and a strong background in Spark-based development, enterprise data lake/lakehouse implementations, and Informatica-based integration.

Key Responsibilities

- Design and develop scalable data ingestion and transformation pipelines to support enterprise data needs.
- Build, optimize, and maintain Spark-based ETL/ELT workloads using PySpark and/or Scala Spark.
- Develop and manage data models and datasets leveraging Apache Iceberg tables for efficient data storage and querying.
- Implement and support data integration workflows using Informatica to ensure seamless data flow across systems.
- Enable analytics and data consumption through platforms such as Denodo, Starburst, and/or Dremio.
- Develop secure, governed, and scalable data pipelines aligned with enterprise standards and compliance requirements.
- Support both batch and near real-time data processing to meet diverse business needs.
- Implement data quality, metadata management, and validation controls to maintain data integrity.
- Collaborate closely with Platform Engineers on deployment, integration, and operationalization activities to ensure smooth production rollouts.
- Participate in migration, testing, and production rollout initiatives to enhance platform capabilities.
- Troubleshoot and optimize data pipeline performance, reliability, and scalability to maximize efficiency.

Required Qualifications

- Proven expertise in Apache Spark, including PySpark and/or Scala Spark development.
- Hands-on experience with Apache Iceberg for data modeling and management.
- Strong knowledge of Informatica Data Integration tools and workflows.
- Familiarity with MinIO Object Storage solutions.
- Experience with data virtualization tools such as Denodo.
- Proficient in SQL and data modeling techniques.
- Solid understanding of data warehousing concepts and ETL/ELT development processes.
- Experience with Git version control systems for managing code and collaboration.

Preferred Qualifications

- Experience with Starburst and/or Dremio platforms for data analytics and querying.
- Knowledge of data governance frameworks and metadata management best practices.
- Familiarity with data quality frameworks to ensure accuracy and consistency.
- Experience implementing CI/CD pipelines for data workflows.
- Proficiency in Python programming language.
- Exposure to Kafka or other streaming technologies for real-time data processing.
- Understanding of lakehouse architecture patterns and enterprise data lake implementations.
- Skills in performance tuning and optimization of large-scale data workloads.

This role offers the opportunity to work on cutting-edge data engineering projects within a dynamic enterprise environment. The ideal candidate will bring a combination of technical expertise, problem-solving skills, and a collaborative mindset to drive the success of our data platform initiatives.