Data Engineer

We are seeking a skilled Data Engineer to join our team in Saudi Arabia on a full-time basis. The successful candidate will be responsible for building, configuring, and optimizing data ingestion, transformation, and consumption workloads on a modern on-premises lakehouse platform. This role requires hands-on experience with Spark development, Informatica integration solutions, and enterprise data platforms. Candidates should have between 5 to 10 years of relevant experience and a strong background in Spark-based development, data lake/lakehouse implementations, and data integration workflows.

Key Responsibilities

- Design and develop scalable data ingestion and transformation pipelines to support enterprise data needs.
- Build, optimize, and maintain Spark-based ETL/ELT workloads using PySpark and/or Scala Spark.
- Develop and manage data models and datasets leveraging Apache Iceberg tables for efficient data storage and querying.
- Implement and support data integration workflows using Informatica to ensure seamless data movement across systems.
- Enable analytics and data consumption through platforms such as Denodo, Starburst, and/or Dremio.
- Develop secure, governed, and scalable data pipelines that comply with enterprise standards and policies.
- Support both batch and near real-time data processing requirements to meet business demands.
- Implement data quality checks, metadata management, and validation controls to maintain data integrity.
- Collaborate closely with Platform Engineers on deployment, integration, and operationalization of data solutions.
- Assist in migration, testing, and production rollout initiatives to ensure smooth transitions and minimal downtime.
- Troubleshoot and optimize data pipeline performance, reliability, and scalability to handle large-scale workloads effectively.

Required Qualifications

- Proven experience with Apache Spark, including PySpark and/or Scala Spark development.
- Strong knowledge of Apache Iceberg for data modeling and dataset management.
- Expertise in Informatica Data Integration tools and techniques.
- Familiarity with MinIO object storage systems.
- Experience with Denodo for data virtualization and analytics enablement.
- Proficient in SQL and data modeling concepts essential for data warehousing.
- Solid understanding of ETL/ELT development processes.
- Experience using Git for version control and collaboration.

Preferred Qualifications and Additional Skills

- Experience with Starburst and/or Dremio platforms is advantageous.
- Knowledge of data governance frameworks and metadata management practices.
- Familiarity with data quality frameworks to ensure reliable data delivery.
- Understanding of CI/CD pipelines tailored for data workflows.
- Programming skills in Python to support automation and data processing tasks.
- Experience with Kafka or other streaming technologies for real-time data processing.
- Strong grasp of lakehouse architecture patterns and enterprise data lake implementations.
- Proven ability to tune and optimize performance for large-scale data workloads.

This role offers an exciting opportunity to work on cutting-edge data engineering projects within a dynamic enterprise environment. The ideal candidate will bring a blend of technical expertise, problem-solving skills, and a collaborative mindset to drive data platform excellence.