Principal Data Engineer - II

We are seeking an experienced Data Engineer to design, develop, and maintain scalable ETL/ELT pipelines that ingest, transform, and integrate data from diverse sources. The ideal candidate will implement both real-time data streaming and batch processing solutions to meet various business requirements. This role involves building and optimizing data storage solutions such as data lakes, warehouses, and databases while ensuring systems are scalable, secure, and high-performing. You will collaborate closely with Data Scientists, Analysts, and cross-functional teams to define data needs and deliver robust solutions that support analytics and machine learning initiatives.

Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, and integration from multiple sources.
- Implement real-time streaming and batch processing solutions tailored to business needs.
- Build and optimize data storage architectures, including data lakes, warehouses, and databases.
- Ensure data systems and pipelines are scalable, secure, and performant to support growing data demands.
- Clean, transform, and prepare raw data into formats suitable for analytics and machine learning models.
- Collaborate with Data Scientists and Analysts to understand data requirements and deliver effective solutions.
- Monitor pipeline and system performance, proactively identifying and implementing improvements.
- Troubleshoot and resolve data-related issues promptly to minimize downtime.
- Implement and enforce data quality checks, validation processes, and monitoring mechanisms.
- Work alongside Data Governance teams to ensure compliance with organizational data standards and policies.
- Partner with Data Architects, Analysts, and business teams to gather requirements and deliver data solutions.
- Provide technical expertise and guidance on data engineering best practices.
- Stay current with emerging tools, technologies, and industry trends to continuously enhance data engineering capabilities.
- Evaluate and recommend new technologies to improve data infrastructure and processes.

Required Qualifications
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
- 7 to 8 years of professional experience as a Data Engineer or in a similar role, with a strong background in building and maintaining data pipelines and infrastructure.
- Hands-on experience with big data tools, cloud platforms, and data integration techniques.
- Expertise in SQL and proficiency working with both relational and NoSQL databases.
- Experience with data orchestration tools such as ODI, SSIS, Apache Airflow, dbt, or Prefect.
- Proficiency in programming languages including Python, Java, or Scala.
- Familiarity with data processing frameworks like Apache Spark and Kafka.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities, capable of working effectively in a fast-paced, team-oriented environment.

Preferred Qualifications and Benefits
- Competitive salary package.
- Fuel card allowance.
- Comprehensive health benefits.
- Opportunities for professional development and continuous learning.
- Inclusive and supportive work culture fostering growth and innovation.

This role offers the chance to work with cutting-edge technologies and contribute to building scalable data solutions that drive business success. If you are passionate about data engineering and eager to collaborate with diverse teams, we encourage you to apply.