Staff Engineer - Data Engineering

We are seeking a Staff Engineer in Data Engineering with 5 to 10 years of experience, specializing in the Finance domain. The ideal candidate will have deep expertise in PySpark, demonstrating advanced skills in big data processing and analytics. This role demands proficiency in handling large datasets, optimizing data pipelines, and executing complex data transformations critical to data engineering functions. A strong command of Python is also essential, enabling effective data manipulation, automation, and the implementation of machine learning models.

Key Responsibilities
- Develop, optimize, and maintain scalable data pipelines using PySpark to process large volumes of financial data efficiently.
- Apply Python programming skills to automate workflows, perform data transformations, and support machine learning initiatives within the data engineering framework.
- Design and implement robust data models that ensure data quality, integrity, and performance optimization across various database architectures.
- Manage data warehousing solutions and ETL processes, integrating multiple data sources to support analytics and reporting needs.
- Utilize cloud platforms such as AWS or Google Cloud to build scalable and reliable data storage and processing environments.
- Collaborate with cross-functional teams to align data engineering efforts with business requirements and domain-specific challenges.
- Employ version control systems like Git to maintain code quality and facilitate collaborative development.
- Work with real-time data streaming technologies, including Kafka, to support continuous data ingestion and processing pipelines.

Required Qualifications
- Master of Technology (M.Tech) in Data Science and Bachelor of Technology (B.Tech) in Computer Science and Engineering, preferably with a focus on Data Engineering or Big Data Analytics.
- 5 to 10 years of professional experience in data engineering, particularly within the Finance sector.
- Advanced proficiency in PySpark for big data processing and analytics.
- Strong programming skills in Python, with demonstrated experience in data manipulation, automation, and machine learning model implementation.
- Solid experience in data warehousing, ETL processes, and integrating diverse data sources.
- Practical knowledge of cloud computing platforms, especially AWS or Google Cloud, for scalable data solutions.
- Proven ability to design and optimize data models and database architectures to ensure high performance and data integrity.

Preferred Qualifications and Benefits
- Certifications such as AWS Certified Data Analytics – Specialty and Google Cloud Professional Data Engineer are highly desirable, reflecting a commitment to continuous professional growth.
- Experience with Kafka for real-time data streaming and processing is advantageous.
- Familiarity with version control tools like Git is preferred to support collaborative development and code management.

This position offers the opportunity to work on cutting-edge data engineering projects within the finance domain, leveraging advanced technologies and cloud platforms. Candidates with a strong technical background and a passion for scalable data solutions will find this role both challenging and rewarding.