We are looking for a talented Data Engineer with strong expertise in Apache Spark and Scala to join our technology team based in Hyderabad. The successful candidate will have practical experience working with cloud platforms, preferably Microsoft Azure, including services such as Azure Data Factory, ADLS Gen2, and Azure Synapse Analytics. A thorough understanding of ETL/ELT processes and data warehousing principles is crucial to efficiently manage and optimize data workflows within the organization.
Key Responsibilities
- Design, develop, and maintain scalable and efficient data pipelines using Apache Spark and Scala.
- Utilize cloud-based data services, primarily on Microsoft Azure, to implement data ingestion, transformation, and storage solutions.
- Collaborate closely with cross-functional teams to gather data requirements and translate them into detailed technical specifications.
- Optimize ETL/ELT workflows to ensure smooth data flow, high data quality, and system performance.
- Support data warehousing projects by applying best practices in data modeling, architecture, and governance.
- Identify, troubleshoot, and resolve data-related issues to maintain system reliability and operational excellence.
Required Qualifications
- Bachelor of Engineering degree or equivalent technical qualification.
- Proven proficiency in Big Data technologies, with a focus on Apache Spark and Scala programming.
- Hands-on experience with cloud platforms, preferably Microsoft Azure, including Azure Data Factory, ADLS Gen2, and Azure Synapse Analytics.
- Strong knowledge of ETL/ELT processes and data warehousing concepts, enabling effective data pipeline design and management.
Preferred Qualifications and Benefits
Candidates with additional experience in related big data tools and a strong analytical mindset will have an advantage. This role offers the chance to work in a fast-paced, innovative technology environment in Hyderabad, contributing to advanced data engineering projects that drive business value. The position falls under the Technology function and is well-suited for professionals aiming to grow their careers in big data and cloud-based data solutions.
This opportunity provides exposure to cutting-edge technologies and collaborative teamwork, ideal for engineers passionate about data engineering and cloud ecosystems.