Senior Data Engineer – Microsoft Fabric

We are seeking a Senior Data Engineer with extensive hands-on experience in Microsoft Fabric to join our team. The successful candidate will be responsible for designing, developing, and optimizing modern data platforms and analytics solutions. This role requires deep expertise in Fabric Data Engineering, Lakehouse architecture, Data Factory, Spark, SQL, Delta Lake, and Power BI. The ideal candidate should be capable of working independently while contributing effectively to large-scale enterprise data engineering projects.

Key Responsibilities:
- Design and develop scalable data pipelines leveraging Microsoft Fabric.
- Build and maintain Lakehouse environments using the Medallion Architecture (Bronze, Silver, Gold) and data transformation frameworks.
- Create ingestion pipelines with Fabric Data Factory or Data Pipelines for both batch and incremental data processing.
- Develop data transformations using Spark, PySpark, and SQL.
- Design and optimize Delta Lake tables and manage large-scale data processing workloads.
- Implement frameworks for data quality, validation, reconciliation, monitoring, and error handling.
- Develop and optimize enterprise data models to support analytics and reporting needs.
- Collaborate closely with Power BI developers and business stakeholders to deliver analytics-ready datasets.
- Optimize Fabric workloads focusing on performance, scalability, reliability, and cost-efficiency.
- Implement CI/CD pipelines and deployment best practices for data engineering workloads.
- Troubleshoot production data pipelines, resolving performance bottlenecks and data quality issues.
- Adhere to engineering standards including version control, documentation, testing, and code reviews.

Required Qualifications:
- Proficient in Microsoft Fabric and its components including Fabric Data Factory/Data Pipelines, Fabric Lakehouse, Fabric Notebooks, and Fabric Data Warehouse.
- Experience with Dataflows Gen2.
- Strong skills in SQL and PySpark/Spark programming.
- Solid understanding of Delta Lake and distributed data processing concepts.
- Hands-on experience with ETL/ELT processes, including incremental and Change Data Capture (CDC) pipelines.
- Expertise in data modeling, especially dimensional modeling techniques.
- Practical experience with Power BI and semantic data models.
- Familiarity with Direct Lake, Import, and DirectQuery modes in Power BI.
- Knowledge of Git, CI/CD methodologies, and deployment automation.
- Strong grasp of data quality frameworks, governance, security, and monitoring best practices.

Preferred Qualifications and Benefits:
- Microsoft Fabric certifications are highly desirable.
- Experience working with Azure Data Services.
- Background in enterprise-scale data migration and modernization initiatives.
- Skills in Power BI performance tuning and DAX optimization.
- Familiarity with APIs, relational databases, cloud platforms, or other enterprise source systems.

We are looking for a candidate with strong analytical and problem-solving abilities who can independently design and implement robust data engineering solutions. Excellent communication and stakeholder management skills are essential, as is the ability to thrive in a fast-paced consulting or project environment. A strong sense of ownership, with a focus on quality, performance, and timely delivery, is critical for success in this role.