Lead Confluent Kafka Engineer

We are seeking a highly skilled Lead Confluent Kafka Engineer to oversee the deployment, configuration, administration, and production operations of Apache Kafka and the Confluent Platform. The ideal candidate will have extensive hands-on experience with Kafka, Java, Spring Boot, Linux, security, troubleshooting, and data streaming. This role is critical for maintaining reliable, secure, and highly available Kafka environments within a dynamic onsite setting in Islamabad. Candidates should possess between 5 to 10 years of relevant experience and hold at least a bachelor’s degree in a related field.

Key Responsibilities

- Deploy, configure, and maintain Apache Kafka and Confluent Platform across development, UAT, and production environments.
- Manage Kafka cluster provisioning, broker configuration, topic creation, partitioning, replication, retention policies, and capacity planning.
- Execute Kafka upgrades, patches, rolling restarts, configuration changes, and cluster expansions with minimal disruption to production services.
- Support deployment and management of Kafka components such as Schema Registry, Kafka Connect, Control Center, and other supporting services.
- Troubleshoot complex production issues related to brokers, partitions, ISR, replication, consumer lag, throughput, latency, storage, and connectivity.
- Administer Kafka security configurations including TLS/mTLS, SASL, ACLs, certificates, keystores, and truststores to ensure secure data streaming environments.
- Support Java and Spring Boot Kafka application deployments, including producer and consumer configuration and integration troubleshooting.
- Collaborate with application, DevOps, infrastructure, and security teams to onboard new Kafka use cases and facilitate environment changes.
- Assist with CI/CD pipelines, deployment automation, configuration management, and release activities for Kafka and Kafka-based applications.
- Conduct health checks, capacity planning, performance tuning, incident management, and root-cause analysis to maintain optimal Kafka operations.
- Maintain comprehensive operational documentation, deployment procedures, configuration standards, and recovery/runbook documentation.
- Support high availability, disaster recovery, backup/restore processes, and multi-cluster replication requirements where applicable.

Required Qualifications

- Proven hands-on experience with Apache Kafka and Confluent Platform in production environments.
- Deep understanding of Kafka architecture, including brokers, partitions, replication, ISR, consumer groups, offsets, retention, and delivery semantics.
- Expertise in Kafka deployment, configuration, upgrades, patching, and operational maintenance.
- Strong proficiency in Java and Spring Boot for Kafka producer and consumer integrations.
- Solid Linux, networking, JVM, troubleshooting, and production support skills.
- Experience with Kafka security mechanisms such as TLS/mTLS, SASL, ACLs, and certificate management.
- Familiarity with Confluent Kafka and data streaming technologies.

Preferred Qualifications and Additional Skills

- Experience with container orchestration platforms such as Kubernetes, OpenShift, Anthos, and Helm for containerized Kafka deployments.
- Knowledge of Kafka Connect, Schema Registry, and Confluent Control Center.
- Familiarity with CI/CD tools like Jenkins, GitLab CI, Azure DevOps, or equivalent platforms.
- Monitoring expertise using Splunk, AppDynamics, Prometheus, Grafana, or Confluent monitoring tools.
- Experience with Kafka disaster recovery, multi-cluster architectures, replication, and migration activities.
- Background supporting Kafka in mission-critical environments such as banking or telecom sectors.

This role offers the opportunity to lead Kafka operations in a challenging and fast-paced environment, contributing to the stability and scalability of critical data streaming infrastructure.