Lead Confluent Kafka Engineer

We are seeking a highly skilled Lead Confluent Kafka Engineer to manage the deployment, configuration, administration, and production operations of the Apache Kafka and Confluent Platform environments. This role demands extensive hands-on experience with Kafka, Java, Spring Boot, Linux, security, troubleshooting, and data streaming. The successful candidate will ensure the reliability, security, and high availability of Kafka clusters while collaborating closely with cross-functional teams to support Kafka-based applications and infrastructure.

Key Responsibilities

- Deploy, configure, and maintain Apache Kafka and Confluent Platform across development, UAT, and production environments.
- Manage Kafka cluster provisioning, broker configuration, topic creation, partitioning, replication, retention policies, and capacity planning.
- Perform Kafka upgrades, patching, rolling restarts, configuration changes, and cluster expansions with minimal disruption to production.
- Support deployment and maintenance of Kafka components such as Schema Registry, Kafka Connect, Control Center, and related services.
- Troubleshoot production issues involving brokers, partitions, ISR, replication, consumer lag, throughput, latency, storage, and connectivity.
- Administer Kafka security configurations, including TLS/mTLS, SASL, ACLs, certificates, keystores, and truststores.
- Support Java/Spring Boot Kafka application deployments, including producer and consumer configuration and integration troubleshooting.
- Collaborate with application, DevOps, infrastructure, and security teams to onboard new Kafka use cases and manage changes across environments.
- Facilitate CI/CD pipelines, deployment automation, configuration management, and release activities for Kafka and Kafka-based applications.
- Conduct health checks, capacity planning, performance tuning, incident management, and root cause analysis.
- Maintain operational documentation, deployment procedures, configuration standards, and recovery/runbook documentation.
- Support high availability, disaster recovery, backup/restore processes, and multi-cluster replication as required.

Required Qualifications

- 5 to 10 years of hands-on experience with Apache Kafka and Confluent Platform in production environments.
- Deep understanding of Kafka architecture, including brokers, partitions, replication, ISR, consumer groups, offsets, retention policies, and delivery semantics.
- Proven expertise in Kafka deployment, configuration, upgrades, patching, and ongoing operational maintenance.
- Strong Java and Spring Boot skills for Kafka producer and consumer integration.
- Proficient in Linux, networking, JVM tuning, troubleshooting, and production support.
- Experience with Kafka security mechanisms such as TLS/mTLS, SASL, ACLs, and certificate management.
- Solid experience with Confluent Kafka and data streaming technologies.
- Bachelor’s degree in a relevant field is required.

Preferred Qualifications and Additional Skills

- Experience with container orchestration platforms such as Kubernetes, OpenShift, Anthos, and Helm for containerized Kafka deployments.
- Familiarity with Kafka Connect, Schema Registry, and Confluent Control Center.
- Knowledge of CI/CD tools such as Jenkins, GitLab CI, Azure DevOps, or equivalents.
- Monitoring and observability experience using tools like Splunk, AppDynamics, Prometheus, Grafana, or Confluent monitoring solutions.
- Exposure to Kafka disaster recovery, multi-cluster architectures, replication, and migration activities.
- Prior experience supporting Kafka in mission-critical environments such as banking or telecommunications is highly advantageous.

This is a full-time, onsite position based in Islamabad, requiring a proactive engineer capable of leading Kafka operations and ensuring seamless streaming data infrastructure for enterprise applications.