Lead Confluent Kafka Engineer

We are seeking a highly skilled Lead Confluent Kafka Engineer to oversee the deployment, configuration, administration, and production operations of Apache Kafka and the Confluent Platform. This role demands extensive hands-on experience with Kafka, Java, Spring Boot, Linux, security, troubleshooting, and data streaming. The successful candidate will be responsible for ensuring reliable, secure, and highly available Kafka environments across multiple stages including development, UAT, and production.

Key Responsibilities

- Deploy, configure, and maintain Apache Kafka and Confluent Platform clusters, including brokers, topics, partitions, replication, retention, and capacity management.
- Manage Kafka upgrades, patches, rolling restarts, configuration changes, and cluster expansions with minimal disruption to production systems.
- Support Kafka ecosystem components such as Schema Registry, Kafka Connect, Control Center, and related services.
- Troubleshoot production issues related to brokers, partitions, ISR, replication, consumer lag, throughput, latency, storage, and connectivity.
- Administer Kafka security configurations including TLS/mTLS, SASL, ACLs, certificates, keystores, and truststores.
- Support Java and Spring Boot Kafka application deployments, focusing on producer and consumer configurations and integration troubleshooting.
- Collaborate with application, DevOps, infrastructure, and security teams to onboard new Kafka use cases and manage environment promotions.
- Assist with CI/CD pipelines, deployment automation, configuration management, and release activities for Kafka and Kafka-based applications.
- Conduct health checks, capacity planning, performance tuning, incident management, and root cause analysis.
- Maintain operational documentation, deployment procedures, configuration standards, and recovery/runbook documentation.
- Support high availability, disaster recovery, backup/restore, and multi-cluster replication requirements where applicable.

Required Qualifications

- 5 to 10 years of hands-on experience with Apache Kafka and Confluent Platform in production environments.
- Deep understanding of Kafka architecture, including brokers, partitions, replication, ISR, consumer groups, offsets, retention policies, and delivery semantics.
- Proven experience in Kafka deployment, configuration, upgrades, patching, and operational maintenance.
- Strong proficiency in Java and Spring Boot for Kafka producer and consumer integrations.
- Solid Linux, networking, JVM, troubleshooting, and production support skills.
- Expertise in Kafka security mechanisms such as TLS/mTLS, SASL, ACLs, and certificate management.
- Familiarity with Confluent Kafka components and data streaming technologies.
- Minimum Bachelor’s degree in a relevant field.

Preferred Qualifications and Additional Skills

- Experience with container orchestration platforms including Kubernetes, OpenShift, Anthos, and Helm for containerized Kafka deployments.
- Knowledge of Kafka Connect, Schema Registry, and Confluent Control Center.
- Familiarity with CI/CD tools such as Jenkins, GitLab CI, Azure DevOps, or equivalents.
- Monitoring and observability experience using tools like Splunk, AppDynamics, Prometheus, Grafana, or Confluent monitoring solutions.
- Experience with Kafka disaster recovery, multi-cluster architectures, replication, and migration activities.
- Background in supporting Kafka within mission-critical environments such as banking or telecommunications.

This is a full-time, onsite position based in Islamabad. The ideal candidate will have a strong technical foundation combined with the ability to collaborate effectively across teams to ensure the stability and scalability of Kafka infrastructure.