Agentic Automation & QA Engineer

We are seeking an Agentic Automation & QA Engineer to serve as a vital link between client business requirements, quality assurance, conversational AI testing, and agentic AI automation. In this client-facing role, you will engage directly with clients to understand their processes, pain points, and desired outcomes. Your responsibility will be to translate these insights into functional workflows, configure AI agents and automated processes using modern low-code/no-code platforms, and ensure the quality and reliability of delivered solutions. This position focuses on requirements analysis, workflow orchestration, chatbot and AI testing, QA automation, and client collaboration rather than software development or AI model building.

Key Responsibilities

Client Coordination & Requirements
- Collaborate with clients to gather detailed business requirements, workflows, pain points, and expected results.
- Translate client inputs into process flows, functional requirements, user journeys, acceptance criteria, and test scenarios.
- Identify opportunities for AI agents, conversational AI, and intelligent automation within client processes.
- Serve as the primary contact for clarifying requirements, conducting solution walkthroughs, providing testing updates, and managing UAT and release readiness.
- Liaise with internal teams including AI, software engineering, DevOps, and project management for custom development or infrastructure needs.

Agentic Workflow Orchestration
- Design and configure agentic workflows using platforms such as n8n, Microsoft Power Automate, Copilot Studio, Make, Zapier, or similar tools.
- Define agent roles, prompts, triggers, actions, decision points, routing logic, integrations, approval steps, human-in-the-loop processes, and escalation paths.
- Integrate AI agents with enterprise applications, APIs, knowledge bases, databases, email, messaging platforms, and other business systems.
- Prototype workflows and present them to clients, incorporating feedback into iterative improvements.

Chatbot & Conversational AI Testing
- Conduct thorough testing of chatbot and conversational AI including conversation flows, intents, responses, context retention, multi-turn dialogues, and fallback scenarios.
- Evaluate RAG-based chatbots and AI assistants for response relevance, factual accuracy, citation correctness, knowledge retrieval, and handling of unavailable information.
- Assess AI responses for accuracy, consistency, hallucinations, ambiguity, inappropriate content, prompt sensitivity, and compliance with business rules and guardrails.
- Test chatbot behavior across positive, negative, edge-case, adversarial, and out-of-scope queries.
- Validate execution of agentic tools and workflows triggered via conversational interfaces.
- Ensure conversation continuity, session handling, user context, role-based behavior, escalation to human agents, and error/fallback management.
- Maintain reusable evaluation datasets, test scenarios, expected responses, and quality benchmarks.

QA, Automation & Delivery
- Perform end-to-end QA of agentic workflows including functional, integration, regression, API, workflow, and exception-path testing.
- Develop and maintain QA automation to enhance regression coverage and reduce manual testing efforts.
- Coordinate UAT with clients, document feedback and defects, and ensure timely resolution.
- Maintain comprehensive documentation covering requirements, workflows, agent configurations, chatbot scenarios, test cases, integrations, defects, and operating procedures.
- Support production validation, monitoring, and continuous improvement of deployed solutions.

Required Qualifications

- 2 to 4 years of experience in software QA, automation, business requirements analysis, or related roles with strong client-facing skills.
- Solid understanding of software QA fundamentals including test planning, test case design, regression, integration testing, and UAT.
- Hands-on experience with API testing using tools such as Postman or equivalents.
- Practical experience in developing and maintaining automated tests for regression coverage.
- Proficiency in defect management using tools like Jira or Azure DevOps.
- Strong written and verbal English communication skills.
- Ability to independently engage with international clients, lead solution walkthroughs, demonstrations, and manage UAT.
- Capability to produce clear requirements, process flows, acceptance criteria, and test documentation.
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, Information Technology, or a related field.

Preferred Qualifications

- Experience testing chatbots, conversational AI, Generative AI, RAG-based applications, or AI assistants.
- Understanding of testing probabilistic AI systems, including evaluation datasets, benchmarks, and response-quality criteria.
- Ability to assess hallucinations, grounding, context handling, guardrail adherence, and adversarial or out-of-scope behavior in AI responses.
- Hands-on exposure to low-code/no-code automation platforms such as n8n, Microsoft Power Automate, Copilot Studio, Make, or Zapier.
- Skilled in configuring workflow elements including triggers, actions, routing logic, approvals, human-in-the-loop steps, and escalation paths.
- Good conceptual knowledge of generative AI, large language models, prompts, retrieval-augmented generation (RAG), knowledge bases, and AI agents.
- Familiarity with agent orchestration, tool/function calling, guardrails, and human-in-the-loop workflows.
- Working knowledge of APIs, JSON, webhooks, authentication, and databases sufficient to configure, test, troubleshoot, and coordinate workflows.

This role is based in Islamabad, Pakistan, offering a competitive salary and a dynamic work environment. Candidates interested in advancing their careers in AI automation and QA with strong client engagement are encouraged to apply.