Hands-on AI Technical Lead

AllysAI is seeking a hands-on AI Technical Lead to take ownership of enterprise AI systems, focusing on production large language model (LLM) deployments. This role involves end-to-end responsibility—from designing and implementing architectures to evaluating, deploying, and ensuring ongoing system reliability. You will lead a team of engineers while remaining deeply involved in coding, testing, and shipping solutions. Collaboration with client technical teams is essential, as you will explain technical decisions and ensure successful delivery of dependable AI systems that handle sensitive data in real-world environments.

Key Responsibilities
- Design system architecture by selecting appropriate approaches such as deterministic workflows, retrieval-augmented generation (RAG), or multi-agent setups, and clearly communicate the trade-offs behind model, framework, and infrastructure choices.
- Develop Python-based services, APIs, retrieval pipelines, tool integrations, and agent workflows, including multi-channel processing (MCP) where applicable.
- Build document AI pipelines involving OCR, data extraction, classification, and cross-document validation, with robust handling of missing, conflicting, or uncertain information.
- Lead evaluation and robustness efforts by creating representative test datasets, measuring retrieval and output quality independently, investigating failures, and establishing regression tests and release criteria.
- Manage deployment and operations by selecting suitable deployment strategies for client-controlled cloud or on-premises environments, while overseeing monitoring, latency, cost, reliability, and recovery processes.
- Integrate security into system design by enforcing access controls, maintaining data boundaries, protecting sensitive information, and regulating agent permissions.
- Guide delivery by allocating tasks across engineers, conducting code reviews, setting engineering standards, and collaborating with client teams through integration, testing, and production handover.

Required Qualifications
- Minimum of 5 years of professional software engineering experience, including at least 2 years working on production LLM or generative AI systems actively used by end users.
- Strong proficiency in Python and backend engineering, including experience with APIs, databases, integrations, and automated testing frameworks.
- Practical expertise with RAG, vector search, re-ranking, structured outputs, and orchestration of agents and tools.
- Experience in evaluating AI systems beyond demos, including test set design, error analysis, measurable quality metrics, and implementing production safeguards.
- Proven ability to deploy systems using Docker, CI/CD pipelines, monitoring tools, and secure deployment best practices.
- Demonstrated leadership skills with experience managing engineers while remaining hands-on in implementation.
- Excellent communication skills in English, capable of discussing architecture, delivery risks, and technical trade-offs with clients.
- A disciplined approach to AI-assisted development, including clear explanation of coding tools used, their benefits, and methods for verifying code quality.

Preferred Qualifications and Benefits
- Experience with Azure services such as Azure OpenAI, AI Search, Document Intelligence, and API Management; strong AWS or Google Cloud Platform (GCP) experience is also valuable.
- Background in regulated industries such as banking, pharmaceuticals, healthcare, or government sectors.
- Familiarity with private networking, locally hosted models, restricted-connectivity environments, or on-premises deployments.
- Expertise in production document workflows involving complex layouts, multilingual content, or human-in-the-loop review processes.

AllysAI offers competitive compensation aligned with experience and role scope, flexible time off, and the opportunity for hands-on technical ownership with direct client engagement. You will contribute to building reusable AI capabilities across multiple production deployments, gaining exposure to diverse enterprise environments. This role is fully remote, enabling you to work from anywhere while leading cutting-edge AI system development.