Senior Data Analyst - LLM / Generative AI

We are seeking a Senior Data Analyst with expertise in Large Language Models (LLM) and Generative AI to join a dynamic team based in Karachi, Pakistan. This full-time, fully onsite role involves hands-on work across data analysis, machine learning, LLM applications, Retrieval-Augmented Generation (RAG) systems, experimentation, and data pipeline development. The successful candidate will handle large and complex datasets, create AI-driven analytical solutions, evaluate LLM performance, and translate data insights into production-ready systems. This position requires moving beyond traditional reporting to focus on GenAI, natural language processing (NLP), prompt engineering, embeddings, and applied machine learning.

Key Responsibilities

- Analyze large, complex datasets using Python and SQL to uncover trends, patterns, anomalies, and actionable insights.
- Build and maintain analytical datasets and data pipelines essential for AI/ML and LLM applications.
- Develop and evaluate LLM-powered solutions, including RAG-based systems, conversational AI, document intelligence, and AI agents.
- Design data ingestion, preprocessing, chunking, embedding, retrieval, and evaluation pipelines tailored for LLM applications.
- Conduct thorough LLM evaluation and experimentation focusing on accuracy, relevance, hallucination, citation quality, retrieval performance, and response consistency.
- Work with diverse data types including structured, unstructured, text, document, and potentially multimodal data.
- Apply statistical analysis and machine learning techniques to address business and operational challenges.
- Build predictive, classification, recommendation, and forecasting models as required.
- Perform exploratory data analysis and develop meaningful KPIs and analytical frameworks.
- Design and execute experiments, A/B tests, model comparisons, and performance evaluations.
- Create dashboards and analytical reports that clearly communicate insights to both technical and business stakeholders.
- Utilize APIs and deploy analytical/ML solutions using tools such as FastAPI, Docker, and cloud platforms.
- Collaborate closely with engineering and product teams to transition AI solutions from experimentation to production.
- Monitor data quality, model performance, pipeline reliability, and overall LLM application effectiveness.
- Continuously evaluate emerging LLMs, GenAI frameworks, embedding models, and AI tools to identify practical implementation opportunities.

Required Qualifications

- Minimum of 4 years’ experience in Data Science, Data Analytics, Machine Learning, AI Engineering, or related fields.
- Proficiency in Python and SQL with strong hands-on experience.
- Solid understanding of statistics, data analysis, experimentation, and machine learning principles.
- Practical experience working with LLMs and Generative AI technologies.
- Experience in building or working with RAG systems, embeddings, semantic search, or vector databases.
- Familiarity with LLM evaluation, prompt engineering, model experimentation, or fine-tuning.
- Strong skills in data preprocessing and handling large datasets.
- Experience with ML/AI frameworks such as PyTorch, TensorFlow, Scikit-learn, or Hugging Face.
- Knowledge of frameworks like LangChain, LangGraph, or LlamaIndex is highly desirable.
- Experience developing data/ETL pipelines using tools such as Airflow, PySpark, BigQuery, Dataflow, or AWS is a plus.
- Experience working with APIs and deploying AI/ML applications is preferred.
- Ability to communicate complex technical findings and analytical insights effectively to non-technical stakeholders.

Preferred Qualifications and Additional Skills

- Experience with multimodal AI involving vision and text data.
- Familiarity with multilingual LLM applications.
- Knowledge of AI agents and agent orchestration.
- Understanding of LLM security concerns such as prompt-injection testing, jailbreak detection, or AI observability.
- Experience with MLflow, model monitoring, or MLOps practices.
- Proficiency in dashboard and visualization tools like Power BI, Tableau, or Plotly.
- Experience working with cloud platforms such as AWS or Google Cloud Platform (GCP).

This role offers the opportunity to work at the forefront of AI innovation, contributing to cutting-edge LLM and Generative AI applications in a collaborative and fast-paced environment.