Gramian Consultancy, a boutique firm specializing in IT professional services and engineering talent solutions, is seeking a physics-focused professional to join an advanced AI research initiative. This role involves designing and implementing rigorous scientific coding tasks that combine physics expertise, Python-based scientific computing, and AI evaluation. The successful candidate will develop complex physics problems, create verified Python solutions, and build tests to assess the accuracy of AI-generated outputs. The position emphasizes scientific correctness, determinism, and high-quality evaluation across structured STEM training tasks, contributing to cutting-edge AI research.
Key Responsibilities
- Design scientific coding tasks featuring one main physics problem accompanied by at least three logically connected sub-problems.
- Implement verified golden solutions in Python with comprehensive unit test coverage.
- Develop discriminative test cases to effectively differentiate correct from incorrect AI-generated outputs.
- Conduct quality control checks using the Turing Central Task Platform (CTP), including Tier 1 structural checks and Tier 2 quality rubrics.
- Revise scientific tasks and solutions based on quality control feedback.
- Optimize tasks to meet Pass@K evaluation criteria across multiple large language model (LLM) judges such as GPT, Gemini, and Nemotron.
- Validate scientific correctness, well-posedness, determinism, and expected outputs for all tasks.
- Maintain high first-submission quality to minimize rework.
- Participate actively in reviews, feedback sessions, and project standups during required overlap hours.
Required Qualifications
- Master’s degree or PhD in Physics.
- Strong proficiency in Python programming, particularly in scientific computing.
- Experience with scientific Python libraries such as NumPy, SciPy, SymPy, or other relevant domain-specific tools.
- Ability to formulate rigorous physics problems with clear constraints and expected outputs.
- Proven experience implementing scientific solutions with unit tests and deterministic outputs.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with LLM evaluation frameworks or coding benchmarks.
- Published research or academic project experience in physics or related STEM disciplines.
Contract Details
This is a freelance/contractor role with a short-term commitment of 8 weeks. The expected workload is 40 hours per week, including 4 hours overlapping with Pacific Standard Time (PST) working hours. The position is fully remote and open to candidates based in Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Kenya, Nigeria, Pakistan, Turkey, or Vietnam.
This opportunity offers the chance to contribute to a pioneering AI research project that blends physics and advanced computational methods, providing valuable experience in scientific AI evaluation and coding task development within a collaborative and innovative environment.