Turing is developing one of the most rigorous STEM AI training datasets through its SciCode project, which focuses on creating high-quality scientific coding tasks. These tasks are essential for training and evaluating advanced AI models. As a SciCode Trainer, you will play a vital role in cutting-edge AI research by authoring, implementing, and reviewing complex scientific problems across core STEM disciplines, particularly chemistry. This position offers an opportunity to contribute directly to the advancement of AI capabilities by ensuring the accuracy and quality of scientific coding challenges.
Key Responsibilities:
- Develop scientific problem specifications that include one main problem along with at least three logically connected sub-problems, progressively building toward the solution of the main problem.
- Implement verified golden solutions in Python, ensuring complete unit test coverage to guarantee correctness.
- Design discriminative test cases that effectively distinguish between correct and incorrect outputs generated by AI models.
- Conduct quality control validation on the Turing Central Task Platform (CTP), performing Tier 1 structural checks and Tier 2 quality rubric assessments.
- Iterate on tasks based on quality control feedback to meet Pass@K evaluation criteria across multiple large language model judges such as GPT, Gemini, and Nemotron.
- Maintain consistently high output quality with minimal rework, aiming for first-submission approval at Level 1 quality standards.
- Participate actively in synchronization calls, including reviews, feedback sessions, and project standups during overlapping working hours.
Required Qualifications:
- Master’s degree or PhD in Chemistry.
- Strong proficiency in Python programming, particularly with experience in scientific computing.
- Ability to craft rigorous, well-posed scientific problems with clearly defined constraints and expected outputs.
- Exceptional attention to detail to ensure tasks meet strict rubrics related to problem clarity, test case effectiveness, scientific accuracy, and determinism.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with large language model evaluation frameworks or coding benchmarks.
- Experience working with scientific libraries such as NumPy, SciPy, SymPy, or other domain-specific scientific tools.
- Demonstrated research or academic project experience in a STEM field, preferably with published work.
Preferred Qualifications and Offer Details:
- Commitment to a 40-hour workweek with a required 4-hour overlap with Pacific Standard Time (PST).
- Engagement as a contractor or freelancer for an 8-week duration, without medical benefits or paid leave.
This role is ideal for candidates passionate about the intersection of STEM and AI, who are eager to contribute to pioneering AI training datasets while applying their scientific expertise in a dynamic, remote work environment.