Job description
Position: SwarmBench Task Engineer Reasoning / Math
Type: Short-Term Contract (4 weeks)
Compensation: $15 per hour
Location: Remote
Commitment: 8 hours per day with 4 hours overlap with PST
Role Responsibilities
• Build multi-agent benchmark tasks requiring multi-step mathematical reasoning, proofs, and algorithmic problem-solving
• Design complex problems across domains like numerical analysis, combinatorics, optimization, and statistical inference
• Create precise problem statements with clear notation, definitions, and expected outputs
• Develop verification scripts to validate correctness (numerical tolerance, proof validity, algorithm outputs)
- Design decomposition strategies to break problems into parallel or independent sub-tasks
• Evaluate and improve AI model reasoning through structured benchmark creation
• Work with evaluation frameworks and agentic workflows for reasoning-heavy tasks
Requirements
• Robust experience in mathematics, quantitative research, or computational science (competition math or university-level exposure preferred)
- Proficiency in Python (NumPy, SciPy, SymPy or similar libraries)
- Experience writing mathematical proofs, derivations, or formal reasoning workflows
• Ability to create objective, verifiable problems (not subjective or open-ended)
- Familiarity with AI coding benchmarks (SWE-bench, Terminal-bench, etc.)
- Comfortable with Linux/terminal workflows, Git, and development environments
• Experience with Docker (writing Dockerfiles, building images, debugging containers)
- Strong understanding of numerical methods (tolerance, convergence, error bounds)
- High attention to detail and ability to work independently in a structured environment
Application Process
• Apply / Easy Apply via LinkedIn
• Fill out the application form shared via email
• Shortlisted candidates will complete an ICF, Online Assessment, and Take-Home Assessment.