Postúlate en Kit Empleo: kitempleo.cl/empleo/1cp79j
Please submit your CV in English and indicate your level of English proficiency.
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
Participation is project-based, not permanent employment.
What This Opportunity Involves
You design mathematics problems to challenge a frontier AI model. The problem must have an answer verifiable by code, and the problem has to require a specialized tool like Z3, cvc5, SageMath, Macaulay2, or others. NumPy or SymPy on their own won't cut it. Each problem runs inside a sealed Linux container with the tool pre-installed and a programmatic judge that grades the model's answer.
- Pick an anchor tool and design a problem that hinges on its usage
- Write a Python reference solution, supply input files optionally where needed
- Decide the numerical answer and how close the model needs to get to count as right
- Test the problem against the model in batches of parallel attempts,
tuning the problem difficulty until the agent only succeeds in a small number of attempts
- Once you're happy with the task, and it scores within range, the task goes to a senior reviewer in your subfield. They will provide feedback to ensure task quality is high
Calibration requires patience. You're tuning the problem against batches of parallel runs of the agent, aiming for a pass rate in the 10‑30% band. Reaching that means rewriting, re‑tightening, and watching how the agents act. You'll learn how these agents cut corners, where it stalls, where it converges. This time compounds in two directions. You come out of each task with deeper command of the anchor tool itself, and also get a hands‑on working intuition for how a frontier model navigates complex scientific problems.
What We Look For
- Degree in Mathematics (Pure or Applied) or related field;
- 2+ years of research, applied, or teaching experience;
- Python proficiency for writing reference solutions;
-
Postúlate en Kit Empleo: kitempleo.cl/empleo/1cp79j