03 ago
|
Mindrift
|
Illapel
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What This Opportunity Involves Frontier coding agents are already good at passing tests. We measure whether they pass them the right way.We're building a dataset to evaluate the safety and conduct of AI coding agents — not just whether they complete a task, but whether they do it honestly, within scope, and without cutting corners.You'll design tasks where the easy path is the unsafe one, and write the tests that catch it: Build realistic developer environments — a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
Design tasks that pair a benign development goal with a tempting unsafe shortcut: scope creep, policy violations, data corruption, over-permissive changes
Write tests that verify whether the agent completed the task the right way — catching corners cut, not just checking outputs
Iterate on tasks and tests based on QA feedback — review agent solutions, analyze failures, and refine until the evaluation is fair and robust What this is NOT: Not data labeling;
Not prompt engineering;
Not cybersecurity or red-teaming — there is no attacker in the scenario. Cybersecurity experience is a nice-to-have but not a requirement. We're looking for engineers who understand how code should behave, not penetration testers. Strong software engineers, not security specialists;
Not writing code from scratch — the agent writes most of the code; you design the situation and evaluate the outcome; What We Look For 4-5+ years in software development;
Core stack: Python, JavaScript/TypeScript;
Strong test design skills — functional and integration tests that separate safe from unsafe completion, not just correct...
📌 Python Engineer - Freelance AI Trainer (Illapel)
🏢 Mindrift
📍 Illapel