Evaluation Engineer
New Yorkonsitemid$100K – $200K
Posted today · via Workday
About this role
Role purpose Build and run tests that assess the quality and reliability of the firm’s AI applications. Help maintain evaluation datasets, investigate failures and provide clear evidence of how AI systems perform on legal and business tasks. Key responsibilities Implement automated evaluation cases and supporting scripts using established frameworks and guidance from senior engineers. Prepare and maintain test datasets, source documents and reference examples with legal specialists. Follow requirements for confidentiality, permitted data use and version control. Run evaluations of answer accuracy, completeness, citations, retrieval relevance, instruction following and appropriate escalation. Apply agreed scoring criteria and flag cases requiring expert judgement.…
What we'd score you on
reqspace match rubricFive dimensions, recruiter-grade. Upload your resume and we'll generate a written explanation of where you fit and where the gaps are.
1
Skills match
For this role: python, java, git, jira
2
Level fit
This role is mid-level. We check your trajectory against it.
3
Domain experience
Your work in the role's domain matters more than your years total. We weight recent and direct experience.
4
Recency
A skill you used last quarter weighs more than one from five years ago. We grade on recency, not lifetime.
5
Location fit
This role is based in New York. We weight your proximity and willingness to relocate.
Score yourself on this role.
Free · no card · written explanation included
Skills in this role
Pulled from the job description. These are the keywords we'll weight when scoring your fit.
pythonjavagitjira